Intermediate · Neural networks from scratch
Initialization and normalization
Train a deep network: start its weights as torch.nn.init starts them, so that the signal keeps its size through every layer, then build batch and layer normalization, each matched to PyTorch's outputs and gradients, until the network trains where it stalled.
- You build
- init.py, then norms.py, then normalization.py, then deep.py
- Stages
- 3 Build, 1 Catch, 1 Ship
- Time
- About 4 hours
Loading the workspace…
Loading the workspace…