Sulba
ReviewSign inAccount
Theme
Editor theme
Light
Dark

Intermediate · Neural networks from scratch

Initialization and normalization

Train a deep network: start its weights as torch.nn.init starts them, so that the signal keeps its size through every layer, then build batch and layer normalization, each matched to PyTorch's outputs and gradients, until the network trains where it stalled.

You build
init.py, then norms.py, then normalization.py, then deep.py
Stages
3 Build, 1 Catch, 1 Ship
Time
About 4 hours

Loading the workspace…