Stage 1 · Foundations
Deep Learning
Neural networks, backpropagation and the building blocks that made large models trainable.
3 lessons1 ready
- 01 Neural Networks and Backpropagation Neurons, layers, activations and how errors flow backwards to update millions of weights. Intermediate 20 min ✓
- 02 Training Deep Nets: Optimisers, Normalisation, Residuals Adam, LayerNorm, residual connections and learning-rate schedules. Advanced 20 min ✎ In progress
- 03 From CNNs and RNNs to Transformers A short history of architectures and why attention replaced recurrence. Intermediate 15 min ✎ In progress
Matching units in the new course
Each unit has goals, algorithms, must-read papers, a project and a self-test.
U30 Neural Networks & Backpropagation From neurons and multilayer perceptrons to backpropagation and activation functions. 10 h4 algorithms2 paperslesson ready U31 Training Deep Nets SGD/momentum/Adam/AdamW/Muon, learning-rate schedules, BatchNorm/LayerNorm/RMSNorm, dropout and residuals. 10 h10 algorithms4 papers U32 CNNs, RNNs and the Birth of Attention Convolutional networks, LSTMs, Seq2Seq, word vectors and Bahdanau attention. 10 h7 algorithms5 papers