Deep Learning
Training Deep Nets: Optimisers, Normalisation, Residuals
Adam, LayerNorm, residual connections and learning-rate schedules.
Advanced 20 min For: Researchers ✎ In progress
✎ This lesson is still being written. Below is the planned outline; it is not indexed by search engines yet.
What this lesson will cover
- SGD vs Adam
- Normalisation layers
- Residual connections
- Learning-rate warm-up and decay