// Stage 1 · Math & Programming Foundations
Calculus & Optimization
Derivatives, partial derivatives, the chain rule and gradients — “walking downhill along the gradient”.
about 12 hours2 algorithms1 paperslesson text in progress
01Goals
- Compute partial derivatives and gradients of multivariate functions
- See how the chain rule powers backpropagation
- Understand convex vs non-convex and the role of the learning rate
Do these first: Python, NumPy & Tooling
02Algorithms to master
Each card has the original paper, prerequisites and resources.
03Must-read papers
- A Stochastic Approximation Method ↗1951 ✓ verified
04Courses & videos
- 3Blue1Brown ↗VideoEN
- mml-book.github.io ↗BooksEN
- Distill：Why Momentum Really Works ↗ExplainersEN
05Hands-on: project, code & exercises
Free, official or author-published pages. Most run in the browser or on a free Colab GPU.
PROJECT
Hand-write gradient descent to fit y = ax + b, try three learning rates, plot the loss curves and explain divergence vs convergence.
- 1github.com ↗Reference code
Read and retype the ~100-line autodiff engine, then train a tiny net with it.
- 2Distill：Why Momentum Really Works ↗Explainers
Tune step size and momentum in the interactive figures and see when descent oscillates or diverges.
06Self-test
Answer in your head, then open to check.
Where does the gradient point?
In the direction of fastest increase of the function.
What happens if the learning rate is too large?
Training may oscillate or even diverge.
What does the chain rule do in a neural network?
It breaks the output error down, layer by layer, into a gradient for every parameter.
On these routes: Algorithm Researchers