// Stage 1 · Math & Programming Foundations
PyTorch & Autograd
Tensors, autograd, data loading and the training loop.
about 10 hours2 algorithms0 paperslesson text in progress
01Goals
- Write a standard training loop
- Understand the autograd computation graph
- Run on a GPU / Colab
Do these first: Python, NumPy & Tooling · Calculus & Optimization
02Algorithms to master
Each card has the original paper, prerequisites and resources.
03Courses & videos
- docs.pytorch.org ↗Official docsEN
- github.com ↗Reference codeEN
- Karpathy：Neural Networks: Zero to Hero ↗VideoEN
- Dive into Deep Learning（D2L） ↗BooksEN
04Hands-on: project, code & exercises
Free, official or author-published pages. Most run in the browser or on a free Colab GPU.
PROJECT
Follow micrograd to write a 100-line autograd engine from scratch, then rewrite the same two-layer network in PyTorch and compare gradients.
- 1
- 2srush/Tensor-Puzzles ↗Exercises
Implement 21 tensor functions with broadcasting only – builds tensor intuition.
- 3
05Self-test
Answer in your head, then open to check.
What does loss.backward() do?
It walks the computation graph backwards and computes gradients for every tensor that needs one.
Why call zero_grad every step?
PyTorch accumulates gradients by default.
Which layers does model.eval() affect?
Layers that behave differently in training and inference, such as Dropout and BatchNorm.
On these routes: Application Developers · Algorithm Researchers · Domain Scientists