// Stage 1 · Math & Programming Foundations
Linear Algebra
Vector spaces, matrix multiplication, eigenvalues and SVD — the language of embeddings, attention and LoRA.
about 15 hours2 algorithms0 paperslesson text in progress
01Goals
- See matrix multiplication as a linear transformation
- Understand dot products and cosine similarity
- Understand eigendecomposition, SVD and low-rank approximation
Do these first: Python, NumPy & Tooling
02Algorithms to master
Each card has the original paper, prerequisites and resources.
03Courses & videos
- 3Blue1Brown ↗VideoEN
- ocw.mit.edu ↗CoursesEN
- mml-book.github.io ↗BooksEN
04Hands-on: project, code & exercises
Free, official or author-published pages. Most run in the browser or on a free Colab GPU.
PROJECT
Use SVD for a low-rank approximation of a greyscale image, plot rank k against reconstruction error, and explain the link to LoRA.
- 1
- 2
05Self-test
Answer in your head, then open to check.
What does a cosine similarity of 0 mean?
The vectors are orthogonal — unrelated in direction.
What is SVD used for?
Dimensionality reduction, compression, denoising and low-rank approximation.
What is the intuition behind matrix rank?
The number of independent directions the matrix can span.
On these routes: Algorithm Researchers