// Stage 2 · Classical Machine Learning
Linear Models, SVM & Nearest Neighbors
From the perceptron to maximum margin and the kernel trick, plus k-nearest neighbours.
about 6 hours5 algorithms2 paperslesson text in progress
01Goals
- Understand margins and support vectors
- Get the intuition behind the kernel trick
- See how kNN relates to vector search
Do these first: The ML Workflow: Data, Loss & Generalization · Linear Algebra
02Algorithms to master
Each card has the original paper, prerequisites and resources.
03Must-read papers
- Support-vector networks ↗1995 ✓ verified
- Nearest neighbor pattern classification ↗1967 ✓ verified
04Courses & videos
- cs229.stanford.edu ↗CoursesEN
- scikit-learn.org ↗Official docsEN
- scikit-learn.org ↗Official docsEN
05Hands-on: project, code & exercises
Free, official or author-published pages. Most run in the browser or on a free Colab GPU.
PROJECT
Classify handwritten digits with SVMs (linear and RBF kernels) and kNN; compare accuracy and prediction speed.
- 1scikit-learn.org ↗Notebooks
Run and tweak the official examples for SVM kernels, kNN and the Lasso path.
- 2
06Self-test
Answer in your head, then open to check.
Which samples define an SVM’s decision boundary?
The support vectors.
What are kNN’s training and prediction costs?
Training is almost free; prediction must compute distances to many samples.
What happens if the RBF kernel’s gamma is too large?
The boundary becomes too wiggly and overfits.