// ◇ Algorithms Atlas
Classical Machine Learning
Statistical learning methods that predate deep learning and still dominate tabular and small-data problems.
13 core5 extended1901–2018
TIMELINE
01Core algorithms
Learn these first. Sorted by year.
Principal Component Analysis (PCA) Reduces dimensions by keeping directions of greatest variance. 1901core Logistic Regression Turns a linear score into a probability with a sigmoid for binary classification. 1958core k-Nearest Neighbors Predicts by majority vote of the k closest training examples. 1967core k-Means Clustering Alternates assigning points and recomputing centers to form k clusters. 1982core Support Vector Machine Finds the decision boundary with the widest margin between classes. 1995core Regularized Regression (Lasso / Ridge) Penalize linear-model weights to curb overfitting and select features. 1996core AdaBoost Chains weak learners, each focusing on the examples the previous ones got wrong. 1997core Naive Bayes Applies Bayes’ rule assuming features are independent given the class. 1997core Random Forest Trains many randomized decision trees and lets them vote. 2001core Gradient Boosting (GBDT) Adds small trees one by one, each fixing the previous ensemble's errors. 2001core XGBoost A highly engineered, scalable gradient-boosted tree system. 2016core Linear Regression Fits a line (hyperplane) between inputs and a numeric output. core Decision Trees (CART) Splits data with a sequence of yes/no questions into pure groups. core
02Extended
Variants and follow-ups worth knowing once the core is clear.
Expectation–Maximization (EM) Alternates inferring hidden variables and updating parameters. 1977ext DBSCAN (Density-Based Clustering) Clusters by local density; finds arbitrarily shaped clusters and flags noise. 1996ext t-SNE Maps high-dimensional data to 2D while keeping neighbors close. 2008ext LightGBM Gradient boosting accelerated by histograms and leaf-wise growth. 2017ext UMAP A faster dimensionality reduction that better preserves global structure. 2018ext
03Course units that teach these
U11 Linear Algebra Vector spaces, matrix multiplication, eigenvalues and SVD — the language of embeddings, attention and LoRA. 15 h2 algorithms0 papers U13 Probability, Statistics & Information Theory Distributions, expectation, Bayes, maximum likelihood, cross-entropy and KL divergence. 12 h3 algorithms0 papers U20 The ML Workflow: Data, Loss & Generalization Train/validation/test splits, overfitting, bias–variance, cross-validation and metrics. 10 h3 algorithms1 paperslesson ready U21 Trees & Ensembles Decision trees, random forests, gradient boosting, XGBoost/LightGBM — the kings of tabular data. 8 h6 algorithms3 papers U22 Linear Models, SVM & Nearest Neighbors From the perceptron to maximum margin and the kernel trick, plus k-nearest neighbours. 6 h5 algorithms2 papers U23 Unsupervised Learning k-means, Gaussian mixtures and EM, PCA, t-SNE/UMAP. 8 h6 algorithms2 papers