// ⌕ Algorithms Atlas

RAG & Vector Retrieval

Embeddings, vector indexes, dense retrieval, reranking and GraphRAG — connecting LLMs to knowledge.

10 core3 extended2009–2024

TIMELINE

2009 BM252011 Product Quanti2016 HNSW (Hierarch2017 FAISS (Billion2018 Contrastive Le2019 Sentence Embed2019 Cross-Encoder 2020 Dense Passage 2020 ColBERT (Late 2020 Retrieval-Augm2022 Matryoshka Rep2022 HyDE (Hypothet2024 GraphRAG

01Core algorithms

Learn these first. Sorted by year.

BM25 Classic keyword ranking by term frequency and rarity. 2009core Product Quantization (PQ) Splits vectors into chunks, each coded by a small codebook, for compressed fast search. 2011core HNSW (Hierarchical Navigable Small World) A layered small-world graph for millisecond approximate nearest-neighbor search. 2016core Contrastive Learning (InfoNCE) Learns embeddings by pulling positives together and pushing negatives apart. 2018core Sentence Embeddings (Sentence-BERT) Encodes sentences into vectors compared by cosine similarity. 2019core Cross-Encoder Reranking Feed query and document together into BERT to score relevance and re-rank candidates. 2019core Dense Passage Retrieval (DPR) Dual encoders map questions and passages into one space for retrieval. 2020core ColBERT (Late Interaction) Keeps per-token vectors and matches them late for precise yet fast retrieval. 2020core Retrieval-Augmented Generation (RAG) Retrieve relevant documents, then generate grounded answers. 2020core GraphRAG Builds a knowledge graph and community summaries to answer global questions. 2024core

02Extended

Variants and follow-ups worth knowing once the core is clear.

FAISS (Billion-scale GPU Similarity Search) Meta's library and GPU algorithms for billion-scale vector search. 2017ext Matryoshka Representation Learning Embeddings whose prefixes are themselves usable smaller embeddings. 2022ext HyDE (Hypothetical Document Embeddings) Generate a hypothetical answer, then retrieve real documents with it. 2022ext

03Course units that teach these

U02 Hallucination, Reliability & Limits Why models confidently make things up, and how to check and reduce the risk. 2 h2 algorithms1 papers U40 Tokenization & Embeddings BPE tokenisation, per-token pricing, embeddings and semantic similarity. 6 h6 algorithms2 paperslesson ready U70 Vision Transformers & Self-Supervised Vision ViT, SimCLR, MAE, JEPA and SAM. 8 h12 algorithms4 papers U80 Embeddings, Vector Search & Reranking BM25, sentence embeddings, DPR, ColBERT, HNSW, PQ and hybrid search. 8 h10 algorithms4 papers U81 RAG Engineering & Evaluation Chunking, query rewriting, HyDE, GraphRAG, citations and evaluation metrics. 8 h4 algorithms3 paperslesson ready