Skip to content
English

Switch theme

Discover▾

Not sure where to start? This week’s hottest software, models and stacks in three minutes.

AI Apps▾

AI software people use: from assistants like ChatGPT to tools for coding, video and workflows, picked by purpose.

For Agents▾

Software for AI: tools that let your agent (Claude Code, Codex, Cursor…) browse, read docs, run code and remember.

Models▾

Choose models, estimate cost, self-host: prices, rankings, benchmarks, free tiers and compute.

Solutions▾

A goal or an error message? Ready-made stacks, steps and fixes by scenario.

Learn▾

A systematic course from zero to the frontier, with an algorithm library, must-read papers and tutorials.

Frontier▾

Daily AI news, weekly highlights, the latest papers and how AI is changing every discipline.

// Stage 8 · RAG, Agents & AI Engineering

Embeddings, Vector Search & Reranking

BM25, sentence embeddings, DPR, ColBERT, HNSW, PQ and hybrid search.

What you will learn ↓ Code & exercises ↓

about 8 hours10 algorithms4 paperslesson text in progress

01Goals

Do these first: Tokenization & Embeddings · Unsupervised Learning

02Algorithms to master

Each card has the original paper, prerequisites and resources.

BM25 Classic keyword ranking by term frequency and rarity. 2009core Sentence Embeddings (Sentence-BERT) Encodes sentences into vectors compared by cosine similarity. 2019core Dense Passage Retrieval (DPR) Dual encoders map questions and passages into one space for retrieval. 2020core ColBERT (Late Interaction) Keeps per-token vectors and matches them late for precise yet fast retrieval. 2020core HNSW (Hierarchical Navigable Small World) A layered small-world graph for millisecond approximate nearest-neighbor search. 2016core Product Quantization (PQ) Splits vectors into chunks, each coded by a small codebook, for compressed fast search. 2011core FAISS (Billion-scale GPU Similarity Search) Meta's library and GPU algorithms for billion-scale vector search. 2017ext Matryoshka Representation Learning Embeddings whose prefixes are themselves usable smaller embeddings. 2022ext Contrastive Learning (InfoNCE) Learns embeddings by pulling positives together and pushing negatives apart. 2018core Cross-Encoder Reranking Feed query and document together into BERT to score relevance and re-rank candidates. 2019core

03Must-read papers

04Courses & videos

05Hands-on: project, code & exercises

Free, official or author-published pages. Most run in the browser or on a free Colab GPU.

PROJECT

On the same 10,000 documents, compare BM25, vector and hybrid retrieval by Recall@10 on 50 labelled questions.

  1. 1
    sbert.net ↗Official docs

    Encode documents with Sentence Transformers, then add semantic search and cross-encoder re-ranking.

  2. 2
    facebookresearch/faiss ↗Reference code

    Build Flat, IVF and HNSW indexes in FAISS and compare recall and speed.

  3. 3
    huggingface.co ↗Tools

    Pick embedding models on the MTEB leaderboard by language and task.

06Self-test

Answer in your head, then open to check.

The main cost of HNSW?

High memory use and slow index builds.

Why mix BM25 with vectors?

Exact keyword matching and semantic matching complement each other.

Are re-rankers slower or faster than retrievers?

Slower and more accurate, so they only see a few candidates.

On these routes: Application Developers

Data updated: AI algorithms & curriculum(UTC+8)