Skip to content
English

Switch theme

Discover▾

Not sure where to start? This week’s hottest software, models and stacks in three minutes.

AI Apps▾

AI software people use: from assistants like ChatGPT to tools for coding, video and workflows, picked by purpose.

For Agents▾

Software for AI: tools that let your agent (Claude Code, Codex, Cursor…) browse, read docs, run code and remember.

Models▾

Choose models, estimate cost, self-host: prices, rankings, benchmarks, free tiers and compute.

Solutions▾

A goal or an error message? Ready-made stacks, steps and fixes by scenario.

Learn▾

A systematic course from zero to the frontier, with an algorithm library, must-read papers and tutorials.

Frontier▾

Daily AI news, weekly highlights, the latest papers and how AI is changing every discipline.

// Course · stage 6 of 9

Inference, Deployment & Efficiency

Decoding and the KV cache, quantisation and distillation, inference engines, local deployment, GPUs and distributed training.

Start the first unit → All stages

5 unitsabout 36 hours24 algorithms

01Units in this stage

In order. Each unit has goals, algorithms, must-read papers, a project and a self-test.

01 · U60 Decoding, Sampling & KV Cache Prefill and decode, temperature/top-p, KV cache and the memory-bandwidth bottleneck. 6 h4 algorithms2 paperslesson ready 02 · U61 Quantization, Pruning & Distillation INT8/INT4, GPTQ, AWQ, SmoothQuant, BitNet and knowledge distillation. 8 h7 algorithms4 papers 03 · U62 Serving Engines PagedAttention, continuous batching, prefix caching, speculative decoding, prefill/decode disaggregation and gateway routing. 8 h7 algorithms4 papers 04 · U63 Running LLMs Locally llama.cpp, Ollama, MLX, memory estimates and choosing a model. 4 h3 algorithms0 papers 05 · U64 GPUs, Distributed Training & AI Chips Memory, bandwidth and compute; mixed precision; data/tensor/pipeline parallelism; ZeRO/FSDP; the chip landscape. 10 h6 algorithms4 papers

02Algorithms you will meet

Algorithms Atlas →

Temperature & Nucleus (Top-p) SamplingKV CacheAttention Sinks (StreamingLLM)Grouped-Query Attention (GQA)LLM.int8() & Outlier HandlingGPTQAWQ (Activation-aware Weight Quantization)SmoothQuantSparseGPTBitNet b1.58 (Ternary LLMs)Knowledge DistillationPagedAttention (vLLM)Continuous (Iteration-Level) BatchingRadixAttention (Prefix Caching)Speculative DecodingMedusaEAGLEPrefill–Decode DisaggregationMixed Precision & FP8 TrainingZeRO / FSDP (Sharded Data Parallelism)Tensor Parallelism (Megatron-LM)Pipeline Parallelism (GPipe)Ring AttentionGShard (Expert Parallelism)

Part of these routes: Application Developers · Algorithm Researchers · Product Managers & Leaders

Data updated: AI algorithms & curriculum(UTC+8)