Stage 3 · Building with AI
Inference & Deployment
How models generate text fast and cheaply — from KV caches to local apps and gateways.
4 lessons1 ready
- 01 How LLMs Generate Text: Decoding, KV Cache and Quantisation Sampling, temperature, the KV cache, batching and quantisation in one lesson. Intermediate 20 min ✓
- 02 Inference Engines: vLLM, llama.cpp and Friends PagedAttention, continuous batching and choosing an engine for your hardware. Advanced 20 min ✎ In progress
- 03 Running LLMs on Your Own Computer Memory math, quantised models and desktop apps for private, offline AI. Intermediate 15 min ✎ In progress
- 04 Model Gateways, Routing and Cost Control One API for many models: fallbacks, caching, rate limits and spend tracking. Advanced 15 min ✎ In progress
Matching units in the new course
Each unit has goals, algorithms, must-read papers, a project and a self-test.
U60 Decoding, Sampling & KV Cache Prefill and decode, temperature/top-p, KV cache and the memory-bandwidth bottleneck. 6 h4 algorithms2 paperslesson ready U62 Serving Engines PagedAttention, continuous batching, prefix caching, speculative decoding, prefill/decode disaggregation and gateway routing. 8 h7 algorithms4 papers U63 Running LLMs Locally llama.cpp, Ollama, MLX, memory estimates and choosing a model. 4 h3 algorithms0 papers