Skip to content
English

Switch theme

Discover▾

Not sure where to start? This week’s hottest software, models and stacks in three minutes.

AI Apps▾

AI software people use: from assistants like ChatGPT to tools for coding, video and workflows, picked by purpose.

For Agents▾

Software for AI: tools that let your agent (Claude Code, Codex, Cursor…) browse, read docs, run code and remember.

Models▾

Choose models, estimate cost, self-host: prices, rankings, benchmarks, free tiers and compute.

Solutions▾

A goal or an error message? Ready-made stacks, steps and fixes by scenario.

Learn▾

A systematic course from zero to the frontier, with an algorithm library, must-read papers and tutorials.

Frontier▾

Daily AI news, weekly highlights, the latest papers and how AI is changing every discipline.

Inference & Deployment

Inference Engines: vLLM, llama.cpp and Friends

PagedAttention, continuous batching and choosing an engine for your hardware.

Advanced 20 min For: Developers ✎ In progress

🖥️Self-hosted computeSelf-hosted software stack: drivers, CUDA, vLLM, SGLang, llama.cpp→

✎ This lesson is still being written. Below is the planned outline; it is not indexed by search engines yet.

What this lesson will cover

  1. PagedAttention and continuous batching
  2. Speculative decoding
  3. GGUF and llama.cpp
  4. Choosing an engine

Software used in this lesson

llama.cpp Local inference & servingb11804 ★ 132k LLM inference in C/C++ Open source Ollama Local inference & servingv0.40.8 ★ 184k Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models. Open source CLIAPI mlx-lm Local inference & serving ★ 7.4k Run LLMs with MLX Open source

Related AI news

Links go to the original publishers.

Part of these paths

② Developer Path: Build AI Apps & Agents

Data updated: Learn AI curriculum · Lesson news(UTC+8)