// ◎ Transformers & Attention
Multi-Query Attention (MQA)
All heads share one key/value head, shrinking the KV cache.
多查询注意力 MQA 2019 Intermediate extended
IN PLAIN WORDS
Instead of per-head keys and values, keep one shared pair. Decoding reads far less memory and is faster, with a small quality cost.
Used in: PaLM, Falcon
TIMELINE · prerequisites → this → what builds on it
01Original paper
Fast Transformer Decoding: One Write-Head is All You Need
Noam Shazeer · 2019
arXiv ↗ ✓ Verified · arXiv API
02Where it sits
Learn the left column first; the right column builds on this one.
Learn first
Multi-Query Attention (MQA)
Then you can learn
Related:Multi-head Latent Attention (MLA)
Left: prerequisites · right: builds on this · dashed: related. Full map →
03Learning resources
Official docs, explainers, courses and reference code. All links checked.
04Where the course teaches it
For AI / data
- Verification: verified, checked 2026-10-21.
- JSON: /api/learn/algorithms.json (id:
mqa)