// ⚖ Alignment & Preference Optimization
Group Relative Policy Optimization (GRPO)
Samples a group of answers per prompt and uses in-group relative scores instead of a critic.
组相对策略优化 GRPO 2024 Advanced core
IN PLAIN WORDS
PPO needs a critic as large as the policy. GRPO samples several answers and uses each answer's reward relative to the group mean as advantage, dropping the critic. DeepSeek-R1 used it for reasoning RL.
Used in: DeepSeekMath, DeepSeek-R1 and many open reasoners; still the default baseline in Oct 2026 papers (e.g. SCOPE 2610.22264, ReMaRk 2610.17771, KLPO 2610.08963)
TIMELINE · prerequisites → this → what builds on it
01Original paper
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
Zhihong Shao, Peiyi Wang, Qihao Zhu, et al. · 2024
arXiv ↗ ✓ Verified · arXiv API
02Where it sits
Learn the left column first; the right column builds on this one.
Then you can learn
Left: prerequisites · right: builds on this · dashed: related. Full map →
03Learning resources
Official docs, explainers, courses and reference code. All links checked.
04Where the course teaches it
On learning paths:Post-training & Reasoning Models
For AI / data
- Verification: verified, checked 2026-10-21.
- JSON: /api/learn/algorithms.json (id:
grpo)