// ♞ Reinforcement Learning
Proximal Policy Optimization (PPO)
A stable, simple policy-gradient method that clips update size.
近端策略优化 PPO 2017 Intermediate core
IN PLAIN WORDS
Optimize ratio × advantage with the ratio clipped to 1±ε to avoid destructive steps. Easy and stable — the default for robotics, games and RLHF.
Used in: RLHF, OpenAI Five, robot control
TIMELINE · prerequisites → this → what builds on it
01Original paper
Proximal Policy Optimization Algorithms
John Schulman, Filip Wolski, Prafulla Dhariwal, et al. · 2017
arXiv ↗ ✓ Verified · arXiv API
02Where it sits
Learn the left column first; the right column builds on this one.
Learn first
Related:Trust Region Policy Optimization (TRPO)Actor-Critic & A3C/A2C
Left: prerequisites · right: builds on this · dashed: related. Full map →
03Learning resources
Official docs, explainers, courses and reference code. All links checked.
Official docs
Explainers
04Where the course teaches it
On learning paths:Post-training & Reasoning Models
For AI / data
- Verification: verified, checked 2026-10-21.
- JSON: /api/learn/algorithms.json (id:
ppo)