// ♞ Reinforcement Learning
Trust Region Policy Optimization (TRPO)
Caps each policy update by a KL “trust region” for monotonic improvement.
信赖域策略优化（TRPO） 2015 Advanced extended
IN PLAIN WORDS
A large policy-gradient step can wreck the policy. TRPO maximizes improvement subject to a cap on KL divergence between old and new policies – a trust region – solved approximately with conjugate gradients. Elegant but complex; PPO approximates the same idea with a clipped objective.
Used in: The direct predecessor of PPO; continuous-control benchmarks
01Original paper
Trust Region Policy Optimization
John Schulman, Sergey Levine, Philipp Moritz, et al. · 2015
arXiv ↗ ✓ Verified · arXiv API
02Where it sits
Learn the left column first; the right column builds on this one.
Learn first
Then you can learn
- Nothing in the atlas builds on it yet
Related:Proximal Policy Optimization (PPO)Actor-Critic & A3C/A2C
Left: prerequisites · right: builds on this · dashed: related. Full map →
03Learning resources
Official docs, explainers, courses and reference code. All links checked.
04Where the course teaches it
For AI / data
- Verification: verified, checked 2026-10-22.
- JSON: /api/learn/algorithms.json (id:
trpo)