// ♞ Reinforcement Learning

DDPG / TD3 (Continuous Control)

Brings DQN-style learning to continuous actions with a deterministic actor and a critic.

DDPG / TD3（连续动作控制） aka DDPG / TD3 / Twin Delayed DDPG 2015 Intermediate core

IN PLAIN WORDS

You cannot enumerate continuous actions like joint angles the way DQN does. DDPG trains an actor that outputs an action directly and a Q-network that scores it and passes back gradients, with replay and target networks. TD3 fixes DDPG’s Q over-estimation with twin critics (take the min), delayed updates and target smoothing.

Used in: Robotics and MuJoCo continuous-control benchmarks

TIMELINE · prerequisites → this → what builds on it

1992 Policy Gradie…2013 Deep Q-Networ…2015 DDPG / TD3 (Conti…

01Original paper

Continuous control with deep reinforcement learning

Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, et al. · 2015

arXiv ↗ ✓ Verified · arXiv API

02Where it sits

Learn the left column first; the right column builds on this one.

Learn first

DDPG / TD3 (Continuous Control)

Then you can learn

Nothing in the atlas builds on it yet

Related:Soft Actor-Critic (SAC)

Left: prerequisites · right: builds on this · dashed: related. Full map →

03Learning resources

Official docs, explainers, courses and reference code. All links checked.

Official docs

Reference code

github.com ↗EN

04Where the course teaches it

U33 Reinforcement Learning Basics MDPs, Q-learning, DQN, policy gradients, PPO and AlphaZero — the prerequisites for post-training and reasoning models. 14 h12 algorithms4 papers

For AI / data

Verification: verified, checked 2026-10-22.
JSON: /api/learn/algorithms.json (id: ddpg-td3)