arXiv 2610.05966 · cs.CL · 2026-10-05
HuatuoGPT-3: RL-Only Domain Adaptation from Base Models
Junying Chen, Xinyuan Xie, Ziniu Li, Wenyuan Gu, Jianquan Li, Xiang Wan, Guangjun Yu, Ruoyu Sun and 2 more
Chinese University of Hong Kong, Shenzhen
In latest · #263 Added on 2026-10-26
One-line takeaway
HuatuoGPT-3 explores RL-only domain adaptation from base models, identifies two failure modes of mixed-policy RL (Gradient Starvation and Teacher-Distribution Anchoring), and proposes One-stage Policy Optimization (OnePO), which treats teacher outputs as transient guidance.
HuatuoGPT-3 探索只用强化学习做领域适配：指出混合策略 RL 的「梯度饥饿」和「教师分布锚定」两种失败模式，提出单阶段策略优化 OnePO，把教师输出当作临时引导。
Read on arXiv PDF Hugging Face discussion Code
Topics & disciplines
More on LLMs
- Jeva: System One Models from First Principles ▲371
- Post-Training Leaves Behavioral Shadows on Unrelated Decisions ▲273
- OmniEdu: Open Foundation Models for Learning and Teaching ▲240
- PAR: Progress-Aligned Recurrence for Length Generalization in State Tracking ▲163
- MemT-1.0: Towards Memory-Native Evolution ▲143
More on Reinforcement learning
- False Frontiers: Diagnosing and Mitigating Co-Cheating in Self-Evolving Search Agents 2026-09-30 · ▲642
- The Teacher Is a Direction, Not a Destination: Extrapolating RL-Induced Representation Residuals in On-Policy Distillation 2026-09-29 · ▲511
- Player-1: Learning to Play from Synthetic Experience 2026-10-14 · ▲347
- Scaling Properties of Same-Family On-Policy Distillation 2026-09-26 · ▲327
- DiffusionART: Efficient, Stable and Scalable Reinforcement of Diffusion Models 2026-10-08 · ▲325
About this data
Metadata from the arXiv API (verified); upvotes from Hugging Face Daily Papers; citations from Semantic Scholar. Last refreshed 2026-10-26.
The takeaway is written by AIMap from the abstract; the abstract itself is not reproduced — read the original.