HarnessRL: Co-Evolving Policies with a Just-in-Time Training Harness for Open-Ended RL

One-line takeaway

HarnessRL dynamically evolves training interfaces to address reward failure in open-ended RL, improving performance and reducing training steps.

HarnessRL通过动态演化任务采样、策略引导和评分标准，解决开放域强化学习中奖励失效问题，在多个领域任务上显著提升性能并减少训练步数。

Key points

HarnessRL addresses reward degradation in open-ended RL by co-evolving training interfaces with the policy. It dynamically updates task sampling, rollout guidance, and rubric criteria using within- and cross-task evidence. A just-in-time critic evaluates proposed changes on a frozen policy to ensure effectiveness. Evaluated on Rubric-Mix-10k, it outperforms baselines significantly and reduces training steps.

• Co-evolves task sampling, rollout guidance, and rubric criteria with the policy during training.
• Uses within- and cross-task evidence to diagnose reward failures and target interface updates.
• Outperforms strongest baselines by 2.8–3.1 points on Rubric-Mix-10k across 4B and 9B models.
• Reduces training steps to best checkpoint by nearly half via just-in-time validation.

Written only from the official arXiv abstract by an open-source model, then checked automatically (every number and name must appear in the abstract). The paper is authoritative.