arXiv 2610.28679 · cs.SE · 2026-10-19
VerifierBench: Benchmarking the Generation of Executable Verifiers for LLM Agents
Jianing Yin, Ziyi Chen, Hao Wang, Yiyou Sun, Zijian Jin, Dawn Song, Daoan Zhang
机构待核实
在最新榜 #320 本站收录于 2026-10-25
一句话要点
VerifierBench 测 agent 能否为任务写出可执行验证器：222 个任务、4,993 个隐藏测试。19 个模型的平衡准确率仅 10.6 到 62.1，生成的验证器拒错比认对更可靠。
VerifierBench tests whether agents can write executable verifiers (222 tasks, 4,993 hidden tests); across 19 models normalized balanced accuracy ranges 10.6–62.1, with verifiers rejecting invalid solutions more reliably than accepting valid ones.
阅读 arXiv 原文 PDF Hugging Face 讨论 代码仓库 项目页
主题与学科
同主题：智能体
- Raven: The Harness of Harnesses for Composable Agentic Intelligence ▲748
- False Frontiers: Diagnosing and Mitigating Co-Cheating in Self-Evolving Search Agents ▲642
- Omni-IO Skills: Harnessing Your Agent Omni-Native ▲485
- BizAgentBench: Unveiling Hidden Challenges in Deploying Long-Horizon Agents for Real-World Business ▲388
- Player-1: Learning to Play from Synthetic Experience ▲347
同主题：评测与数据
- Spectral-Spatial Interpretation 2026-10-15 · ▲753
- StreamEval: Evaluating Interactive Streaming Video Generation Models 2026-10-20 · ▲649
- Has GPT-6-Astra Achieved Embodied Intelligence? 2026-10-15 · ▲421
- RealCompanion: Benchmarking Human Understanding from Reasoning over Longitudinal Real-World Conversations 2026-10-01 · ▲287
- FDE-Gym: The Last Mile of AI Delivery 2026-10-15 · ▲270
数据说明
元数据来自 arXiv API（已核实）；点赞来自 Hugging Face 每日论文；引用来自 Semantic Scholar。最近刷新 2026-10-26。
要点由本站根据摘要自写，不转载摘要原文，请阅读原文。