Dual Exam: Controlled Evaluation of Multimodal Agents and World Models

One-line takeaway

Dual Exam evaluates multimodal agents and world models across domains, revealing discrepancies in task performance and state evolution, with coupled success still unverified.

Dual Exam通过固定环境响应和动作干预，分别评估多模态代理与世界模型在多个领域的表现，发现两者在任务执行和状态演化上存在不一致，且耦合协同效果尚待验证。

Key points

Dual Exam introduces a structured evaluation framework to assess multimodal agents and world models independently and in interaction across diverse domains. It reveals performance gaps in action selection and state evolution, with significant failures in preserving task-relevant states during joint operation. The results highlight the need for deeper alignment between agent behaviors and world model dynamics.

• Dual Exam evaluates agents and world models separately across physical, digital, social, and scientific domains.
• GPT 6 Astra leads in agent performance (31.3), while GPT-Image-2 excels in full-coverage world model evaluation (21.7).
• A pilot coupling test shows zero task success, indicating failure to preserve task-relevant state evolution.
• The framework reveals misalignment between agent actions and world model state dynamics, with reciprocal learning unverified.

Written only from the official arXiv abstract by an open-source model, then checked automatically (every number and name must appear in the abstract). The paper is authoritative.