Right Screen, Wrong Transition: World Models as Verifiers for GUI Agents

One-line takeaway

For GUI agents, safety is a property of transitions, not screens: a login page after tapping Sign in is fine, after View order it is an attack. LGWM, a decoder-free action-conditioned world model trained on 1.85M real GUI transitions, predicts the next screen's representation so verification is a vector comparison — 0.987 AUC at 17 ms on RSWT-BENCH, on par with the strongest closed VLMs.

GUI agent 的安全取决于“状态转移”而不只是当前屏幕：点“登录”后出现登录页正常，点“查看订单”后出现登录页就是攻击。LGWM 是不带解码器、按动作条件直接预测下一屏表示的世界模型，用 185 万条真实 GUI 转移训练，验证只需向量比较；在 RSWT-BENCH 上免训练打分 AUC 0.987、每次 17 毫秒，与最强闭源 VLM 相当。