跳转至

文章背景与核心概要

随着计算机使用智能体（CUAs）在数字生态系统中的迅猛发展，评估智能体是否成功完成给定任务，对于强化学习和数据筛选至关重要。由于人工验证无法实现规模化扩展，目前该领域主要依赖视觉语言模型（VLMs）作为自动化裁判。然而，当前最先进的VLM裁判存在系统的宽松偏见，经常将失败误判为成功。

本文介绍了 OSReward，这是一个旨在评估VLM裁判在跨平台CUA轨迹表现的高质量基准。研究人员开源了包含推理标注轨迹判断的数据集 OS-Shepherd-100K，并训练了 OS-Shepherd（9B和35B） 开源奖励模型，以极低的成本实现了商业级别的可靠性，有效解决了当前开源模型落后而商业模型成本高昂的痛点。

OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward Models

arXiv: 2607.28609 [cs.AI]
Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
Submitted: July 30, 2026; Last revised: August 6, 2026 (v2)
Authors: Qiushi Sun, Kanzhi Cheng, Yian Wang, Bowen Yang, Hang Yan, Liheng Chen, Fangzhi Xu, Zichen Ding, Nuo Chen, Jialin Cao, Xingdong Gong, Zehao Li, Kaiming Jin, Xinfeng Yuan, Zhoumianze Liu, Jingyang Gong, Zhangyue Yin, Jiahui Gao, Zhiyong Wu, Tianbao Xie, Jianbing Zhang, Ben Kao, Lingpeng Kong

arXiv: 2607.28609 [cs.AI]
Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
Submitted: July 30, 2026; Last revised: August 6, 2026 (v2)
Authors: Qiushi Sun, Kanzhi Cheng, Yian Wang, Bowen Yang, Hang Yan, Liheng Chen, Fangzhi Xu, Zichen Ding, Nuo Chen, Jialin Cao, Xingdong Gong, Zehao Li, Kaiming Jin, Xinfeng Yuan, Zhoumianze Liu, Jingyang Gong, Zhangyue Yin, Jiahui Gao, Zhiyong Wu, Tianbao Xie, Jianbing Zhang, Ben Kao, Lingpeng Kong

📌 Executive Summary

As computer-using agents (CUAs) rapidly evolve across the digital landscape, evaluating whether an agent has successfully completed a given task has become paramount for reinforcement learning and data curation. Because human verification does not scale, the field relies on Vision-Language Models (VLMs) as automated judges.

This paper introduces OSReward, a high-quality benchmark designed to evaluate VLM judges on cross-platform CUA trajectories. The findings reveal that current state-of-the-art VLM judges suffer from a systematic leniency bias, often mislabeling failures as successes. To address this, the authors release OS-Shepherd-100K (an open corpus of reasoning-annotated trajectory judgments) and train OSShepherd (9B and 35B) open reward models—achieving commercial-grade reliability at a fraction of the cost.

📌 执行摘要

随着计算机使用智能体（CUAs）在数字生态系统中的迅猛发展，评估智能体是否成功完成给定任务已成为强化学习和数据筛选的重中之重。由于人工验证无法实现规模化扩展，该领域目前主要依赖视觉语言模型（VLMs）充当自动化裁判。

本文介绍了 OSReward，这是一个旨在评估VLM裁判在跨平台CUA轨迹上表现的高质量基准。研究结果表明，当前最先进的VLM裁判存在系统的宽松偏见（leniency bias），经常将失败误判为成功。为了解决这一问题，作者发布了 OS-Shepherd-100K（一个包含推理标注轨迹判断的开源语料库），并训练了 OS-Shepherd（9B和35B） 开源奖励模型——以极低的成本实现了商业级的可靠性。

🧭 Abstract

Computer-using agents (CUAs) are advancing rapidly across the digital world. A CUA trajectory records the agent's actions, states, and reasoning. Verifying whether it fulfilled the task instruction is central to CUA evaluation, data curation, and reinforcement learning.

Neither human-written verifiers nor human annotators can provide such verification at scale, so the field increasingly turns to vision-language models (VLMs) as judges of CUA trajectories. But a fundamental question has long gone unexamined: are these VLM judges reliable enough?

To study this systematically, the researchers introduce OSReward, a realistic, high-quality benchmark that evaluates VLM judges on CUA trajectories. The trajectories come from diverse agent backbones executing human-verified instructions across platforms, and are then rigorously labeled with ground-truth verdicts through multi-stage human annotation. Building on it, the work derives: * OSReward-Hard: A challenge set concentrating genuinely hard cases. * OSReward-Multi: A set for fine-grained efficiency and alignment scoring.

Key Findings

The most comprehensive evaluation of VLM judges to date finds that even state-of-the-art models fall short of an ideal judge, sharing a systematic leniency bias that mislabels failed runs as successes. While the most reliable commercial models are too expensive for large-scale operations, affordable open-source models trail far behind.

Solutions & Contributions

  1. OS-Shepherd-100K: An open corpus of reasoning-annotated trajectory judgments for the CUA community.
  2. OS-Shepherd (9B & 35B): Open reward models that supply low-cost, stable, and reliable reward signals, matching commercial judges at 30–60x lower cost than the frontier.

🧭 摘要

计算机使用智能体（CUAs）在数字世界中正快速发展。CUA轨迹记录了智能体的操作、状态和推理过程。验证其是否完成了任务指令是CUA评估、数据筛选和强化学习的核心。

无论是人工编写的验证程序还是人工标注员都无法大规模提供这种验证，因此该领域越来越多地转向使用视觉语言模型（VLMs）来担任CUA轨迹的裁判。但一个根本性问题长期未得到检验：这些VLM裁判是否足够可靠？

为了系统性地研究这一点，研究人员推出了 OSReward，这是一个真实、高质量的基准，用于在CUA轨迹上评估VLM裁判。这些轨迹来自跨平台执行经人工验证指令的各种智能体骨干网络，并通过多阶段人工标注进行了严格的真实结果（ground-truth）标记。在此基础上，本研究衍生出： * OSReward-Hard：一个专注于真正高难度案例的挑战集。 * OSReward-Multi：一个用于细粒度效率和对齐评分的数据集。

核心发现

迄今为止对VLM裁判最全面的评估发现，即使是最先进的模型也远非理想裁判，它们普遍存在系统性宽松偏见，会将失败的运行误判为成功。虽然最可靠的商业模型对于大规模运营来说过于昂贵，但价格实惠的开源模型却远远落后。

解决方案与贡献

  1. OS-Shepherd-100K：为CUA社区提供的一个包含推理标注的轨迹判断开源语料库。
  2. OS-Shepherd (9B & 35B)：开源奖励模型，提供低成本、稳定且可靠的奖励信号，其性能与商业裁判相当，但成本比前沿模型低30到60倍。

🔗 链接与资源

License: Creative Commons Attribution 4.0 International license icon

许可协议：知识共享署名 4.0 国际许可协议 license icon