跳转至

文章背景与核心概要

本文介绍了 HarnessLens，这是一个专为自动化大语言模型智能体（Agent）Harness 演进而设计的预算感知（budget-aware）创新框架。传统的“提出-验证”（propose-and-verify）方法在适配智能体指令、工具和运行时组件时，往往需要在固定的任务集上对每个候选方案进行评分，这种蛮力方法不仅在不相关的行为上浪费了大量的评估 Rollout，还容易让总体得分掩盖关键的性能退化。

为了克服这些局限性，HarnessLens 联合探索了任务空间与用户可配置的组件，直接从执行轨迹中衍生出候选修改方案，并通过可归因证据门控（attributable-evidence gate）在行为相关的任务上对候选方案进行选择性验证。在三个智能体 Harness 和四个基准测试上的评估表明，HarnessLens 在大幅减少评估预算的同时，显著提升了未见过任务（held-out）的泛化性能。这项工作证明了结合显式归因的行为感知验证能够在受限的交互预算下实现更可靠、更高样本效率的 Harness 演进。

Verify Smarter, Evolve Further: Efficient Harness Evolution through Behavior-Aware Verification

📝 Summary

HarnessLens is an innovative, budget-aware framework designed for automated agent harness evolution. Traditional propose-and-verify methods for adapting language-model agent instructions, tools, and runtimes often score every candidate on a fixed task set. This brute-force approach wastes evaluation rollouts on unrelated behaviors and causes aggregate scores to obscure critical performance regressions.

To overcome these limitations, HarnessLens jointly explores the task space and user-configurable components, derives candidate modifications directly from execution trajectories, and selectively verifies candidates using an attributable-evidence gate. Across evaluation on three agent harnesses and four benchmarks, HarnessLens significantly improves held-out performance while requiring substantially fewer evaluation budgets than existing baseline methods.

📌 Metadata

  • arXiv ID: arXiv:2608.27311 [cs.AI]
  • Submitted On: August 27, 2026
  • Primary Subject: Artificial Intelligence (cs.AI)
  • Authors:
  • Jinghan Xu
  • Yikai Zhang
  • Aili Chen
  • Weiyuan Li
  • Jiaqing Liang
  • Deqing Yang
  • Code Repository: GitHub - HarnessLens

🔍 Abstract

Agent harnesses shape how language-model agents use instructions, tools, and runtime components, but adapting these harnesses requires costly verification. Existing propose-and-verify methods typically score every candidate on a fixed task set, wasting rollouts on unrelated behaviors and allowing aggregate scores to obscure specific regressions. We introduce HarnessLens, a budget-aware framework for automated harness evolution. HarnessLens jointly explores the task space and user-configurable components, derives candidate modifications from execution trajectories, and selectively verifies each candidate on behavior-relevant tasks using an attributable-evidence gate. Across three agent harnesses and four benchmarks, HarnessLens improves average held-out performance by 7.6-13.6% while consuming substantially less evaluation budget than competing baselines. These results demonstrate that behavior-aware verification with explicit attribution enables more reliable and sample-efficient harness evolution under constrained interaction budgets.