跳转至

SBCO：面向规划智能体的自监督、验证器基础测试框架优化

文章背景与核心概要

本文介绍了 SBCO（Self-supervised Block Coordinate Optimizer，自监督块坐标优化器），这是一种专为规划智能体设计的、基于验证器的测试框架（harness）优化框架。现有的自改进系统（如 Darwin 和 Huxley 哥德尔机器）主要依赖自指涉的代码修改（即编码智能体修改自身的代码），这种方式仅适用于任务能力与自我修改能力高度契合的领域。

对于具有显式约束的规划任务，SBCO 提供了一种高效得多的替代方案。SBCO 属于闭环、从经验中改进（improve-from-experience）的算法家族，它是“自监督”的而非“自指涉”的。它利用固定的元智能体（meta-agent）和零人工标签，通过近似块坐标上升法学习一组分解的验证器和一个测试策略，直接使用智能体自身的评分反馈来精炼智能体的输出。在两个领域的实验结果表明，SBCO 能够匹敌或超越定制的自修改基线，同时将所需的计算预算减少 4 到 5.5 倍。

SBCO: Self-Supervised, Verifier-Grounded Harness Optimization For Planning Agents

Summary

The paper introduces SBCO (Self-supervised Block Coordinate Optimizer), a novel verifier-grounded harness optimization framework designed for planning agents. While existing self-improving systems (such as the Darwin and Huxley Gödel Machines) rely on self-referential code modification—where a coding agent edits its own code—this approach is limited to domains where task competence aligns tightly with self-modification capabilities.

For planning tasks with explicit constraints, SBCO provides a much more efficient alternative. Operating within the closed-loop, improve-from-experience family, SBCO is self-supervised rather than self-referential. It leverages a fixed meta-agent and zero human labels to learn a decomposed bank of verifiers and a harness policy via approximate block coordinate ascent, directly refining the agent's outputs using its own graded feedback. Experimental results across two domains demonstrate that SBCO matches or outperforms customized self-modifying baselines while reducing the required compute budget by 4 to 5.5 times.

Paper Metadata

Authors

  • Vivek Kulkarni
  • Sudipta Paul
  • Aounon Kumar
  • Nicholas Tzou
  • Srinivas Chappidi

Abstract

Self-improving agents seek to reduce the human engineering effort behind AI systems by enabling them to evolve and self-improve their performance over time. Recently, methods like the Darwin Gödel Machine and the Huxley Gödel Machine have been proposed which enable open-ended, recursive self-improvement through self-reference where a coding agent edits its own code. Such self-referential self-improvement methods require that the competence required to perform the task coincides or aligns well with the competence required for self-modification which is the case for coding tasks. For domains or tasks, which do not satisfy the alignment needed, self-referential self-improvement is not available. In such cases, it is possible to adapt the above algorithms to other tasks by removing the self-referential aspect or introducing explicit self-modification of a meta-agent -- both computationally expensive, relying on population or self-modification search over many candidate agents. For planning tasks with explicit constraints, we propose a far cheaper alternative. We introduce SBCO (Self-supervised Block Coordinate Optimizer), a verifier-grounded harness optimizer in the same closed-loop, improve-from-experience family as the Gödel-machine methods, but self-supervised rather than self-referential. Given an agentic harness, SBCO learns a decomposed bank of verifiers and a harness policy via approximate block coordinate ascent, improving the agent's outputs from its own graded feedback---with a fixed meta-agent and no human labels. Across two domains SBCO matches or exceeds a customized self-modifying baseline while using 4-5.5 times less compute budget.