跳转至

文章背景与核心概要

自进化的运行时脚手架 (Runtime Harness) 是增强大语言模型智能体 (AI Agent) 执行表现与系统自适应能力的关键基础设施。然而，现有的脚手架进化策略主要依赖单任务实例反馈的迭代搜索，不仅因反复执行智能体和修改代码带来高昂的时间成本，更容易因缺乏针对系统性故障的严谨归因而过拟合于特定任务。为此，本文提出了名为 Ecdysis（意为“蜕皮/蜕变”）的创新框架。Ecdysis 严格区分了模型特定缺陷与系统性脚手架缺陷，采用批次级跨实例故障聚合机制与故障驱动协作精炼方案，实现了高达 1.84 倍的脚手架训练加速，并将推理准确率提升了 18.56%。

Ecdysis：面向大模型智能体运行时脚手架的高效进化训练

Ecdysis: Efficient and Effective Training of Runtime Harnesses for LLM Agents

概要

Summary

Ecdysis 是一个旨在提升大语言模型智能体 (AI Agent) 自进化运行时脚手架 (Runtime Harness) 训练效率与效果的创新框架。传统的脚手架进化技术通常依赖迭代搜索过程——即基于单一任务实例的执行反馈，反复对候选脚手架进行评估与修改。虽然这种方法能够实现持续优化，但存在两大核心痛点： 1. 高昂的时间开销： 由智能体的反复执行与频繁代码修改所致； 2. 泛化性能较差： 由于缺乏系统性的故障诊断机制，极易过拟合于已见任务和特定的失败模式。

Ecdysis is an innovative framework designed to improve the training efficiency and effectiveness of self-evolving runtime harnesses for Large Language Model (LLM) agents. Traditional harness evolution techniques typically rely on iterative search processes—repeatedly evaluating and modifying candidate harnesses based on single task instance feedback. While this approach allows for continuous optimization, it suffers from two major limitations: 1. High Time Overhead: Caused by repetitive agent executions and code revisions. 2. Poor Generalization: Overfitting to observed tasks and specific failure patterns because of a lack of principled failure diagnosis.

为了攻克这些难题，Ecdysis 明确区分了“针对特定模型的权宜适应” (model-specific accommodations) 与“系统性的脚手架级修复” (systematic harness-level repairs)。通过采用批次级跨实例故障聚合范式与故障驱动协作精炼机制，Ecdysis 能够跨多个任务联合分析故障证据。这不仅大幅缩短了训练时间，还避免了为特定模型的怪癖做无谓的调整。实验结果表明，与现有方法相比，Ecdysis 在脚手架训练上实现了高达 1.84 倍的加速，并将最终推理准确率提升了 18.56%。

To overcome these challenges, Ecdysis differentiates between model-specific accommodations and systematic harness-level repairs. By leveraging a batch-level cross-instance failure aggregation paradigm alongside Failure-Driven Collaborative Refinement, Ecdysis jointly analyzes failure evidence across multiple tasks. This significantly reduces training time while avoiding unnecessary adjustments for model-specific quirks. Experimental results demonstrate that Ecdysis achieves up to a 1.84× speedup in harness training and boosts reasoning accuracy by 18.56% compared to existing methods.

论文元数据

Paper Metadata

作者

Authors

  • Ruiqing Yue
  • Yu Cui
  • Zhuoyu Sun
  • Sicheng Pan
  • Xianhong Xue
  • Tingyu Li
  • Ting Li
  • Wenzhuo Zhu
  • Yi Chen
  • Yifei Liu
  • Baohan Huang
  • Zhe Cui
  • Haibin Zhang
  • Cong Zuo
  • Ruiqing Yue
  • Yu Cui
  • Zhuoyu Sun
  • Sicheng Pan
  • Xianhong Xue
  • Tingyu Li
  • Ting Li
  • Wenzhuo Zhu
  • Yi Chen
  • Yifei Liu
  • Baohan Huang
  • Zhe Cui
  • Haibin Zhang
  • Cong Zuo

摘要

Abstract

自进化运行时脚手架能够显著提升大语言模型智能体 (LLM Agent) 的能力，并为优化智能体执行提供了前景广阔的范式。现有的脚手架进化方法通常依赖迭代搜索，根据任务实例的执行反馈反复评估和修订候选脚手架。虽然该范式能够实现脚手架的持续优化，但频繁的智能体执行和代码修改带来了巨大的时间开销，并且可能过拟合于已知任务和特定失败模式，导致在未见任务上的泛化能力下降。我们发现，缺乏规范严谨的故障诊断机制是脚手架进化的关键瓶颈：观察到的失败既可能源自特定模型的固有缺陷，也可能源于系统性的脚手架缺陷，直接针对个别失败进行优化容易导致不必要的模型特定迎合。为此，我们提出了 Ecdysis——一个兼具高效性与有效性的框架，它将特定模型的权宜迎合与系统性脚手架修复区分开来，通过识别跨任务的复现性失败模式，引导系统优先解决脚手架的系统性缺陷。Ecdysis 采用了批次级跨实例故障聚合范式来联合分析多个任务实例的故障证据，并进一步引入故障驱动协作精炼机制来诊断失败根因并迭代完善脚手架修改规范。通过将跨实例故障分析与多角色诊断相结合，Ecdysis 以更短的训练时间实现了更有效的脚手架进化。实验表明，与现有的脚手架进化方法相比，Ecdysis 在脚手架训练上实现了高达 1.84 倍的加速，同时将生成脚手架的推理准确率提升了 18.56%。

Self-evolving runtime harnesses can substantially improve the capabilities of large language model (LLM) agents and provide a promising paradigm for optimizing agent execution. Existing harness evolution methods typically rely on iterative search, repeatedly evaluating and revising candidate harnesses based on execution feedback from task instances. While this paradigm enables continuous harness optimization, it incurs substantial time overhead due to repeated agent executions and code modifications, and may overfit to observed tasks and specific failure patterns, resulting in degraded generalization to unseen tasks. We identify the lack of principled failure diagnosis as a key bottleneck in harness evolution: an observed failure can reflect either model-specific deficiencies or systematic harness deficiencies, and directly optimizing against individual failures can lead to unnecessary model-specific accommodation. We therefore propose Ecdysis, an efficient and effective framework that distinguishes model-specific accommodation from harness-level repair and biases adaptation toward systematic harness deficiencies by identifying recurring cross-task failure patterns. Ecdysis adopts a batch-level cross-instance failure aggregation paradigm to jointly analyze failure evidence from multiple task instances and further introduces Failure-Driven Collaborative Refinement to diagnose failure causes and iteratively refine harness modification specifications. By combining cross-instance failure analysis with multi-role diagnosis, Ecdysis enables more effective harness evolution with lower training time. Experiments show that Ecdysis achieves up to a 1.84x speedup in harness training compared with existing harness evolution methods, while improving the reasoning accuracy of the resulting harnesses by 18.56%.

获取与资源

Access & Resources