面向长程应用开发的高级 Harness 架构设计：生成-评估对抗与多智能体闭环工程

Harness Design for Long-Running Application Development

作者：Prithvi Rajasekaran，Anthropic Labs 团队成员。

Written by Prithvi Rajasekaran, a member of our Labs team.

文章背景与核心概要

在构建复杂的自主智能体系统时，如何让大语言模型 (Large Language Model, LLM) 在无需人类介入的情况下完成长达数小时的高质量软件研发，一直是 AI 工程界亟待攻克的瓶颈。来自 Anthropic Labs 的研究员 Prithvi Rajasekaran 针对这一难题，指出了当前单智能体系统普遍存在的两大痛点：上下文耗尽带来的“上下文焦虑”以及审视自身代码时的“自我评估盲目自信”。为此，他借鉴生成对抗网络 (Generative Adversarial Networks, GAN) 的精髓，创新性地构建了由“生成器”与独立“评估器”构成的多智能体对抗评测 Harness 架构。该架构通过把主观模糊的前端设计标准转化为可量化指标，并引入冲刺契约与多阶段持续测试机制，成功实现了包含物理引擎、关卡编辑器在内的全栈复杂应用的自主迭代开发，为长程智能体系统的工程落地提供了极具价值的实战范式。

核心内容摘要

Summary

在过去的数月里，Prithvi Rajasekaran 针对两个紧密交织的技术难题展开了深入探索：一是如何引导 Claude 打造出高水准的前端界面设计，二是如何在无需任何人类干预的前提下构建出完整的全功能应用程序。尽管单纯依赖提示词工程 (Prompt Engineering) 以及初代智能体测试框架 (Harness) 确实能够提升基准表现，但这两种方案最终都触碰到了难以逾越的能力天花板。

Over several months, Prithvi Rajasekaran investigated two interconnected problems: getting Claude to produce high-quality frontend designs and building complete applications without human intervention. While prompt engineering and initial agent harnesses improved baseline performance, both approaches eventually hit ceilings.

为了打破这些局限，Rajasekaran 从生成对抗网络 (Generative Adversarial Networks, GAN) 中汲取灵感，设计了一套包含生成器 (Generator) 与评估器 (Evaluator) 的多智能体协作架构。通过将主观审美判断 (例如“这个设计究竟好不好看？”) 拆解转化为具体、可量化的评分标准，并将负责编写代码的智能体与负责评判审查的智能体完全剥离解耦，该架构在长达数小时的自主编码会话中，成功构建出了功能丰富、交互完整的全栈应用程序。

To break through these limitations, Rajasekaran took inspiration from Generative Adversarial Networks (GANs) to design a multi-agent structure featuring a generator and an evaluator. By turning subjective qualities (like "is this design good?") into concrete, gradable criteria and separating the agent doing the work from the agent judging it, the architecture successfully produced rich, full-stack applications over multi-hour autonomous coding sessions.

为什么朴素的实现方案难以奏效

Why Naive Implementations Fall Short

早期的实验充分表明，Harness 的架构设计对长程智能体编程的实际成效有着决定性的深远影响。通过引入一个初始化智能体 (Initializer Agent) 将产品需求规格说明书拆解为清晰的任务清单，再指派一个编码智能体 (Coding Agent) 逐一落实各个功能特性，这种做法确实显著拓展了智能体的能力边界。开源社区中类似的流行方案，例如“Ralph Wiggum”模式，也能够让智能体保持持续的循环迭代。

Earlier experiments demonstrated that harness design profoundly impacts the effectiveness of long-running agentic coding. Using an initializer agent to decompose a product spec into a task list—and a coding agent to implement tasks one feature at a time—pushed capabilities forward. Similar community methods like the "Ralph Wiggum" approach keep agents in continuous iteration.

然而，面对复杂的工程任务时，由于以下两种主要失效模式的存在，智能体依然极易“偏离正轨”并导致任务失败：

However, complex tasks still cause agents to go off the rails due to two major failure modes:

1. 连贯性丧失与“上下文焦虑” (Loss of Coherence & "Context Anxiety")： 随着上下文窗口 (Context Window) 逐渐被填满，模型会不可避免地丧失逻辑连贯性，并往往表现出明显的“上下文焦虑”——每当感知到自己正逼近上下文容量上限时，便会草率地提前收尾工作。虽然就地压缩技术能够对历史上下文进行浓缩摘要，但上下文重置 (Context Resets) (即彻底清空当前上下文窗口，并通过结构化状态交接启动一个全新的智能体) 能够为模型重新提供一张纯净的“白纸”，在彻底解决这一问题的同时，代价则是系统编排复杂度的上升、额外的 Token 开销以及更高的响应延迟。
2. 自我评估偏差 (Self-Evaluation Bias)： 当要求大语言模型对其自身的输出成果进行评估时，它们往往会表现出盲目的过度自信，对平庸的产出不吝赞美之词。这种现象在 UI/UX 界面设计等偏主观的任务中表现得尤为突出。将生成器与一个完全独立、且天生持怀疑立场的评估器隔离开来，能够构建起行之有效的负反馈闭环，为生成器后续的精细化迭代指明清晰坚实的方向。

1. Loss of Coherence & "Context Anxiety": As context windows fill up, models lose coherence and often exhibit "context anxiety," prematurely wrapping up work as they approach perceived limits. While compaction summarizes history in place, context resets (clearing the context window entirely and starting a fresh agent with a structured state handoff) provide a clean slate, solving the issue at the cost of orchestration complexity, token overhead, and latency.
2. Self-Evaluation Bias: When asked to evaluate their own output, LLMs tend to confidently praise mediocre work. This is especially pronounced in subjective tasks like UI/UX design. Separating the generator from a standalone, skeptical evaluator creates a functional feedback loop that gives the generator concrete direction for iteration.

前端设计：让主观审美品质变为可量化评分的客观标准

Frontend Design: Making Subjective Quality Gradable

为了彻底攻克前端设计中的自我评估偏差难题，Rajasekaran 为生成器智能体与评估器智能体统一确立了四大维度的打分评判标准：

To tackle the self-evaluation issue in frontend design, Rajasekaran established four grading criteria for both the generator and evaluator agents:

• 设计质感 (Design Quality)： 色彩搭配、字体排印、版面布局与图像设计是否融合为统一的情感氛围与鲜明的品牌辨识度？
• 独创性 (Originality)： 是否展现出深思熟虑的创意构思？还是完全依赖千篇一律的通用模板布局、组件库默认样式以及千篇一律的“AI 劣质套版 (AI Slop)” (例如在白色卡片上方叠加毫无生气的紫色渐变) ？
• 工艺水平 (Craft)： 考察技术执行细节，诸如字体层级结构、间距一致性、色彩协调性以及无障碍对比度等。
• 功能完备度 (Functionality)： 脱离纯美学因素，评估界面的交互可用性与功能健全程度。

• Design Quality: Do colors, typography, layout, and imagery combine into a coherent mood and identity?
• Originality: Are there deliberate creative choices, or does the output rely on generic template layouts, library defaults, and "AI slop" (such as purple gradients over white cards)?
• Craft: Technical execution like typographic hierarchy, spacing consistency, color harmony, and contrast ratios.
• Functionality: Usability independent of aesthetics.

对设计质感与独创性赋予更高的权重倾向，成功打破了 Claude 倾向于输出稳妥、平庸默认设计的保守倾向。结合使用 Claude Agent SDK 与 Playwright MCP，评估器能够直接在真实环境中访问并操作实时渲染的 HTML/CSS/JS 页面，捕获实际屏幕截图并提交详尽深入的审查批评。通过每次生成任务执行 5 至 15 轮高强度迭代 (单次会话耗时长达 4 小时) ，生成器智能体学会了在实践中抉择：要么对极具潜力的视觉路线进行精细雕琢，要么彻底推倒重来、开辟崭新的美学风格。

Weighting design quality and originality more heavily pushed Claude past safe, bland defaults. Using the Claude Agent SDK alongside the Playwright MCP, the evaluator directly navigated the live HTML/CSS/JS page, taking screenshots and submitting detailed critiques. Running 5 to 15 iterations per generation (taking up to four hours), the generator learned to either refine a promising direction or pivot entirely to a fresh aesthetic approach.

扩展至全栈工程开发

Scaling to Full-Stack Coding

将这种受 GAN 启发的对抗模式拓展应用到全栈开发中，能够非常自然地映射到传统的软件开发生命周期 (Software Development Life Cycle, SDLC) 之上——在此场景下，严谨的代码审查 (Code Review) 与质量保证测试 (QA) 取代了原本的前端审美设计评估。

Applying this GAN-inspired pattern to full-stack development maps naturally onto the software development lifecycle, where code review and QA replace design evaluation.

核心架构设计

The Architecture

借助 Claude Agent SDK 的自动化压缩机制维持连续会话，升级后的全栈架构围绕三大核心智能体角色展开分工协作：

Using a continuous session managed by the Claude Agent SDK's automatic compaction, the updated architecture relies on three core agent personas:

• 规划智能体 (Planner)： 接收仅包含 1 至 4 句话的简短提示词，并将其展开为详尽完备的产品需求规格说明书。为了避免底层错误像滚雪球般层层传导扩散，它专注于高维度的产品业务逻辑与整体技术方案设计，刻意规避过早介入细枝末节的实现细节，并在产品设计中积极整合内嵌的 AI 智能体功能。
• 生成智能体 (Generator)： 按照敏捷冲刺 (Sprint) 节奏分阶段工作，基于 React、Vite、FastAPI 以及 SQLite/PostgreSQL 技术栈逐一落地功能特性，并完全依赖 Git 进行严谨的版本控制与代码追踪。
• 评估智能体 (Evaluator)： 借助 Playwright MCP 深度操控并检测正在运行的应用系统——覆盖前端 UI 交互组件、后端 API 接口端点以及数据库状态一致性。它对照严格的达标门槛对冲刺成果进行评判，并在发生偏离预期或功能异常时输出精准详细的缺陷工单。

• Planner: Takes a short 1-4 sentence prompt and expands it into a comprehensive product spec. It focuses on high-level product context and technical design rather than granular implementation details to prevent error cascading, and actively integrates AI agent features into the product specs.
• Generator: Works in sprints, implementing features using a React, Vite, FastAPI, and SQLite/PostgreSQL stack, relying on Git for version control.
• Evaluator: Uses the Playwright MCP to interact with the running application—testing UI elements, API endpoints, and database states. It grades sprints against specific thresholds and reports concrete bugs when divergences occur.

在开启每个冲刺之前，生成器与评估器会共同协商并订立一份冲刺契约 (Sprint Contract)，以此弥合顶层抽象需求规格与底层可测试代码实现之间的语义鸿沟。

Before each sprint, the generator and evaluator negotiate a sprint contract to bridge the gap between the high-level spec and testable code.

Harness 运行实战对比

Running the Harness

在一项以“开发复古怀旧电子游戏制作器”为提示词的实测任务中，单智能体独立运行与完整 Harness 架构协同运行展现出了天壤之别的成果差距：

Testing the system with a retro video game maker prompt yielded stark differences between a solo agent run and the full harness:

• 单智能体运行 (Solo Run)： 耗时 20 分钟，Token 成本为 9 美元。虽然表面上初具雏形，但界面排版极度僵化，且由于代码中存在严重的逻辑串联缺陷，诸如实体移动与键盘操作响应等最基础的核心游戏交互完全陷入瘫痪。
• 完整 Harness 运行 (Full Harness)： 耗时长达 6 小时，Token 成本为 200 美元。规划智能体将简短的需求拆解拓展为一个跨越 10 轮冲刺、囊括 16 项核心功能特性的专业规格方案。最终交付的系统拥有自适应视口的现代化布局、功能强悍的像素精灵与关卡编辑器、内置的 AI 辅助创作工具，以及表现出色、手感流畅的真实物理引擎交互逻辑。

• Solo Run: 20 minutes duration, $9 token cost. While superficially functional, layout was rigid, and core game interactions (like entity movement and inputs) were completely broken due to wiring errors in the code.
• Full Harness: 6 hours duration, $200 token cost. The planner expanded the prompt into a 16-feature spec across 10 sprints. The resulting app featured viewport-filling layouts, robust sprite and level editors, integrated AI generation tools, and fully playable game physics.

Harness	Duration	Cost
Solo	20 min	$9
Full harness	6 hr	$200

评估器缺陷发现实例

Evaluator Findings Example

Contract criterion	Evaluator finding
Rectangle fill tool allows click-drag to fill a rectangular area with selected tile	FAIL — Tool only places tiles at drag start/end points instead of filling the region. fillRectangle function exists but isn't triggered properly on mouseUp.
User can select and delete placed entity spawn points	FAIL — Delete key handler at LevelEditor.tsx:892 requires both selection and selectedEntityId to be set, but clicking an entity only sets selectedEntityId. Condition should be selection
User can reorder animation frames via API	FAIL — PUT /frames/reorder route defined after /{frame_id} routes. FastAPI matches 'reorder' as a frame_id integer and returns 422: "unable to parse string as an integer."

Harness 架构的演进迭代

Iterating on the Harness

随着 Claude Opus 4.6 的发布——该模型展现出了更强大的任务规划能力、更出色的超长任务耐力、更精准的代码库检索导航以及显著提升的自主 Debug 能力——开发者终于能够在保障工程质量的前提下，大幅简化 Harness 自身的编排复杂度。

With the release of Claude Opus 4.6—which features enhanced planning, extended task sustainability, improved codebase navigation, and better self-debugging—harness complexity could be safely reduced.

在去除了僵化的冲刺节奏限制后，Opus 4.6 得以在单次连贯的开发会话中端到端地完成整个工程流水线，评估器也相应后移至整个构建流程终局时的统一质检验收阶段。在挑战紧贴模型能力极限的最尖端任务时，独立的评估器依然具有举足轻重的价值；但对于那些模型早已能够独立稳定交付的常规工作，繁复的评估环节反而会沦为不必要的性能与算力累赘。

Removing the strict sprint construct allowed Opus 4.6 to handle the build stream in a single continuous session, moving the evaluator to a final end-of-run pass. The evaluator remains valuable when tackling tasks at the absolute edge of model capabilities, but becomes unnecessary overhead for capabilities the model handles reliably on its own.

针对一款基于浏览器的数字音频工作站 (Digital Audio Workstation, DAW) 进行构建测试，得到了如下各个阶段的具体耗时与成本明细：

Running a test for a browser-based Digital Audio Workstation (DAW) produced the following breakdown:

Agent & Phase	Duration	Cost
Planner	4.7 min	$0.46
Build (Round 1)	2 hr 7 min	$71.08
QA (Round 1)	8.8 min	$3.24
Build (Round 2)	1 hr 2 min	$36.89
QA (Round 2)	6.8 min	$3.09
Build (Round 3)	10.9 min	$5.88
QA (Round 3)	9.6 min	$4.06
Total V2 Harness	3 hr 50 min	$124.70

最终打造出的 DAW 应用程序涵盖了完整可用的编曲视图 (Arrangement View)、混音台 (Mixer)、播放控制面板 (Transport Controls)，以及一个内嵌的 AI 智能体——该智能体能够全流程自主生成旋律、设定节拍速率 (BPM) 并完成音轨的自动编排。

The resulting DAW included a working arrangement view, mixer, transport controls, and an integrated agent capable of generating melodies, setting tempos, and arranging tracks end-to-end.

未来展望与工程启示

What Comes Next

随着底层基座模型能力的不断跃升，基础的脚手架封装将变得不再那么关键，开发者完全可以让原生模型直接处理较为直观的问题。然而，精心设计的高级 Harness 架构组合却能解锁极其复杂的综合能力，将系统表现推向远超基线模型的全新高度。

As underlying models improve, basic scaffolding matters less, allowing developers to let baseline models handle simpler problems. However, sophisticated harness combinations unlock complex capabilities that extend far beyond baseline performance.

面向未来的 AI 工程师关键实践经验总结：

Key takeaways for AI engineers moving forward:

• 深入剖析真实轨迹： 仔细审视模型在面对真实世界复杂工程问题时的运行轨迹 (Execution Traces)，并根据期望的最终成果进行针对性微调与引导。
• 按需拆解专业角色： 当任务复杂度逼近模型天花板、需要争取更高的能力空间 (Headroom) 时，果断采用专业化的智能体角色对任务进行模块化分解。
• 伴随模型演进敏捷瘦身： 随着下一代新模型的推陈出新，持续重构并精简 Harness 体系，在果断剥离已过时冗余组件的同时，大胆探索全新维度的工程可能性。

• Inspect model traces on realistic problems and tune for desired outcomes.
• Decompose complex tasks using specialized agent personas when headroom is required.
• Continuously re-examine and simplify harnesses as new models launch, stripping away obsolete components while introducing new capabilities.

致谢

Acknowledgements

特别鸣谢 Mike Krieger、Michael Agaby、Justin Young、Jeremy Hadfield、David Hershey、Julius Tarng、Xiaoyi Zhang、Barry Zhang、Orowa Sidker、Michael Tingley、Ibrahim Madha、Martina Long 以及 Canyon Robbins 对本研究工作的突出贡献。

Special thanks to Mike Krieger, Michael Agaby, Justin Young, Jeremy Hadfield, David Hershey, Julius Tarng, Xiaoyi Zhang, Barry Zhang, Orowa Sidker, Michael Tingley, Ibrahim Madha, Martina Long, and Canyon Robbins for their contributions to this work.

同时感谢 Jake Eaton、Alyssa Leonard 和 Stef Sequeira 在本文撰写与打磨过程中提供的悉心帮助。

Thanks also to Jake Eaton, Alyssa Leonard, and Stef Sequeira for their help shaping the post.

附录

Appendix

规划智能体生成的计划示例

Example plan generated by planner agent

RetroForge - 2D Retro Game Maker

Overview
RetroForge is a web-based creative studio for designing and building 2D retro-style video games. It combines the nostalgic charm of classic 8-bit and 16-bit game aesthetics with modern, intuitive editing tools—enabling anyone from hobbyist creators to indie developers to bring their game ideas to life without writing traditional code.

The platform provides four integrated creative modules: a tile-based Level Editor for designing game worlds, a pixel-art Sprite Editor for crafting visual assets, a visual Entity Behavior system for defining game logic, and an instant Playable Test Mode for real-time gameplay testing. By weaving AI assistance throughout (powered by Claude), RetroForge accelerates the creative process—helping users generate sprites, design levels, and configure behaviors through natural language interaction.

RetroForge targets creators who love retro gaming aesthetics but want modern conveniences. Whether recreating the platformers, RPGs, or action games of their childhood, or inventing entirely new experiences within retro constraints, users can prototype rapidly, iterate visually, and share their creations with others.

Features
1. Project Dashboard & Management
The Project Dashboard is the home base for all creative work in RetroForge. Users need a clear, organized way to manage their game projects—creating new ones, returning to works-in-progress, and understanding what each project contains at a glance.

User Stories: As a user, I want to:

- Create a new game project with a name and description, so that I can begin designing my game
- See all my existing projects displayed as visual cards showing the project name, last modified date, and a thumbnail preview, so that I can quickly find and continue my work
- Open any project to enter the full game editor workspace, so that I can work on my game
- Delete projects I no longer need, with a confirmation dialog to prevent accidents, so that I can keep my workspace organized
- Duplicate an existing project as a starting point for a new game, so that I can reuse my previous work

Project Data Model: Each project contains:

Project metadata (name, description, created/modified timestamps)
Canvas settings (resolution: e.g., 256x224, 320x240, or 160x144)
Tile size configuration (8x8, 16x16, or 32x32 pixels)
Color palette selection 
All associated sprites, tilesets, levels, and entity definitions

...