跳转至

文章背景与核心概要

本文探讨了选择 AI 智能体脚手架（scaffolding）是否对运营成本产生比工具接口（模型上下文协议 [MCP] 与普通命令行接口 [CLI]）更大的影响。通过在一套标准化的软件任务（对私有在线 git 仓库执行六项操作）中测试七种智能体脚手架和五种语言模型，并通过检查仓库状态而非依赖自我报告来验证结果，作者发现：脚手架对成本起决定性作用，不原生支持 MCP 的脚手架完全通过 CLI 完成任务，其成本比支持 MCP 的脚手架低 5.0 倍到 28 倍；MCP 并非必需品；且 MCP 运行在失败的无效工作上浪费了更高比例的资金。

The Scaffolding Matters More Than the Interface: A Controlled Comparison of MCP and CLI Tool Use Across Seven Agent Scaffoldings, Five Language Models, and One Software Task

脚手架的重要性胜过接口：对七种智能体脚手架、五种语言模型以及一项软件任务中 MCP 与 CLI 工具调用的对照比较

Summary

摘要

This paper investigates whether the choice of AI agent scaffolding has a larger impact on operational costs than the tool interface (Model Context Protocol [MCP] versus ordinary Command-Line Interface [CLI]). By running a standardized software task (six operations against a private online git repository) across seven agent scaffoldings and five language models—and verifying results through repository state inspection rather than self-reporting—the authors discovered that: * Scaffolding dominates cost: Scaffoldings without native MCP support completed tasks entirely via CLI and were 5.0x to 28x cheaper than MCP-supporting scaffoldings. * MCP is optional: Two scaffoldings proved that MCP is unnecessary for this category of work. * Cost of failure varies: MCP runs wasted a significantly higher percentage of spent money on failed, non-productive work (12.9%) compared to CLI runs (2.2%), despite failure frequencies remaining similar across interfaces. * Model sensitivity: Local models (e.g., a 27-billion-parameter model) experienced massive cost fluctuations (up to 139x) purely based on the choice of scaffolding.

本文研究了 AI 智能体脚手架的选择是否对运营成本产生比工具接口（模型上下文协议 [MCP] 与普通命令行接口 [CLI]）更大的影响。通过在一套标准化的软件任务（对私有在线 git 仓库执行六项操作）中测试七种智能体脚手架和五种语言模型，并通过检查仓库状态而不是依赖智能体自我报告来验证结果，作者发现： * 脚手架决定成本： 不具备原生 MCP 支持的脚手架完全通过 CLI 完成任务，其成本比支持 MCP 的脚手架低 5.0 倍到 28 倍。 * MCP 是可选的： 有两种脚手架证明，对于这类工作而言，MCP 并非必需。 * 失败成本有所不同： 尽管不同接口的失败频率相似，但 MCP 运行在失败的、非生产性工作上浪费的资金比例（12.9%）明显高于 CLI 运行（2.2%）。 * 模型敏感度： 本地模型（例如 270 亿参数的模型）仅由于脚手架的选择就经历了巨大的成本波动（高达 139 倍）。

Document Metadata

文档元数据

Field Detail
arXiv ID arXiv:2608.08654 [cs.AI]
Authors Marc Alier Forment, María José Casañ Guerrero, Francisco José García-Peñalvo, Juanan Pereira
Submitted August 9, 2026
Primary Subject Artificial Intelligence (cs.AI)
ACM Classes D.2.8; I.2.11
License Creative Commons Attribution 4.0 license iconview license
字段 详情
arXiv ID arXiv:2608.08654 [cs.AI]
作者 Marc Alier Forment, María José Casañ Guerrero, Francisco José García-Peñalvo, Juanan Pereira
提交时间 2026年8月9日
主要学科 人工智能 (cs.AI)
ACM 分类 D.2.8; I.2.11
许可证 知识共享署名 4.0 许可证图标查看许可证

Abstract

摘要

How much an AI coding agent costs to run can depend more on the agent scaffolding that drives it than on the interface through which it reaches its tools. We set out to measure the cost of tool use over the Model Context Protocol (MCP) against tool use over an ordinary command-line interface (CLI), a difference on which published estimates disagree by more than an order of magnitude while resting on practitioner reports that cannot be reproduced.

运行一个 AI 编程智能体的成本，在很大程度上可能取决于驱动它的智能体脚手架，而不是它用来调用工具的接口。我们着手测量通过模型上下文协议 (MCP) 使用工具的成本与通过普通命令行接口 (CLI) 使用工具的成本差异。此前已发布的估算结果在这一点上意见不一，差异超过一个数量级，且其依据是无法复现的从业者报告。

We ran one fixed software task—six operations against a private online git repository—across seven agent scaffoldings and five language models, and we verified completion by inspecting the repository state rather than trusting the agent's self-report. The dominant effect was the scaffolding. Two of the seven ship no MCP support at all; they completed every run using only the CLI, which shows that MCP is unnecessary for this class of work, and they were 5.0x to 28x cheaper than the five scaffoldings that do support MCP, comparing CLI runs alone with no MCP server attached anywhere.

我们在一套固定的软件任务（对私有在线 git 仓库执行六项操作）中测试了七种智能体脚手架和五种语言模型，并通过检查仓库状态而不是信任智能体的自我报告来验证完成情况。起决定性作用的是脚手架。七个脚手架中有两个根本不提供 MCP 支持；它们完全通过 CLI 完成了每一次运行，这表明对于这类工作而言 MCP 并非必需，并且如果仅对比不连接任何 MCP 服务器的 CLI 运行，它们的成本比支持 MCP 的五个脚手架低 5.0 倍到 28 倍。

The effect was largest for a small 27-billion-parameter model running locally, whose cost varied 139x across scaffoldings while it completed the task under all of them. The comparison we set out to make proved unstable: thirteen strictly paired MCP-to-CLI ratios span 0.43x to 29x, with outliers on both sides. The two interfaces separate on the cost of failure, where 12.9 per cent of the money spent on MCP runs bought no completed work against 2.2 per cent on CLI runs, but not on its frequency: failures were equally common in both, in the original runs and in their repetitions alike. Agents frequently ignored the interface they were assigned, so comparisons that do not verify actual behaviour measure an unknown mixture. The harness, the task, the verification and the complete dataset are released as open source.

这种影响对于本地运行的小型 270 亿参数模型最为明显，该模型在不同脚手架下的成本波动达到了 139 倍，尽管它在所有脚手架下都完成了任务。我们最初试图进行的比较被证明是不稳定的：13 个严格配对的 MCP 与 CLI 比例跨度从 0.43 倍到 29 倍不等，两侧均存在异常值。这两个接口在失败成本上存在分歧——MCP 运行中花费的资金有 12.9% 没有换来已完成的工作，而 CLI 运行中这一比例为 2.2%——但在失败频率上没有区别：无论是原始运行还是重复运行，两者中失败的概率同样常见。智能体经常会忽略分配给它们的接口，因此未验证实际行为的比较衡量的是一种未知的混合状态。测试工具、任务、验证方法以及完整的数据集均已开源发布。

Additional Information & Resources

附加信息与资源