你的边缘设备是以单次前向传递进行基准测试的，但你的Agent将运行循环

文章背景与核心概要

本文深入探讨了当前边缘AI硬件基准测试与实际智能体（Agent）工作负载之间的严重脱节问题。行业虽然已经意识到边缘AI硬件受限于热设计功耗而非单纯的计算能力，但现有的测试仍基于单次“前向传递”进行，完全忽略了智能体在实际运行中处于无界循环（决策、行动、再评估）的状态。

文章指出，这种持续的热负载会导致现代边缘设备迅速发生热节流甚至硬件性能崩溃。为了解决这一痛点，开发者和工程师必须将评估标准从“每秒生成Token数”转变为“完成单项任务所需的焦耳数（Joules per Task）”，并在智能体设计中明确引入热预算和步骤限制，以确保端侧智能体的稳定落地。

评估体系的缺陷

关于边缘AI的硬件讨论已经发生了转变：功耗和热管理现在已经优先于原始吞吐量。然而，这些指标仍然是在“单次前向传递”（模型接收输入并产生输出，从而给芯片留出冷却时间）的假设下进行预算的。

The hardware conversation regarding edge AI has shifted: power and thermal management now take precedence over raw throughput. However, these metrics are still budgeted against a "single forward pass"—a model receiving input and producing output, allowing the silicon time to cool.

但这并不是智能体（Agent）的工作方式。智能体执行的是决策和工具调用的循环。这个循环的长度不是由硬件或模型决定的，而是由智能体正在解决的具体问题决定的。在实践中，智能体运行的成本往往被隐藏，直到为时已晚。

This is not how agents function. An agent performs a loop of decision-making and tool-calling. The length of this loop is not a property of the hardware or the model, but of the specific problem the agent is solving. In practice, the cost of an agentic run is often hidden until it is too late.

循环是无界的

像 OpenAI Agents SDK 这样的框架旨在运行循环，直到达到轮数限制或完成任务。在服务器上，这是一个计费问题；但在边缘设备上，这是一个热量问题。由于循环长度决定了占空比（duty cycle），而被动散热无法克服由此产生的热量，因此智能体的运行时间对硬件稳定性构成了直接威胁。

Frameworks like the OpenAI Agents SDK are built to run loops until a turn limit is reached or the task is completed. On a server, this is a billing concern; on an edge device, it is a thermal concern. Because loop length dictates duty cycle, and passive cooling cannot overcome the resulting heat, the agent’s runtime becomes a direct threat to hardware stability.

持续负载 vs. 基准测试

2026年3月的一项针对四种平台在持续负载下进行测试的基准测试，揭示了当前旗舰设备的脆弱性： * iPhone 16 Pro： 在两次推理内，由于热节流，吞吐量下降了 44%。 * Galaxy S24 Ultra： 温控器施加了硬性的频率底线（frequency floor），导致系统变得无法使用，而不仅仅是变慢。

A March 2026 benchmark testing four platforms under sustained load revealed the fragility of current flagship devices: * iPhone 16 Pro: Experienced a 44% reduction in throughput due to thermal throttling within two inferences. * Galaxy S24 Ultra: The thermal governor imposed a hard frequency floor, rendering the system unusable rather than just slower.

这些测试使用了相同的提示词重复二十次——这是最“友好”的工作负载。现实情况是，随着智能体循环的进行，其上下文（以及因此受内存带宽限制的成本）在增长，而设备散热的能力却在下降。产品恰好在这两条曲线交汇的地方失效。

These tests used the same prompt for twenty repetitions—the "friendliest" possible workload. The reality is that as an agent loops, its context (and thus its memory-bandwidth-bound cost) grows, while the device’s ability to dissipate heat decreases. The product fails exactly where these two curves intersect.

重新定义成功：单任务焦耳数

我们必须停止根据“每秒Token数”来选择硬件。相反，我们应该根据以可预测成本完成有界循环的能力来评估硬件。 * 指标： “完成单项任务所需的焦耳数（Joules per completed task）”，并附带方差数据。 * 经验教训： 小型 NPU（如 Hailo-10H）可能比 GPU 慢，但它为智能体循环提供了高得多的稳定性和效率。

We must stop selecting hardware based on "tokens per second." Instead, we should evaluate hardware based on the ability to complete a bounded loop at a predictable cost. * The Metric: "Joules per completed task," accompanied by a variance figure. * The Lesson: A small NPU (like the Hailo-10H) may be slower than a GPU, but it offers significantly higher stability and efficiency for agentic loops.

为性能上限进行设计

制程节点会不断改进，但固定硬件预算与可变智能体需求之间的根本不对称性将依然存在。为了构建强健的边缘智能体： 1. 设定硬性限制： 根据设备的热设计余量，在产品规格中定义轮数限制。 2. 上下文感知： 向智能体输入其自身的“余量”（电池状态、热状态），以便它能够在操作系统强制关机之前进行总结和提交。 3. 测试尾部情况： 停止仅测试中位数。要在仿真中测试“十四步”失败案例，因为长时间运行的复杂任务正是你的产品不可避免会崩溃的地方。

Process nodes will improve, but the fundamental asymmetry between fixed hardware budgets and variable agent demands will remain. To build robust edge agents: 1. Set Hard Limits: Define turn limits in the product specification based on the device's thermal envelope. 2. Contextual Awareness: Feed the agent its own "headroom" (battery state, thermal status) so it can summarize and commit before the OS forces a shutdown. 3. Test the Tail: Stop testing for the median. Test for the "fourteen-step" failure case in simulation, as long-running, complex tasks are where your product will inevitably break.

我们目前出货的设备，其热预算尺寸仅够拍一张照片，但我们却要求它们像工作站一样运行。是时候承认工作负载的形态已经发生改变了。

We are currently shipping devices whose thermal budgets are sized for a single photograph, but we are asking them to perform like a workstation. It is time to acknowledge that the workload has changed shape.

作者：Chase W. Hughes