跳转至

从大模型推理到智能体工作负载：服务系统的特征分析与启示

文章背景与核心概要

随着人工智能应用从单一的大语言模型（LLM）推理转向长周期的“智能体（Agentic）工作负载”（即LLM协调工具、持久化状态及外部环境），现有的服务系统因过度针对传统推理进行优化而面临严峻挑战。

本文通过引入 AgentSysBench——一套包含十个代表性智能体应用的基准测试套件与测量工具，深入分析了受控部署与生产环境下的运行数据。研究揭示了智能体工作负载区别于标准LLM推理的六大独特属性，并提出了针对性的系统设计方案，在提升系统效率、降低延迟及优化内存利用率方面取得了显著成果。

📝 摘要

随着AI应用从孤立的大语言模型（LLM）推理转向长运行的智能体工作负载（即LLM协调工具、持久化状态和外部环境），现有的服务系统因严格针对传统推理进行优化而显得力不从心。

为了解决这一问题，作者引入了 AgentSysBench，这是一套包含十个代表性智能体应用的基准测试套件和测量工具。通过分析受控部署和生产环境的追踪数据，该研究确定了智能体工作负载区别于标准LLM服务的六个独特属性，并提出了可行的设计探索，显著改善了系统效率、延迟和内存利用率。

As AI applications shift from isolated Large Language Model (LLM) inference to long-running agentic workloads (where LLMs coordinate tools, persistent state, and external environments), current serving systems struggle because they are optimized strictly for conventional inference.

To address this, the authors introduce AgentSysBench, a benchmark suite and measurement toolkit featuring ten representative agentic applications. By analyzing both controlled deployments and production traces, the study identifies six unique properties of agentic workloads that diverge from standard LLM serving and proposes actionable design explorations that significantly improve system efficiency, latency, and memory utilization.

👥 作者与所属机构

  • Chaokun Chang, Yukun Zhou, Kaihua Fu, Dakai An, Tianyu Feng, Hanfeng Lu, Sheng Yao, Pu Guo, Yinghao Yu, Yizhou Shan, Bo Li, Binhang Yuan, Wei Wang
  • Chaokun Chang, Yukun Zhou, Kaihua Fu, Dakai An, Tianyu Feng, Hanfeng Lu, Sheng Yao, Pu Guo, Yinghao Yu, Yizhou Shan, Bo Li, Binhang Yuan, Wei Wang

🔬 智能体工作负载的关键特征

该研究对比了传统LLM推理与智能体工作负载，并强调了六个决定性的系统特征：

  1. 重量级且有状态的执行： 在10个应用中有5个，非LLM组件占据了延迟的主导地位，沙盒工作集内存峰值达到每会话28 GB。
  2. 异构资源亲和性： 应用结合了需求迥异的组件——GPU密集型推理、内存密集型检索和CPU密集型沙盒，导致任务延迟差异高达 32倍。
  3. 瓶颈转移： 系统瓶颈在请求、模型和部署环境之间动态波动。
  4. 长空闲状态： 生产会话在活跃执行步骤之间保持空闲状态，时长从几分钟到几小时不等。
  5. 控制平面税： 由工具模式和环境观测引入的辅助LLM调用及上下文开销，消耗了宝贵的计算能力和上下文窗口空间。
  6. 跨请求冗余： 来自三个应用的生产追踪显示，搜索查询和网页抓取操作存在大量冗余，这提供了巨大的缓存优化空间。

The research contrasts traditional LLM inference with agentic workloads and highlights six defining system characteristics:

  1. Heavyweight and Stateful Execution:
    Non-LLM components dominate latency in 5 out of 10 applications, with sandbox working-set memory peaking at 28 GB per session.
  2. Heterogeneous Resource Affinity:
    Applications combine components with vastly different requirements—GPU-bound inference, memory-bound retrieval, and CPU-bound sandboxes—leading to task latencies diverging by up to 32×.
  3. Shifting Bottlenecks:
    System bottlenecks fluctuate dynamically across requests, models, and deployment environments.
  4. Long Idle States:
    Production sessions maintain idle states for minutes to hours between active execution steps.
  5. Control-Plane Tax:
    Auxiliary LLM calls and context overhead introduced by tool schemas and environment observations consume valuable compute power and context window space.
  6. Cross-Request Redundancy:
    Production traces from three applications reveal heavy redundancy in search queries and web fetch operations, presenting a major caching opportunity.

⚙️ 启示与设计探索

该论文通过四个核心设计探索证明了这些发现具有极高的实践价值：

  • 任务感知服务： 将延迟降低了 29% 至 40%。
  • 通信感知部署： 将性能提升了高达 4.5倍。
  • 状态卸载： 将内存使用量降低了 4.6倍。
  • 工具结果缓存： 消除了 35.2% 的冗余搜索调用，并节省了 19.3% 的总搜索延迟。

The paper demonstrates that these findings are highly actionable through four core design explorations:

  • Task-Aware Serving: Reduces latency by 29% to 40%.
  • Communication-Aware Placement: Improves performance by up to 4.5×.
  • State Offloading: Cuts memory usage down by 4.6×.
  • Tool-Result Caching: Eliminates 35.2% of redundant search calls and saves 19.3% of aggregate search latency.

🔗 链接与资源