跳转至

就绪与下发解耦：面向智能体大模型工作流的长尾感知调度

Decoupling Readiness from Release for Tail-Aware Scheduling of Agentic LLM Workflows

arXiv: 2609.10964 [cs.AI]
Subjects: Artificial Intelligence (cs.AI); Software Engineering (cs.SE)
Submitted: September 10, 2026
Authors: Bochao Feng, Jianjiang Li, Haojie Wang, Lin Qiao, Yinghui Li, Yukun Yan, Jidong Zhai

文章背景与核心概要

智能体大语言模型 (Agentic LLM) 工作流由多轮模型生成与频繁的外部工具交互交织而成，其端到端整体执行延迟不仅取决于单次模型推理的速度，更直接受制于各个已就绪轮次 (Turns) 何时被正式下发至执行引擎。当前主流运行时普遍采用“急切下发策略 (Eager Release Policy)”，即一旦轮次就绪便立即派发；但在系统高负载并发拥堵时，这种策略会导致大量已下发却未完成的工作在流水线中淤积，剥夺了工作流调度器动态重排序的控制权，进而引发极严重的长尾延迟恶化。为此，研究团队提出了 长尾风险感知轮次下发调度方法，创新性地将“轮次就绪”与“实际下发”彻底解耦，动态裁决何时派发以及控制管道内的未完成工作预算。该方法引入均值-条件风险价值 (CVaR) 优化目标，在高负载资源争用场景下将工作流的 P95 流转延迟大幅降低，带来最高达 3.50 倍的性能加速，为大规模多智能体系统的工业级调度提供了全新思路。

📌 执行概要

📌 Executive Summary

智能体大语言模型 (Agentic Large Language Model, LLM) 工作流高度依赖于交替迭代的模型思考轮次与外部工具交互，这意味着整体任务的完成时间不仅受到纯模型推理速度的制约，更取决于已就绪的轮次在何时被正式下发到后端执行。

Agentic Large Language Model (LLM) workflows rely heavily on iterative model turns interleaved with tool interactions, meaning that overall completion times are bottlenecked not just by inference speed, but by when ready turns are released for execution.

标准运行时系统通常采用急切下发策略 (Eager Release Policy)，即只要某一轮次就绪就立即派发执行。然而在严重的系统资源争用下，这种做法会在底层积压过多已下发但尚未完成的任务。由于这些轮次已经被提交进执行队列，工作流层级的调度器彻底失去了对它们重新排序调度的灵活性，导致长尾延迟 (Tail Latency) 严重恶化。

Standard runtime environments utilize an eager release policy, dispatching every turn immediately upon readiness. However, under heavy system contention, this approach accumulates excessive uncompleted work. Because these turns are already submitted, workflow-level schedulers lose the ability to reorder them, causing severe tail latency degradation.

为了化解这一困境，作者团队提出了一种长尾风险感知的轮次下发调度方法。该方案通过动态裁决以下两个核心问题，将轮次的“就绪状态”与“下发动作”实现了解耦： 1. 下一个应当下发哪一个已就绪的轮次。 2. 流水线中应当维持多少处于已下发但未完成状态的工作量。

To resolve this issue, the authors introduce a tail-risk-aware turn release scheduling method. This approach decouples turn readiness from release by dynamically deciding: 1. Which ready turn should be released next. 2. How much released yet unfinished work should be maintained in the pipeline.

核心亮点与实验成果

Key Highlights & Results

  • 优化目标： 借助均值-条件风险价值 (Mean–Conditional Value-at-Risk, CVaR) 目标，动态平抑未完成工作流中不断累积的长尾风险。
  • 智能优先级排序： 结合对轮次工作量的在线精准估计，优化已就绪轮次队列的派发次序。
  • 自适应工作预算： 根据实时监测到的队列压力，自适应调整允许下发的工作总量预算。
  • 显著性能收益： 基于真实软件工程任务中的智能体执行轨迹（涵盖不同 LLM 与多样化请求到达率）进行评估，该方法在轻负载下与急切下发性能完全持平，而在高负载争用下将工作流周转时间的 P95 长尾延迟降低多达 \(3.50\times\) 加速。
  • Optimization Objective: Leverages a mean–Conditional Value-at-Risk (CVaR) objective to mitigate evolving tail risks in unfinished workflows.
  • Smart Prioritization: Integrates online estimations of turn workloads to optimize ready-turn queues.
  • Adaptive Work Budgeting: Automatically tunes the released work budget in response to real-time queue pressure.
  • Performance Gains: Evaluated against real agent execution traces (software engineering tasks across diverse LLMs and arrival rates), the proposed method matches eager release performance under light loads while cutting P95 workflow flow time by up to a \(3.50\times\) speedup under contention.

📖 论文摘要

📖 Abstract

💬 [原文引用 / Original Quote]: Agentic LLM workflows consist of sequences of model turns interleaved with tool interactions, so their end-to-end completion time depends not only on inference speed but also on when ready turns are released. Most runtimes release each turn immediately upon readiness. Under contention, this eager release policy can accumulate released but unfinished work; once submitted, those turns can no longer be reordered by the workflow-level policy, increasing tail latency. We present a tail-risk-aware turn release scheduling method that jointly decides which ready turn to release next and how much released but unfinished work to maintain. The method uses a mean--Conditional Value-at-Risk (CVaR) objective to capture the evolving tail risk of unfinished workflows, incorporates online estimates of turn work when prioritizing ready turns, and adapts the released work budget to observed queue pressure. We evaluate the method using real agent execution traces from software engineering tasks across multiple LLMs and workflow arrival rates. The method performs comparably to eager release under light load and substantially reduces the P95 of workflow flow time under contention, achieving up to a \(3.50\times\) speedup.

智能体大模型工作流由模型轮次与工具交互交替构成的序列组成，因此其端到端完成时间不仅取决于推理速度，还取决于就绪轮次何时被下发执行。大多数运行时系统在每个轮次就绪后立即予以派发。在资源争用环境下，这种急切下发策略会导致已下发但未完成的任务不断堆积；一旦提交，工作流级调度策略便无法再对这些轮次重新排序，从而显著推高了长尾延迟。我们提出了一种长尾风险感知的轮次下发调度方法，联合决策接下来应当下发哪个就绪轮次以及维持多大体量的已下发未完成工作。该方法采用均值-条件风险价值 (CVaR) 目标来刻画未完成工作流不断演变的长尾风险，在就绪轮次优先级排序中融入对轮次计算量的在线估计，并根据观测到的队列压力自适应动态调整下发工作预算。我们基于来自软件工程任务的真实智能体执行轨迹，在多种大语言模型与工作流到达率下对该方法进行了全面评估。实验表明，该方法在轻负载下的表现与急切下发相当，而在资源争用严重时大幅降低了工作流周转时间的 P95 长尾延迟，取得了高达 \(3.50\times\) 的显著加速。

🔗 链接与资源