OpWeave：面向异构大模型推理服务的灵活算子级解耦框架

OpWeave: Flexible Operator Disaggregation for Heterogeneous LLM Serving

arXiv ID: arXiv:2609.14237 [cs.DC]

Submitted On: September 13, 2026

Primary Subject: Distributed, Parallel, and Cluster Computing (cs.DC)

Secondary Subjects: Artificial Intelligence (cs.AI)

Authors: Zikun Li, Yixuan Mei, Shiqi Pan, Zixuan Chen, Xiaowen Zhang, Mengdi Wu, Shuhuai Lin, Yutong Yang, Zhihao Zhang, Xupeng Miao, Rashmi Vinayak, Zhihao Jia

文章背景与核心概要

在当今大语言模型 (Large Language Model, LLM) 的云端推理部署中，传统架构习惯将注意力机制 (Attention) 与前馈神经网络 (FFN/MoE) 等不同算子捆绑在同规格芯片上同机运行。然而，这两类核心算子在计算特征上存在天然分化：注意力机制重度受限于显存带宽，而前馈网络则极度消耗计算算力，导致高昂的 GPU 资源往往无法被充分利用。针对这一工业落地瓶颈，研究团队推出了端到端算子级解耦推理服务框架 OpWeave，首次实现了跨异构算力环境的灵活解耦调度。OpWeave 创新性地结合了严谨的理论成本模型、可高效应对混合注意力架构的规律感知规划器，以及基于 vLLM 的高性能执行运行时，打破了以往系统僵化的算子切分边界。在严格满足端到端延迟服务等级目标 (Service Level Objective, SLO) 的前提下，OpWeave 在同构与异构 GPU 集群上分别实现了高达 1.78 倍与 1.89 倍的服务成本削减，为大模型时代构建高性价比的基础设施开辟了全新路径。

核心概述

Summary

OpWeave 是一套专为大语言模型 (Large Language Model, LLM) 基础设施打造的端到端框架，专门用于实现异构环境下的算子级解耦服务 (Operator-Level Disaggregated Serving, ODS)。

OpWeave is an end-to-end framework designed for heterogeneous Operator-Level Disaggregated Serving (ODS) in Large Language Model (LLM) infrastructures.

传统的 LLM 推理服务系统习惯将所有计算算子集中部署在同一类芯片上同机运行；而前沿的系统优化方案正逐步将推理流水线拆解为更细粒度的阶段——例如在 Token 生成的解码 (Decode) 阶段，将注意力机制 (Attention) 与前馈网络 (Feed-Forward Network, FFN) 或混合专家网络 (Mixture of Experts, MoE) 的计算彻底分离开来，以此更好地契合不同硬件的性能特长，并支持各算子模块的独立弹性扩缩容。然而，现有的工业实现方案大多受制于固化僵硬的算子切分边界，且缺乏一套统一的分析框架来量化评估“何时进行解耦才真正划算与最具性价比”。

While traditional LLM serving systems colocate all operations, modern optimizations increasingly disaggregate inference into finer-grained stages (such as separating attention from FFN or MoE execution during the decode phase) to improve hardware matching and enable independent scaling. However, existing implementations suffer from rigid operator boundaries and lack unified frameworks to determine when disaggregation is cost-effective.

针对上述核心挑战，OpWeave 依托三大核心组件给出了系统性解决方案：

OpWeave addresses these challenges through three main components:

理论分析成本模型 (Analytical Cost Model)：从数学原理上严格推导并界定了同构与异构 ODS 相比传统同机部署服务的效率收益理论边界。
规律感知规划器 (Regularity-Aware Planner)：联合协同优化算子切分策略与底层集群部署配置，即便面对架构复杂的混合注意力 (Hybrid-Attention) 模型，也能确保庞大的解空间保持高效可解。
基于 vLLM 的执行运行时 (vLLM-Based Runtime)：能够在各类异构设备资源池中平滑无缝地调度并执行带有动态算子阶段的自动化编排计划。

Analytical Cost Model: Mathematically bounds the efficiency gains of both homogeneous and heterogeneous ODS over traditional colocated serving.

Regularity-Aware Planner: Jointly optimizes operator partitioning and deployment configuration, ensuring the search space remains tractable even for complex hybrid-attention architectures.

vLLM-Based Runtime: Executes synthesized plans seamlessly across diverse device groups with dynamic operator stages.

核心性能亮点

Key Performance Highlights

在同构 GPU 集群上，最高可将推理服务成本降低 \(1.78\times\)。
在异构 GPU 集群上，相比可行的最强基准方案，最高可将服务成本削减 \(1.89\times\)。
在大幅降本的同时，始终严格保障严苛的端到端延迟服务等级目标 (Service Level Objectives, SLOs)。

Reduces serving costs by up to \(1.78\times\) on homogeneous GPU clusters.

Reduces serving costs by up to \(1.89\times\) on heterogeneous GPU clusters relative to the best feasible baseline.

Consistently maintains strict latency Service Level Objectives (SLOs).

论文摘要

Abstract

当今的大语言模型 (LLM) 推理服务系统正日益将推理任务拆解为更细粒度的执行阶段；近期的前沿探索已开始在解码阶段尝试将注意力机制 (Attention) 与前馈网络 (FFN) 或混合专家网络 (MoE) 分离执行。这种算子级解耦服务 (Operator-Level Disaggregated Serving, ODS) 不仅能够实现计算特征与底层硬件的精准匹配，还支持面向各类异构设备展开独立的弹性扩展。然而，现有的系统往往采用固化的算子切分边界，且缺乏一套统一的理论分析框架来明确判定“算子解耦究竟在何种场景下能够真正降低服务成本”。为此，我们提出了面向异构算力环境的端到端算子级解耦服务框架 OpWeave。OpWeave 构建了一套分析成本模型，从理论上严格给出了同构与异构 ODS 相对于传统同机部署服务的收益上界。同时，它设计了一个规律感知规划器，对算子切分策略与集群部署配置进行联合协同优化，即使面对复杂的混合注意力模型，也能保证巨大的搜索空间高效可解。此外，基于 vLLM 构建的执行运行时能够跨越多类异构设备组，平滑调度并执行包含灵活算子阶段的生成方案。评估结果显示，在严格满足延迟服务等级目标 (SLO) 的前提下，OpWeave 相比当前表现最佳的可行基线，在同构 GPU 集群上最高降低了 \(1.78\times\) 的服务成本，在异构 GPU 集群上最高实现了 \(1.89\times\) 的成本削减。

LLM serving systems increasingly disaggregate inference into finer-grained stages, with recent approaches separating attention from FFN or MoE execution during decode. This operator-level disaggregated serving (ODS) can improve hardware matching and enable independent scaling, particularly across heterogeneous devices. However, existing systems fix operator boundaries and lack a unified characterization of when disaggregation reduces serving cost. We present OpWeave, an end-to-end framework for heterogeneous ODS. OpWeave provides an analytical cost model that bounds the gains of homogeneous and heterogeneous ODS over colocated serving. It jointly optimizes operator partitioning and deployment configuration through a regularity-aware planner that keeps the search tractable even for hybrid-attention models. A vLLM-based runtime executes the synthesized plans with flexible operator stages across heterogeneous device groups. In our evaluation, OpWeave reduces serving cost by up to \(1.78\times\) on homogeneous and \(1.89\times\) on heterogeneous GPU clusters relative to the best feasible baseline, while meeting latency SLOs.