跳转至

基于平均场代理模型的共享GPU上并发异构AI推理可扩展运行时调度

文章背景与核心概要

在共享GPU上并发部署多个异构AI模型会产生复杂的资源竞争，从而使运行时调度变得异常复杂。传统的代理模型有助于避免高昂的在线基准测试成本，但其性能分析开销会随着共运行模型数量（\(N\)）的增加而成组合级增长。

为了解决这一问题，本文作者提出了一种平均场代理模型（MeanField surrogate model）。该模型无需显式映射所有的联合交互，而是利用局部配置和聚合的GPU状态来预测单个模型的性能。实验表明，在并发的大语言模型（LLM）和视觉工作负载（\(N \in \{2, 3, 4, 5, 6\}\)）测试中，该方法实现了高预测准确率（\(R^2 \approx 0.96\)），且经验样本预算呈线性而非组合级增长。当集成到针对 \(N=5\) 场景（包含 78,732 种可行联合配置）的遗传算法（GA）调度器中时，该系统在八种动态工作负载场景下实现了与穷举搜索相差仅 0.10% 的结果，且零SLA违规。此外，完整的在线遗传算法决策中位时间仅需 26 毫秒——比穷举代理搜索快大约 \(5\times\)。

📋 Summary

Deploying multiple heterogeneous AI models concurrently on a shared GPU creates complex resource contention, complicating runtime scheduling. Traditional surrogate models help avoid costly online benchmarking, but their profiling overhead scales combinatorially with the number of co-running models (\(N\)).

To solve this, the authors propose a MeanField surrogate model. Instead of explicitly mapping all joint interactions, it predicts individual model performance using local configuration and aggregate GPU state. * Accuracy & Scalability: Experiments on concurrent LLM and vision workloads (\(N \in \{2, 3, 4, 5, 6\}\)) demonstrate high predictive accuracy (\(R^2 \approx 0.96\)) with an empirical sample budget that grows linearly rather than combinatorially. * Scheduler Performance: When integrated into a genetic algorithm (GA) scheduler for an \(N=5\) scenario (comprising 78,732 feasible joint configurations), the system achieved results within 0.10% of an exhaustive search with zero SLA violations across eight dynamic workload scenarios. Furthermore, complete online GA decisions required a median of just 26 ms—approximately \(5\times\) faster than exhaustive surrogate searching.

📌 Metadata & Publication Details

  • arXiv Identifier: arXiv:2609.02109 [cs.DC]
  • Primary Subject: Distributed, Parallel, and Cluster Computing (cs.DC)
  • Secondary Subjects: Artificial Intelligence (cs.AI)
  • Authors: Youssef Ennouri, Soonhoi Ha
  • Submission Date: September 2, 2026
  • Publication Venue: Accepted for publication in IEEE Embedded Systems Letters
  • Related DOI: 10.1109/LES.2026.3724013