跳转至

DCO：基于预测性管理的面向大模型加速器动态缓存编排

DCO: Dynamic Cache Orchestration for LLM Accelerators through Predictive Management

arXiv:2512.07312 [cs.AR]
Journal: IEEE Transactions on Computers (2026)
Subjects: Hardware Architecture (cs.AR); Artificial Intelligence (cs.AI); Distributed, Parallel, and Cluster Computing (cs.DC)
Authors: Zhongchun Zhou, Chengtao Lai, Yuhang Gu, Wei Zhang
Submitted: 10 Sep 2026 (v2)
DOI / Paper: 10.1109/TC.2026.3733368

文章背景与核心概要

大语言模型 (Large Language Model, LLM) 的爆发式普及正推动 AI 专用加速芯片向着更高算力与更强专用化演进。然而，当前主流硬件普遍采用的深层次软件显式管理便签存储器 (Scratchpad Memory, SPM) 及其复杂的异步搬运机制，给上层软件编译与开发者带来了极为沉重的开发包袱。

发表于计算机体系结构权威期刊《IEEE Transactions on Computers》的这项研究另辟蹊径，探索了硬件设计光谱的另一端：为多核 AI 加速器配备应用感知的共享系统级缓存，从而大幅降低编程复杂度。研究提出了 DCO (Dynamic Cache Orchestration) 框架，充分利用软件栈中的数据流先验信息，对缓存替换（含死块预测）、执行旁路和抖动抑制进行全方位智能协调。在周期精确仿真、解析建模与 15nm 工艺 RTL 物理落地（芯片面积仅 \(0.064\,\text{mm}^2\) ，主频达 2 GHz）的多维度验证下，DCO 较传统缓存架构实现了最高达 1.80 倍的加速性能。

概述

Summary

随着大语言模型 (LLM) 推动 AI 加速器朝着更加专业化与高算力的方向演进，复杂的软件管理便签存储器 (SPM) 往往会引入巨大的底层编程负担。本研究探讨了一种截然不同的硬件设计思路：构建一个配备共享系统级缓存并通过应用感知策略进行管理的多核 AI 加速器，以兼顾极致性能与适度的编程开发成本。

As large language models (LLMs) drive AI accelerators toward increasingly specialized and powerful designs, complex software-managed scratchpad memories (SPMs) often introduce heavy programming overheads. This research explores an alternative hardware approach: a multi-core AI accelerator equipped with a shared system-level cache managed through application-aware policies.

通过充分利用软件栈中蕴含的数据流先验信息，本论文提出的 DCO (Dynamic Cache Orchestration) 架构能够全方位协同指导缓存替换（包括死块预测）、执行旁路 (Execution Bypassing) 以及缓存抖动抑制 (Thrashing Mitigation)。结合周期精确模拟器、解析数学建模以及物理 RTL 级流片综合（基于 15nm 工艺，面积仅为 \(0.064\,\text{mm}^2\) ，运行频率达 2 GHz）的严密评测表明，该架构展现出强大的规模扩展能力，无论是跨核心数据共享还是相互独立的负载场景，都能较传统缓存体系结构取得高达 1.80 倍的加速比 。

By leveraging software stack dataflow information, the proposed Dynamic Cache Orchestration (DCO) framework guides cache replacement (including dead-block prediction), execution bypassing, and thrashing mitigation. Evaluations using cycle-accurate simulations, analytical modeling, and physical RTL implementation (15nm process, \(0.064\,\text{mm}^2\) area, 2 GHz clock frequency) demonstrate robust scalability, handling both inter-core data sharing and independent workloads to achieve up to a 1.80× speedup over conventional cache architectures.

文档概览

Document Overview

  • arXiv 标识符： arXiv:2512.07312 [cs.AR] (2026年9月10日提交 v2 版本)
  • 学科分类： 硬件架构 (cs.AR)；人工智能 (cs.AI)；分布式、并行与集群计算 (cs.DC)
  • 作者团队： Zhongchun Zhou, Chengtao Lai, Yuhang Gu, Wei Zhang
  • 期刊发表： IEEE Transactions on Computers (2026)
  • 关联 DOI： 10.1109/TC.2026.3733368
  • arXiv Identifier: arXiv:2512.07312 [cs.AR] (v2 submitted on September 10, 2026)
  • Subjects: Hardware Architecture (cs.AR); Artificial Intelligence (cs.AI); Distributed, Parallel, and Cluster Computing (cs.DC)
  • Authors: Zhongchun Zhou, Chengtao Lai, Yuhang Gu, Wei Zhang
  • Journal Reference: IEEE Transactions on Computers (2026)
  • Related DOI: 10.1109/TC.2026.3733368

摘要

Abstract

大语言模型 (LLM) 的迅速普及正促使 AI 加速器迈向更加强大和专用化的硬件架构。与以往采用深层次便签存储器 (SPM) 及其异步调度机制从而进一步加剧软件开发复杂度的路线不同，我们研究了设计空间另一极的全新可能：设计一个配备共享系统级缓存且受应用感知管理策略驱动的多核 AI 加速器，从而使软件编程工作量保持在适度区间。我们的方案深度挖掘软件栈中可用的数据流信息，协同指导缓存块替换（包含死块预测）、旁路绕行决策以及抑制缓存抖动的动态机制。

The rapid adoption of large language models (LLMs) is pushing AI accelerators toward increasingly powerful and specialized designs. Instead of further complicating software development with deeply hierarchical scratchpad memories (SPMs) and their asynchronous management, we investigate the opposite point of the design spectrum: a multi-core AI accelerator equipped with a shared system-level cache and application-aware management policies, which keeps the programming effort modest. Our approach exploits dataflow information available in the software stack to guide cache replacement (including dead-block prediction), in concert with bypass decisions and mechanisms that alleviate cache thrashing.

我们利用周期精确模拟器对该方案进行了全面评估，观察到与传统缓存架构相比，系统获得了实质性的性能提升（最高达 1.80 倍加速比 ）。此外，我们构建并验证了一个能够兼顾实际计算与通信重叠重叠行为的解析模型，将策略的度量结论成功推广到了更大规模的真实工作负载中。实验结果表明，当旁路策略与抖动抑制机制协同工作时，能够同时驾驭具备跨核心数据共享以及无共享的独立计算场景，并带来显著的加速效果。

We assess the proposal using a cycle-accurate simulator and observe substantial performance gains (up to 1.80× speedup) compared with conventional cache architectures. In addition, we build and validate an analytical model that takes into account the actual overlapping behaviors to extend the measurement results of our policies to real-world larger-scale workloads. Experiment results show that when functioning together, our bypassing and thrashing mitigation strategies can handle scenarios both with and without inter-core data sharing and achieve remarkable speedups.

最后，我们在 RTL 级实现了该硬件设计：在 15nm 工艺下，核心面积仅为 \(0.064\,\text{mm}^2\) ，且能够稳定运行在 2 GHz 的时钟频率上。我们的研究结论全面挖掘了共享缓存设计在辅助下一代 AI 加速系统演进中的巨大潜力。

Finally, we implement the design in RTL and the area of our design is \(0.064\,\text{mm}^2\) with a 15nm process, which can run at a 2 GHz clock frequency. Our findings explore the potential of the shared cache design to assist the development of future AI accelerator systems.

补充信息

Additional Information