跳转至

比 Flash 更快：利用注意力稀疏性实现高效的长文本解码

文章背景与核心概要

长文本大语言模型（LLMs）在解码阶段通常面临内存带宽瓶颈以及注意力机制带来的二次方复杂度增长。为了解决内存开销与计算效率低下的固有矛盾，研究人员推出了 Faster Flash Decoding (FFD)——这是一个旨在打破内存墙的硬件算法协同设计框架。

通过将选择器（selector）和计算器（computer）集成到完全融合的内核中、利用低比特量化进行内容感知扫描，并应用动态的 top-delta 块过滤策略，FFD 在大规模场景下（最高支持 256K 上下文长度）实现了高达 11.6 倍的内核级加速 和 2.37 倍的端到端吞吐量提升，同时完美保持了模型的准确率。

Paper Metadata / 论文元数据

Field / 字段 Detail / 详情
arXiv ID arXiv:2609.00097 [cs.LG]
Subjects Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
ACM Classes I.2.7
Status Accepted at ICML 2026
Submitted August 31, 2026
Authors Zhigeng Liu, Zhiyuan Ning, Ruixiao Li, Xiaoran Liu, Yuerong Song, Min Zhang, Ziwei He, Xipeng Qiu
Code Repository GitHub - faster-flash-decoding
Field Detail
arXiv ID arXiv:2609.00097 [cs.LG]
Subjects Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
ACM Classes I.2.7
Status Accepted at ICML 2026
Submitted August 31, 2026
Authors Zhigeng Liu, Zhiyuan Ning, Ruixiao Li, Xiaoran Liu, Yuerong Song, Min Zhang, Ziwei He, Xipeng Qiu
Code Repository GitHub - faster-flash-decoding

Abstract / 摘要

长文本大语言模型（LLMs）的发展受到解码过程中内存带宽瓶颈以及注意力机制二次方复杂度的限制。为了克服基于元数据的指标在内存开销与自适应选择策略在计算效率低下之间的内在权衡，我们提出了 Faster Flash Decoding (FFD)，这是一个旨在打破长文本解码中内存墙的新型硬件算法协同设计框架。FFD 将选择器和计算器集成到一个完全融合的内核中，通过低比特量化进行内容感知扫描来替代外部元数据索引。此外，我们引入了 top-delta 策略，该策略能够动态过滤块，在无需全局同步的情况下实现分布自适应的稀疏性。作为一种无需训练且即插即用的解决方案，FFD 还支持将扫描结果重复用于计算，实现了高达 11.6 倍的内核级加速，上下文长度扩展至 256K，端到端吞吐量提升 2.37 倍。在 RULER 和 LongBench 上的经验验证证实，FFD 在保持高比例稀疏性的同时维持了模型的准确率。

The development of long-context Large Language Models (LLMs) is constrained by the memory bandwidth bottleneck and quadratic complexity of the attention mechanism during decoding. To overcome the inherent trade-offs between the memory overhead of metadata-based metrics and the computational inefficiency of adaptive selection strategies, we present Faster Flash Decoding (FFD), a novel hardware-algorithm co-design framework designed to break the memory wall in long-context decoding. FFD integrates the selector and computer into a fully fused kernel, replacing external metadata indices with content-aware scanning via low-bit quantization. Furthermore, we introduce the top-delta strategy, which dynamically filters blocks to achieve distribution-adaptive sparsity without global synchronization. Offering a training-free and plug-and-play solution, FFD also enables the reuse of scanning results for computation, achieving up to 11.6x kernel-level speedup and scaling to 256K context length, with 2.37x end-to-end throughput improvement. Empirical validation on RULER and LongBench confirms that FFD maintains model accuracy while delivering high-ratio sparsity.

Key Innovations of FFD / FFD 的核心创新

  • 完全融合的内核（Fully Fused Kernel）： 结合了选择器和计算器组件，消除了不必要的内存往返。
  • 内容感知扫描（Content-Aware Scanning）： 用低比特量化扫描替代了外部元数据索引。
  • Top-Delta 策略： 动态过滤注意力块，在无需全局同步的情况下实现分布自适应的稀疏性。
  • 即插即用且无需训练（Plug-and-Play & Training-Free）： 无缝集成到现有架构中，并直接重用扫描结果进行计算。
  • Fully Fused Kernel: Combines the selector and computer components to eliminate unnecessary memory round-trips.
  • Content-Aware Scanning: Replaces external metadata indices with low-bit quantization scanning.
  • Top-Delta Strategy: Dynamically filters attention blocks to accomplish distribution-adaptive sparsity without requiring global synchronization.
  • Plug-and-Play & Training-Free: Seamlessly integrates into existing architectures and reuses scanning outcomes directly for computation.

Associated Resources & Full-Text Links / 相关资源与全文链接