跳转至

HISA：面向细粒度稀疏注意力的高效分层索引机制

HISA: Efficient Hierarchical Indexing for Fine-Grained Sparse Attention

arXiv:2603.28458 [cs.LG]
Conference: Published as a conference paper at COLM 2026
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Authors: Yufei Xu, Fanxu Meng, Fan Jiang, Yuxuan Wang, Ruijie Zhou, Zhaohui Wang, Jiexi Wu, Zhixin Pan, Xiaojuan Tang, Wenjie Pei, Tongxuan Liu, Di Yin, Xing Sun, Muhan Zhang
Submitted: 30 Mar 2026; Last revised: 10 Sep 2026 (v4)
DOI / Paper: 10.48550/arXiv.2603.28458

文章背景与核心概要

在大语言模型 (Large Language Model, LLM) 处理超长上下文时，细粒度稀疏注意力机制（以 DeepSeek 提出的 DeepSeek Sparse Attention, 简称 DSA 为代表）能够显著压缩注意力计算量。然而，这类机制通常依赖一个轻量级索引器对历史键值进行全局遍历，当上下文急剧延长时，索引器对每个 Query 的前缀全量扫描便成为制约推理吞吐的核心计算瓶颈。

为了攻克这一瓶颈，研究人员在 COLM 2026 上提出了 HISA (Hierarchical Indexed Sparse Attention)。HISA 是一种完全即插即用的索引器替换方案，它将单调的扁平扫描重构为两阶段分层检索：“块级粗筛”快速丢弃不相关区域，“Token 级精炼”则在保留的候选块中精准定位关键 Token。在无需任何额外训练或微调的情况下，HISA 可直接无缝集成至 DeepSeek-V3.2 与 GLM-5 等主流模型中，在 64K 上下文下大幅提升内核执行速度，并在各项长文本基准中维持了与原生 DSA 媲美的精度表现。

概述

Summary

Token 级别的细粒度稀疏注意力机制（例如 DeepSeek Sparse Attention，即 DSA）通过在计算注意力子集之前利用轻量级索引器为每个查询 (Query) 筛选出最相关的历史键 (Key) ，从而实现高精度且低计算量的注意力计算。然而，由于索引器必须针对每一个 Query 扫描全部的前缀历史，在处理长文本序列时，这反而成为了阻碍吞吐提升的计算瓶颈。

Token-level sparse attention mechanisms (such as DeepSeek Sparse Attention, or DSA) select fine-grained historical keys for each query using a lightweight indexer before calculating attention on the subset. However, because the indexer must perform a full-prefix scan for every query, it becomes a computational bottleneck at scale.

HISA (Hierarchical Indexed Sparse Attention) 是一种即插即用的索引器替换方案，它将这一线性检索路径重构为一个精巧的两阶段分层流程： 1. 块级粗筛 (Block-Level Coarse Filtering)： 对池化后的块级表征进行快速打分，果断剔除不相关的上下文区间； 2. Token 级精细定位 (Token-Level Refinement)： 仅在被保留的候选块内部运行原始的细粒度索引器。

HISA (Hierarchical Indexed Sparse Attention) is a plug-and-play indexer replacement that optimizes this search path into a two-stage hierarchical procedure: 1. Block-Level Coarse Filtering: Scores pooled block representations to discard irrelevant regions. 2. Token-Level Refinement: Applies the original indexer exclusively within the retained candidate blocks.

无需任何额外的微调或训练，HISA 便可无缝整合到 DeepSeek-V3.2 和 GLM-5 等尖端大模型中。它在 64K 上下文长度下带来了显著的内核加速比，并在保证与原生 DSA 同等生成质量的同时，大幅超越了传统的块级稀疏基准方法。

Without requiring any additional training or fine-tuning, HISA seamlessly integrates into models like DeepSeek-V3.2 and GLM-5, achieving quality comparable to original DSA while delivering significant speedups (up to \(\times\) speedup at 64K context) and outperforming standard block-sparse baselines.

论文元数据

Paper Metadata

字段 (Field) 详情 (Details)
arXiv 编号 arXiv:2603.28458 [cs.LG]
论文标题 HISA: Efficient Hierarchical Indexing for Fine-Grained Sparse Attention
作者团队 Yufei Xu, Fanxu Meng, Fan Jiang, Yuxuan Wang, Ruijie Zhou, Zhaohui Wang, Jiexi Wu, Zhixin Pan, Xiaojuan Tang, Wenjie Pei, Tongxuan Liu, Di Yin, Xing Sun, Muhan Zhang
主要学科 机器学习 (cs.LG)，人工智能 (cs.AI)
发表会议 COLM 2026 会议收录论文
提交时间 2026年3月30日 (v1)；最新修订：2026年9月10日 (v4)
Field Details
arXiv ID arXiv:2603.28458 [cs.LG]
Title HISA: Efficient Hierarchical Indexing for Fine-Grained Sparse Attention
Authors Yufei Xu, Fanxu Meng, Fan Jiang, Yuxuan Wang, Ruijie Zhou, Zhaohui Wang, Jiexi Wu, Zhixin Pan, Xiaojuan Tang, Wenjie Pei, Tongxuan Liu, Di Yin, Xing Sun, Muhan Zhang
Primary Subject Machine Learning (cs.LG), Artificial Intelligence (cs.AI)
Publication Published as a conference paper at COLM 2026
Submitted 30 Mar 2026 (v1); Last revised: 10 Sep 2026 (v4)

摘要

Abstract

以 DeepSeek Sparse Attention (DSA) 为代表的 Token 级稀疏注意力机制，通过轻量级索引器为每个查询 (Query) 对全部历史键进行打分，并仅在精选出的子集上计算注意力，从而达成了极为细粒度的键筛选。尽管下游的稀疏注意力本身的计算复杂度优势明显，但其索引器在每个 Query 计算时依然必须全量扫描前缀序列，这带来了随上下文长度增长而急剧恶化的逐层计算瓶颈。我们提出了 HISA (Hierarchical Indexed Sparse Attention)，作为索引器的即插即用替代模块。它将扁平的 Token 级全扫描路径重写为两阶段分层流程：(1) 块级粗筛阶段，对池化后的块表征打分以快速剔除无关区域；紧接着 (2) Token 级精炼阶段，仅在保留下来的候选块内部调用原索引器。HISA 完全保留了下游 Sparse MLA 算子所消费的相同 Token 级 Top-sparse 稀疏模式，且不需要任何额外的模型训练。在算子级基准测试中，HISA 在 64K 上下文下实现了显著加速。在 Needle-in-a-Haystack 和 LongBench 评测中，我们在未经任何微调的情况下直接用 HISA 索引器替换了 DeepSeek-V3.2 与 GLM-5 中的原始索引器。测试结果显示，HISA 在生成质量上与原始 DSA 几乎完全一致，同时大幅优于常规的块级稀疏基准。

Token-level sparse attention mechanisms, exemplified by DeepSeek Sparse Attention (DSA), achieve fine-grained key selection by scoring every historical key for each query through a lightweight indexer, then computing attention only on the selected subset. While the downstream sparse attention itself scales favorably, the indexer must still scan the entire prefix for every query, introducing an per-layer bottleneck that grows prohibitively with context length. We propose HISA (Hierarchical Indexed Sparse Attention), a plug-and-play replacement for the indexer that rewrites the search path from a flat token scan into a two-stage hierarchical procedure: (1) a block-level coarse filtering stage that scores pooled block representations to discard irrelevant regions, followed by (2) a token-level refinement stage that applies the original indexer exclusively within the retained candidate blocks. HISA preserves the identical token-level top-sparse pattern consumed by the downstream Sparse MLA operator and requires no additional training. On kernel-level benchmarks, HISA achieves up to speedup at 64K context. On Needle-in-a-Haystack and LongBench, we directly replace the indexer in DeepSeek-V3.2 and GLM-5 with our HISA indexer, without any finetuning. HISA closely matches the original DSA in quality, while substantially outperforming block-sparse baselines.

全文与资源链接

Full-Text & Resources