文章背景与核心概要
指代表达式分割（Referring Expression Segmentation, RES）是一项基础且具有挑战性的任务，要求模型根据复杂的文本查询生成像素级的分割掩码。尽管多模态大语言模型（MLLMs）的引入显著提升了RES的性能，但也带来了巨大的计算瓶颈。如何在不损失精度的前提下对视觉Token进行压缩，一直是该领域的研究热点。
本文深入探讨了标准的Token压缩技术，并揭示了一个显着的性能下降现象。通过大量实验，作者发现RES的Token压缩必须严格保留原始位置嵌入以及局部相邻的空间结构，这表明视觉位置信息对于该任务具有独特且至关重要的作用。
为了解决这一问题，作者推出了 PAYN（Position Is All You Need），这是一种即插即用、无需训练且仅依赖位置信息的Token压缩方法。PAYN通过保留充分分布在局部相邻区域中的Token并维持原始位置索引，从而保持了空间关系的一致性。在多个基准测试上的实验证明，PAYN在无需重新训练的情况下，超越了现有的各种Token压缩策略。
Position Is All You Need: A Free Lunch Token Compression Strategy for MLLM-based Referring Expression Segmentation
Position Is All You Need: A Free Lunch Token Compression Strategy for MLLM-based Referring Expression Segmentation
- Authors: Yuhan Liu, Yixiong Zou, Yuhua Li, Ruixuan Li
- Subjects: Computation and Language (
cs.CL); Artificial Intelligence (cs.AI) - Conference/Status: Accepted by ICML 2026
- arXiv ID: arXiv:2608.26142 [cs.CL]
- DOI: 10.48550/arXiv.2608.26142
- Code Repository: GitHub - PAYN
- Submission Date: June 26, 2026
- Authors: Yuhan Liu, Yixiong Zou, Yuhua Li, Ruixuan Li
- Subjects: Computation and Language (
cs.CL); Artificial Intelligence (cs.AI)- Conference/Status: Accepted by ICML 2026
- arXiv ID: arXiv:2608.26142 [cs.CL]
- DOI: 10.48550/arXiv.2608.26142
- Code Repository: GitHub - PAYN
- Submission Date: June 26, 2026
📌 Summary
📌 Summary
Referring Expression Segmentation (RES) involves generating pixel-wise segmentation masks based on complex textual queries. Although Multimodal Large Language Models (MLLMs) enhance RES performance, they introduce a significant computational bottleneck.
Referring Expression Segmentation (RES) involves generating pixel-wise segmentation masks based on complex textual queries. Although Multimodal Large Language Models (MLLMs) enhance RES performance, they introduce a significant computational bottleneck.
This paper investigates standard token compression techniques and uncovers a notable performance drop. Through extensive experiments, the authors discover that RES token compression requires preserving original position embeddings and local neighboring spatial structures, showing that visual position information is uniquely critical for this task.
This paper investigates standard token compression techniques and uncovers a notable performance drop. Through extensive experiments, the authors discover that RES token compression requires preserving original position embeddings and local neighboring spatial structures, showing that visual position information is uniquely critical for this task.
To address this, the authors introduce PAYN (Position Is All You Need), a plug-and-play, training-free token compression method reliant strictly on position information. PAYN maintains spatial relational consistency by retaining tokens adequately distributed across local neighboring regions while preserving original positional indices. Experiments on multiple benchmarks confirm that PAYN outperforms existing token compression strategies without requiring retraining.
To address this, the authors introduce PAYN (Position Is All You Need), a plug-and-play, training-free token compression method reliant strictly on position information. PAYN maintains spatial relational consistency by retaining tokens adequately distributed across local neighboring regions while preserving original positional indices. Experiments on multiple benchmarks confirm that PAYN outperforms existing token compression strategies without requiring retraining.
🔗 Full-Text & Resources
🔗 Full-Text & Resources