跳转至

文章背景与核心概要

多模态大语言模型（MLLMs）在充当多步骤智能体时，往往会面临“文本负债”（textual debt）的问题——即海量积累的自生成推理和工具调用文本占据了上下文窗口的主导地位，从而压制了关键的视觉证据。为了解决这一痛点，本文作者引入了 SPARE（一种基于Kullback–Leibler散度引导的上下文剪枝框架）。SPARE利用紧凑的任务状态摘要作为特权诊断上下文，并采用在线策略自蒸馏（OPSD）技术，在不丢弃核心视觉证据或误导后续推理的前提下，安全地移除冗余文本。

在多步骤视觉工具调用基准测试中，SPARE成功剪枝了 37.89% 至 64.58% 的推理token，同时在现有的各种剪枝方法中取得了最高的平均准确率。这一优异的准确率-上下文权衡表明，减少文本主导地位能够恢复模型对视觉证据的依赖，并减轻对自生成语言的过度条件依赖。

Buried in Textual Debt: Context Pruning with Visual Evidence Preservation for MLLM Agents

arXiv: 2608.22963 [cs.AI]
Submitted: August 24, 2026
Authors: Yuchen Huang, Sijia Li, Jun Zhang, Yi R. Fung
Links: View PDF | HTML Version | TeX Source

arXiv: 2608.22963 [cs.AI]
Submitted: August 24, 2026
Authors: Yuchen Huang, Sijia Li, Jun Zhang, Yi R. Fung
Links: View PDF | HTML Version | TeX Source

📌 Summary

Multimodal Large Language Models (MLLMs) acting as multi-step agents often suffer from textual debt—a phenomenon where accumulated self-generated reasoning and tool-use text dominate the context window and suppress critical visual evidence. To solve this, the authors introduce SPARE, a Kullback–Leibler (KL)-guided context pruning framework. SPARE uses compact task-state summaries as privileged diagnostic contexts and employs on-policy self-distillation (OPSD) to safely remove redundant text without discarding essential visual evidence or misguiding future inference.

Across multi-step visual tool-use benchmarks, SPARE successfully prunes 37.89% to 64.58% of reasoning tokens while achieving the highest average accuracy among existing pruning methods.

📌 Summary

Multimodal Large Language Models (MLLMs) acting as multi-step agents often suffer from textual debt—a phenomenon where accumulated self-generated reasoning and tool-use text dominate the context window and suppress critical visual evidence. To solve this, the authors introduce SPARE, a Kullback–Leibler (KL)-guided context pruning framework. SPARE uses compact task-state summaries as privileged diagnostic contexts and employs on-policy self-distillation (OPSD) to safely remove redundant text without discarding essential visual evidence or misguiding future inference.

Across multi-step visual tool-use benchmarks, SPARE successfully prunes 37.89% to 64.58% of reasoning tokens while achieving the highest average accuracy among existing pruning methods.

📑 Abstract

Multimodal Large Language Models (MLLMs) are increasingly deployed as multi-step agents, where explicit reasoning supports task decomposition and tool coordination but also accumulates self-generated text. Over long trajectories, this text can dominate the context and suppress visual evidence, creating textual debt.

We observe that reasoning becomes redundant once task-relevant visual evidence is grounded, while stale hypotheses can misguide later inference when grounding remains uncertain. Pruning must therefore remove redundant text without discarding visual evidence.

We propose SPARE, a Kullback–Leibler (KL)-guided framework for pruning accumulated reasoning in multimodal tool-use agents. SPARE uses a compact task-state summary as privileged diagnostic context. For each candidate segment, it replays the same model under the original and summary-conditioned contexts. Reverse-KL divergence from on-policy self-distillation (OPSD) then tests whether the summary sufficiently covers the segment without disrupting future reasoning. We further fine-tune the summarizer with supervised fine-tuning (SFT), enabling more compact summaries, broader coverage, and more aggressive pruning.

Across multi-step visual tool-use benchmarks, SPARE achieves the highest average accuracy among pruning methods while removing 37.89–64.58% of reasoning tokens. This favorable accuracy–context trade-off shows that reducing textual dominance restores reliance on visual evidence and mitigates over-conditioning on self-generated language.

📑 Abstract

Multimodal Large Language Models (MLLMs) are increasingly deployed as multi-step agents, where explicit reasoning supports task decomposition and tool coordination but also accumulates self-generated text. Over long trajectories, this text can dominate the context and suppress visual evidence, creating textual debt.

We observe that reasoning becomes redundant once task-relevant visual evidence is grounded, while stale hypotheses can misguide later inference when grounding remains uncertain. Pruning must therefore remove redundant text without discarding visual evidence.

We propose SPARE, a Kullback–Leibler (KL)-guided framework for pruning accumulated reasoning in multimodal tool-use agents. SPARE uses a compact task-state summary as privileged diagnostic context. For each candidate segment, it replays the same model under the original and summary-conditioned contexts. Reverse-KL divergence from on-policy self-distillation (OPSD) then tests whether the summary sufficiently covers the segment without disrupting future reasoning. We further fine-tune the summarizer with supervised fine-tuning (SFT), enabling more compact summaries, broader coverage, and more aggressive pruning.

Across multi-step visual tool-use benchmarks, SPARE achieves the highest average accuracy among pruning methods while removing 37.89–64.58% of reasoning tokens. This favorable accuracy–context trade-off shows that reducing textual dominance restores reliance on visual evidence and mitigates over-conditioning on self-generated language.

📊 Metadata & Additional Information

  • Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
  • Cite as: arXiv:2608.22963 [cs.AI] (or arXiv:2608.22963v1 [cs.AI] for this version)
  • DOI: 10.48550/arXiv.2608.22963
  • Document Info: 14 pages, 2 figures, 4 tables

📊 Metadata & Additional Information

  • Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
  • Cite as: arXiv:2608.22963 [cs.AI] (or arXiv:2608.22963v1 [cs.AI] for this version)
  • DOI: 10.48550/arXiv.2608.22963
  • Document Info: 14 pages, 2 figures, 4 tables

license icon

license icon