问两次，看两次：提示词回显解决了视觉语言模型中的“问题前置悖论”
文章背景与核心概要
本文深入探讨了视觉语言模型（VLM）中的“问题前置悖论”（question-first paradox）：直觉上，在展示图像之前先提出问题应该能帮助模型知道该往哪里看，然而在视觉问答基准测试中，问题前置的提示词表现却始终不如图像前置。作者追踪发现，这源于一个根本性的计算冲突：虽然将问题放在前面可以成功引导视觉感知（使图像块表征向与问题相关的概念偏移），但在因果解码器的下游，问题被困在数百个图像Token后面，导致最终的答案Token几乎无法访问它，从而只能默认输出由图像驱动的（且通常是错误的）答案。
为了在不进行训练、微调或架构修改的前提下解决这一问题，作者提出了提示词回显（prompt echoing）——在图像的两侧都重述问题，这样一份拷贝负责引导感知，另一份拷贝则在回答时保持可用。同时回显图像还能带来进一步的准确率提升。在多个基准测试（包括 NaturalBench 和 Winoground）以及开源 VLM 上的实验表明，回显提示词有效地弥补了性能差距，并且经常超越标准的提示词策略。
摘要 (Summary)
This paper investigates the "question-first paradox" in Vision-Language Models (VLMs): while intuition suggests that asking a question before showing an image should help the model know where to look, question-first prompting consistently underperforms image-first prompting on visual question-answering benchmarks.
本文研究了视觉语言模型（VLM）中的“问题前置悖论”：直觉上，在展示图像之前提出问题应该能帮助模型知道该往哪里看，然而在视觉问答基准测试中，问题前置的提示词表现却始终不如图像前置。
The authors trace this issue to a fundamental computational conflict: 1. Placing the question first successfully steers visual perception (shifting image patch representations toward question-relevant concepts). 2. However, downstream in causal decoders, the question gets stranded behind hundreds of image tokens, making it barely accessible to the final answer token, which then defaults to image-driven (and often incorrect) answers.
作者将此问题归结为两个计算阶段之间的根本冲突： 1. 将问题置于首位成功地引导了视觉感知（将图像补丁表征向与问题相关的概念偏移）。 2. 然而，在因果解码器的下游，问题被滞留在数百个图像 Token 后面，导致最终的答案 Token 几乎无法访问它，从而使模型默认输出由图像驱动的（且通常是错误的）答案。
To resolve this without training, fine-tuning, or architecture changes, the authors propose prompt echoing—restating the question on both sides of the image so one copy steers perception while the other remains available at answer time. Echoing the image as well yields even further accuracy gains. Across multiple benchmarks (including NaturalBench and Winoground) and open VLMs, echoed prompts effectively close the performance gap and often surpass standard prompting strategies.
为了在无需训练、微调或架构修改的情况下解决这个问题，作者提出了提示词回显（prompt echoing）——在图像的两侧都重述问题，这样其中一份拷贝用于引导感知，另一份拷贝在回答时保持可用。同时回显图像还可以带来更高的准确率提升。在多个基准测试（包括 NaturalBench 和 Winoground）以及开源 VLM 的测试中，回显提示词有效地缩小了性能差距，并且通常超越了标准的提示词策略。
论文元数据 (Paper Metadata)
- arXiv Identifier: arXiv:2607.15565 [cs.CV]
- Subjects: Computer Vision and Pattern Recognition (
cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Image and Video Processing (eess.IV) - Authors: Rakshanda Hassan Abhinandan, John Galeotti, Deva Ramanan, Gautam Rajendrakumar Gare
- Submitted: 17 July 2026 (v1); Last revised: 3 September 2026 (this version, v2)
- Conference Context: Accepted at the eXCV Workshop, ECCV 2026
- Project Page & Resources: Project Website
- arXiv 标识符: arXiv:2607.15565 [cs.CV]
- 学科分类: 计算机视觉与模式识别 (
cs.CV); 人工智能 (cs.AI); 机器学习 (cs.LG); 图像和视频处理 (eess.IV)- 作者: Rakshanda Hassan Abhinandan, John Galeotti, Deva Ramanan, Gautam Rajendrakumar Gare
- 提交时间: 2026年7月17日 (v1)；最后修订：2026年9月3日 (当前版本 v2)
- 会议背景: 被 ECCV 2026 eXCV Workshop 接受
- 项目页面与资源: 项目网站
摘要 (Abstract)
Where should the question go in a vision-language model (VLM) prompt: before the image or after it? Intuition says before: knowing what is asked should tell the model where to look. Yet across visual question answering benchmarks, question-first prompting consistently underperforms the image-first ordering recommended for frontier VLMs, a phenomenon we term the question-first paradox.
在视觉语言模型（VLM）的提示词中，问题应该放在哪里：图像之前还是图像之后？直觉告诉我们应该在前面：知道被问什么应该能告诉模型该看哪里。然而，在各个视觉问答基准测试中，问题前置的提示词表现始终不如前沿 VLM 所推荐的图像前置顺序，我们将这种现象称为“问题前置悖论”。
We trace this paradox to a conflict between two stages of VLM computation. Logit-lens and attention probes show that question-first prompting steers perception, shifting image patch representations toward question-relevant concepts. But downstream, stranded behind hundreds of image tokens, the question is barely attended by the answer token, which instead commits to image-driven, often wrong answers. Causal attention knockout confirms that the answer reads the question only when it follows the image.
我们将这一悖论追溯到 VLM 计算的两个阶段之间的冲突。Logit-lens 和注意力探针显示，问题前置的提示词确实引导了感知，将图像补丁表征向与问题相关的概念偏移。但在下游，由于被滞留在数百个图像 Token 后面，答案 Token 几乎无法对该问题给予注意力，从而转而提交由图像驱动的、通常是错误答案。因果注意力敲除（Causal attention knockout）证实，只有当问题跟在图像后面时，答案才能读取到问题。
This diagnosis yields a training-free fix: question echoing, restating the question on both sides of the image so one copy steers perception while the other is available at answer time. A similar division of labor appears in a fifty-year-old finding on human 'adjunct questions', where repeating a question before and after a passage improves comprehension. Echoing the image as well brings further gains by restoring the whole-image view otherwise lost by a causal decoder.
这一诊断带来了无需训练的修复方案：问题回显（question echoing），即在图像的两侧重述问题，使一份拷贝用于引导感知，另一份拷贝在回答时可用。关于人类“附加问题（adjunct questions）”的一项拥有五十年历史的研究中也出现了类似的劳动分工，即在段落前后重复问题可以提高理解力。同样地，回显图像通过恢复因果解码器原本丢失的全景图像视图，带来了进一步的性能提升。
The paradox holds across five open VLMs, costing up to 17.5 group-accuracy points. Echoed prompts recover most of the gap and, on NaturalBench and Winoground, surpass the best single-pass ordering by up to 19 group-accuracy points on Winoground, with no training, fine-tuning, or architecture change. The paradox reveals a tension between steering what a model sees and preserving access to what it was asked; echoing resolves this through prompt design.
该悖论存在于五个开源 VLM 中，导致高达 17.5 个群组准确率点数的损失。回显提示词恢复了大部分差距，并且在 NaturalBench 和 Winoground 上，在无需任何训练、微调或架构更改的情况下，比最佳单遍顺序在 Winoground 上高出多达 19 个群组准确率点数。该悖论揭示了引导模型“看到什么”与保持对“被问内容”的访问权限之间的张力；回显通过提示词设计解决了这一问题。
访问与全文链接 (Access & Full-Text Links)
引用 (Citation)
If you use this work in your research, please cite it via: * BibTeX Reference: Available directly through the arXiv Abs Page * External Citations: Google Scholar | Semantic Scholar | NASA ADS
如果您在研究中使用了这项工作，请通过以下方式引用： * BibTeX 参考: 可直接通过 arXiv 摘要页面 获取 * 外部引用: Google Scholar | Semantic Scholar | NASA ADS