跳转至

文章背景与核心概要

人类观察者在自然视觉观察中会根据具体的任务目标自然地优先处理特定的视觉信息。然而，大多数计算视觉模型在训练时通常仅采用纯粹的自由观看（free viewing）模式，且缺乏眼动（gaze）监督数据。这引发了一个问题：在没有专门进行眼动监督训练的情况下，目标导向的注意力机制是否能够在大模型中自然涌现？

为了解答这一问题，本研究测试了现成的视觉语言模型（VLM）在面对数千个自然场景时的表现。研究发现，这些模型能够生成与人类高度一致、且具备目标导向特征的空间注意力优先级，从而为跨任务预测人类视觉注意力的走向提供了可扩展的高效工具。

Emergent Goal-Directed Attention in Large Vision-Language Models

Summary

Human observers naturally prioritize visual information based on specific task goals. However, most computational models of naturalistic viewing are trained purely for free viewing without gaze supervision. This study investigates whether goal-directed attention can spontaneously emerge in off-the-shelf Vision-Language Models (VLMs) without gaze-specific training. By testing models like Qwen3-VL-32B-Thinking and Gemma-4-26B-A4B-it on thousands of naturalistic scenes, researchers discovered that these models generate human-aligned, goal-directed spatial priorities, providing scalable tools for predicting visual attention across different tasks.

Human observers naturally prioritize visual information based on specific task goals. However, most computational models of naturalistic viewing are trained purely for free viewing without gaze supervision. This study investigates whether goal-directed attention can spontaneously emerge in off-the-shelf Vision-Language Models (VLMs) without gaze-specific training. By testing models like Qwen3-VL-32B-Thinking and Gemma-4-26B-A4B-it on thousands of naturalistic scenes, researchers discovered that these models generate human-aligned, goal-directed spatial priorities, providing scalable tools for predicting visual attention across different tasks.

Metadata

  • arXiv ID: arXiv:2609.05517 [cs.CV]
  • Subjects: Computer Vision and Pattern Recognition (cs.CV), Artificial Intelligence (cs.AI), Computation and Language (cs.CL)
  • Authors: Han Zhang
  • Submitted: August 31, 2026
  • DOI: 10.48550/arXiv.2609.05517
  • arXiv ID: arXiv:2609.05517 [cs.CV]
  • Subjects: Computer Vision and Pattern Recognition (cs.CV), Artificial Intelligence (cs.AI), Computation and Language (cs.CL)
  • Authors: Han Zhang
  • Submitted: August 31, 2026
  • DOI: 10.48550/arXiv.2609.05517

Abstract

人类观察者会根据任务目标来优先处理视觉信息。大多数关于自然视觉观看的计算模型都针对自由观看进行了眼动训练，这使得我们无法确定在缺乏眼动监督的系统中，目标导向的注意力是否能够涌现。

我们对两个现成的视觉语言模型（VLM）——Qwen3-VL-32B-Thinking 和 Gemma-4-26B-A4B-it——进行了测试，测试场景涵盖 4,887 个自然场景，并分别施加了视觉搜索（visual-search）和自由观看（free-viewing）的指令。我们将模型的预测结果与人类在相同任务、相同图像下的注视点进行了对比。

Human observers prioritize visual information according to task goals. Most computational models of naturalistic viewing are gaze-trained for free viewing, leaving open whether goal-directed attention can emerge in systems without gaze supervision.

We tested two off-the-shelf vision-language models (VLMs), Qwen3-VL-32B-Thinking and Gemma-4-26B-A4B-it, on 4,887 naturalistic scenes under visual-search and free-viewing instructions. Model predictions were compared with human fixations on the same images under corresponding tasks.

主要发现包括： * 目标一致性（Goal Alignment）： 在匹配的任务目标下，两个模型与人类注视点的契合度均高于目标不匹配的情况。 * 稳健的交叉特征（Robust Crossover）： 这种交叉现象甚至在目标缺席的场景中依然持续存在（此时的对齐无法用简单的视觉基础来解释），并且在解码器层（decoder-layer）的读出中清晰呈现。 * 语义基础与视觉基础（Semantic vs. Visual Grounding）： 在视觉搜索过程中，模型的思维轨迹植根于目标语义（target semantics）；而在自由观看时，则植根于视觉显著性（visual prominence）。

这些发现表明，通用视觉语言模型在没有经过特定眼动训练的情况下，能够生成与人类一致的目标导向空间优先级。这为目标导向注意力的理论提供了见解，并为跨任务预测人类视线落点提供了可扩展的工具。

Key findings include: * Goal Alignment: Both models aligned more closely with human fixations under matching goals than under mismatched goals. * Robust Crossover: This crossover persisted even in target-absent scenes (where alignment could not be explained by simple visual grounding) and appeared clearly in decoder-layer readouts. * Semantic vs. Visual Grounding: Model-thinking traces were grounded in target semantics during visual search and in visual prominence during free viewing.

These findings show that general-purpose VLMs can generate human-aligned, goal-directed spatial priorities without gaze-specific training, informing theories of goal-directed attention and offering scalable tools for predicting where people look across tasks.

Access & Resources