跳转至

文章背景与核心概要

在多模态多项选择问答（MCQA）基准测试中，视觉语言模型（VLM）常常受到意料之外的文本偏差困扰。由于候选选项直接嵌入在输入上下文中，模型往往会倾向于选择语言学上合理的选项，而忽视了视觉证据的存在。

为了解决这一问题，研究人员推出了 BUZZY 这一无需训练的解码方法。BUZZY 的核心假设是：只有当模型的“多模态概率分布”与其“纯文本分布”发生显著偏离时，模型才真正依赖了视觉信息。通过减去纯文本分布来惩罚基于文本的先验，BUZZY 能够有效放大视觉信号。在五个 VLM 和五个基准测试上的实验表明，与先前的对比解码方法相比，BUZZY 在实现最先进的平均准确率的同时，将推理延迟降低了 28% 以上。

BUZZY: Contrastive Scoring to Mitigate Text-Induced Bias in Multimodal Multiple-Choice QA

摘要与概述

在多模态多项选择问答（MCQA）中评估视觉语言模型（VLM）时，模型往往会受到文本诱导偏差的影响。候选选项直接嵌入在输入上下文中，导致模型倾向于选择语言学上合理的选项，而忽视视觉证据。为此，本文提出了 BUZZY 这种无需训练的解码方法，通过惩罚纯文本先验来放大视觉信号。实验证明，该方法不仅达到了最先进的平均准确率，还将推理延迟降低了 28% 以上。

Vision-language models (VLMs) evaluated on multimodal multiple-choice question answering (MCQA) benchmarks often suffer from unintended textual biases. Because candidate choices are embedded directly into the input context, models tend to favor linguistically plausible options regardless of visual evidence.

To address this, researchers introduce BUZZY, a training-free decoding method. BUZZY operates on the hypothesis that models genuinely rely on visual information only when their multimodal probability distribution diverges significantly from their text-only distribution. By subtracting the text-only distribution to penalize text-based priors, BUZZY amplifies the visual signal. Experiments across five VLMs and five benchmarks demonstrate that BUZZY achieves state-of-the-art average accuracy while reducing inference latency by over 28% compared to previous contrastive decoding approaches.

元数据与文档信息

属性 详情
标识符 arXiv:2603.28026 [cs.AI]
主要学科 计算机科学 > 人工智能 (cs.AI)
作者 Taeyun Roh, Suhyeong Park, Dongyoung Lee, Eunyeong Jo, Wonjune Jang, Junha Jung, Jaewoo Kang
提交历史 2026年3月30日提交 (v1)；2026年8月14日最后修订 (v2)
DOI 10.48550/arXiv.2603.28026
Attribute Details
Identifier arXiv:2603.28026 [cs.AI]
Primary Subject Computer Science > Artificial Intelligence (cs.AI)
Authors Taeyun Roh, Suhyeong Park, Dongyoung Lee, Eunyeong Jo, Wonjune Jang, Junha Jung, Jaewoo Kang
Submission History Submitted on 30 Mar 2026 (v1); Last revised 14 Aug 2026 (v2)
DOI 10.48550/arXiv.2603.28026

摘要

多模态多项选择问答（MCQA）为评估视觉语言模型（VLM）提供了一个标准化且客观可测量的环境。然而，由于 MCQA 格式将候选选项纳入了输入上下文，它引入了若干非预期的偏差。先前的研究主要集中在结构偏差上（例如对特定选项的偏好）。与之不同，我们认为选项本身充当了文本先验，导致模型无论视觉内容如何，都倾向于选择语言学上合理的选项。我们提出并通过实验验证了一个假设：只有当模型的多模态分布与其纯文本分布显着偏离时，模型才真正依赖了视觉证据。基于这一观察，我们提出了 BUZZY，这是一种无需训练的解码方法，它通过减去纯文本分布来校正多模态预测。在五个多模态 MCQA 基准上对五个 VLM 进行的实验表明，与先前的对比解码方法相比，BUZZY 在现有最先进方法中实现了最高的平均准确率，同时将推理延迟降低了 28% 以上。总体而言，这些结果表明，通过惩罚纯文本偏好来放大视觉信号，是实现高效且鲁棒的多模态 MCQA 推理的关键。

Multimodal multiple-choice question answering (MCQA) provides a standardized and objectively measurable setting for evaluating vision-language models (VLMs). However, because the MCQA format incorporates the candidate choices into the input context, it introduces several unintended biases. Previous work has primarily focused on structural biases, such as preferences for certain choices. Instead, we argue that the choices act as textual priors, causing models to favor linguistically plausible options regardless of the visual content. We hypothesize and empirically verify that a model genuinely relies on visual evidence only when its multimodal distribution significantly diverges from its text-only distribution. Based on this observation we propose BUZZY, a training-free decoding method that corrects multimodal predictions by subtracting the text-only distribution. Experiments with five VLMs on five multimodal MCQA benchmarks demonstrate that BUZZY achieves the highest average accuracy among state-of-the-art methods while reducing inference latency by over 28% compared to prior contrastive decoding approaches. Overall, these results suggest that amplifying the visual signal by penalizing text-only preferences is key to efficient and robust multimodal MCQA reasoning.

访问与资源

外部参考与工具