跳转至

文章背景与核心概要

多模态大语言模型（MLLMs）在面对利用跨模态交互（涵盖文本、图像和音频）的恶意查询时，其安全对齐常常会遭到破坏。为解决这一漏洞，且无需依赖外部分类器、辅助输出头或模态特定的微调，作者们推出了 CASA（Classification Augmented with Safety Attention，基于安全注意力增强分类）。

CASA 是一种条件解码策略，它在响应生成之前，利用 MLLM 的内部表征预测二进制安全标记。在安全注意力机制的引导下——该机制计算专用查询提示与输入之间的注意力——CASA 显著提升了模型检测恶意提示的能力。

在涵盖 7 种攻击类型以及 MM-SafetyBench、JailbreakV-28k 和对抗性音频测试等基准的评估中，CASA 取得了卓越的性能： * 当与标准安全提示配合使用时，平均攻击成功率实现了 >90% 的相对降低。 * 仅使用其内部安全机制即可实现 54–61% 的降幅。 * 通过自动化评估和人工标注（由 13 名受过培训的标注员进行）验证，该方法在良性输入上仍保持了出色的实用性。

CASA: Classification Augmented with Safety Attention for Robust Multimodal Alignment

arXiv ID: arXiv:2604.00310
Subjects: Machine Learning (cs.LG), Artificial Intelligence (cs.AI)
Authors: Anurag Kumar, Raghuveer Peri, Jon Burnsky, Alexandru Nelus, Rohit Paturi, Srikanth Vishnubhotla, Yanjun Qi
Submission History: Submitted on March 31, 2026; Last revised on August 6, 2026 (v2).

📌 Summary

多模态大语言模型（MLLMs）经常在恶意查询利用跨模态交互（跨越文本、图像和音频）时遭受安全对齐受损的困扰。为了在不依赖外部分类器、辅助头或模态特定微调的情况下解决这一漏洞，作者引入了 CASA（Classification Augmented with Safety Attention，基于安全注意力增强分类）。

CASA 是一种条件解码策略，它在响应生成之前使用内部 MLLM 表征来预测二进制安全标记。在安全注意力机制的引导下——该机制计算专用查询提示与输入之间的注意力——CASA 增强了模型检测恶意提示的能力。

在跨越 7 种攻击类型以及 MM-SafetyBench、JailbreakV-28k 和对抗性音频测试等基准进行评估时，CASA 取得了显著的性能： * 当与标准安全提示配对时，平均攻击成功率实现了 >90% 的相对降低。 * 仅使用其内部安全机制即可实现 54–61% 的降低。 * 在良性输入上保持了强大的实用性，这已通过自动化评估和人工标注（通过 13 名受过培训的标注员）得到验证。

Multimodal large-language models (MLLMs) frequently suffer from compromised safety alignment when malicious queries exploit cross-modal interactions (spanning text, images, and audio). To address this vulnerability without relying on external classifiers, auxiliary heads, or modality-specific fine-tuning, the authors introduce CASA (Classification Augmented with Safety Attention).

CASA is a conditional decoding strategy that predicts a binary safety token prior to response generation using internal MLLM representations. Guided by a safety attention mechanism—which calculates attention between a dedicated query prompt and the input—CASA sharpens the model's capacity to detect malicious prompts.

When evaluated across 7 attack types and benchmarks like MM-SafetyBench, JailbreakV-28k, and adversarial audio tests, CASA achieves remarkable performance: * >90% relative reduction in average attack success rate when paired with a standard safety prompt. * 54–61% reduction using its internal safety mechanism independently. * Maintains strong utility on benign inputs, validated through both automated evaluation and human annotation (via 13 trained annotators).

📑 Abstract

多模态大语言模型（MLLMs）在有害查询利用跨模态交互时，常常会经历安全对齐退化。仅在文本上对齐的模型在扩展到两个或更多模态时，会表现出更高的成功攻击率。我们提出了一种简单的条件解码策略 CASA（Classification Augmented with Safety Attention），它利用 MLLM 的内部表征在响应生成之前预测二进制安全标记。该分类受安全注意力机制引导，该机制计算专用查询提示与输入之间的注意力，以缩放分类对数几率（logits），从而增强模型检测恶意查询的能力。我们的设计确保了鲁棒的安全对齐，而无需依赖任何外部分类器或辅助头，也无需进行模态特定的安全微调。与先前附加外部分类器或审核头、或者用奖励模型重新缩放每个生成标记的安全解码方法不同，CASA 在共享的多模态表征上做出单一的、内部的、受注意力门控的安全决策，这正是从纯文本训练中学到的安全性得以迁移到图像和音频输入的关键所在。与跨越文本、图像和音频模态的 7 种攻击类型的 4 个基准模型相比，在 MM-SafetyBench、JailbreakV-28k 和对抗性音频测试等基准上进行评估，当与标准安全提示配合使用时（应用于所有基准以进行公平比较），CASA 将平均攻击成功率相对降低了 >90%，而仅使用其内部安全机制则降低了 54--61%。我们的实证评估还表明，CASA 在良性输入上保持了强大的实用性，这通过自动化和人工评估（通过 13 名受过培训的标注员）得到了验证。总之，这些结果突显了 CASA 作为提高多模态 LLM 安全性的简单且可推广的框架。

Multimodal large-language models (MLLMs) often experience degraded safety alignment when harmful queries exploit cross-modal interactions. Models aligned on text alone show a higher rate of successful attacks when extended to two or more modalities. We propose a simple conditional decoding strategy, CASA (Classification Augmented with Safety Attention) that uses internal representations of MLLMs to predict a binary safety token before response generation. The classification is guided by a safety attention mechanism that computes attention between a dedicated query prompt and the input to scale the classification logits, sharpening the model's ability to detect malicious queries. Our design ensures robust safety alignment without relying on any external classifier or auxiliary head, and without modality-specific safety fine-tuning. Unlike prior safety-decoding methods that attach an external classifier or moderation head, or rescale every generated token with a reward model, CASA makes a single, internal, attention-gated safety decision over the shared multimodal representation, which is precisely what lets safety learned from text-only training transfer to image and audio inputs. Compared against 4 baseline models across 7 attack types spanning text, image, and audio modalities, evaluated on benchmarks such as MM-SafetyBench, JailbreakV-28k, and adversarial audio tests, CASA lowers the average attack success rate by >90% relative when paired with a standard safety prompt (applied to all baselines for fair comparison), and by 54--61% using its internal safety mechanism alone. Our empirical evaluations also show that CASA maintains strong utility on benign inputs, validated through both automated and human evaluations (via 13 trained annotators). Together, these results highlight CASA as a simple and generalizable framework to improve multimodal LLM safety.