跳转至

语音大模型会听从自己的观点吗？诊断并缓解流式情感理解中的先验信念污染

文章背景与核心概要

流式语音大模型（SpeechLMs）在处理实时音频时，通常会将历史预测结果反馈回模型以维持上下文连贯性。然而，本文研究发现这种机制会导致严重的“先验信念污染”（Previous-Belief Contamination, PBC），即模型过度依赖自身的历史输出，从而忽视了当前的实际音频证据。

为了解决这一问题，作者提出了 EmoUpdate 框架。该框架无需重新训练模型，通过引入先验盲声学防火墙、因果信念过滤器以及闭式去污染算子，成功实现了当前声学感知与历史信念更新的解耦。实验结果表明，EmoUpdate 在多个流式情感理解基准测试中显著提升了预测准确率，有效缓解了模型对历史预测的盲目依赖。

1. 引言与问题陈述

流式情感理解要求模型在持续解释输入音频的同时，兼顾历史对话状态。一种常见的架构方法是历史条件化（history conditioning），即将模型先前的预测结果注入上下文窗口，作为后续步骤的参考。

Streaming emotion understanding requires models to continuously interpret incoming audio while factoring in historical conversational states. A common architectural approach is history conditioning, where a model's prior prediction is injected back into the context window as a guide for subsequent steps.

虽然这种做法旨在提供时间上的连贯性，但它引入了一个关键漏洞：模型可能会过度受到自身过去输出的偏见影响，从而覆盖音频流中真实的声学证据。

While intended to provide temporal coherence, this practice introduces a critical vulnerability: models can become overly biased by their own past outputs, overriding actual acoustic evidence in the audio stream.

2. 诊断：先验信念污染 (PBC)

为了评估这种效应，作者部署了一个平衡的 CREMA-D-Stream 反事实诊断： * 测试方法： 在保持音频输入完全固定的情况下，仅人工改变注入的前一个情感标签。 * 发现： 改变先验标签会导致当前音频的准确率从 72.50% 骤降至 30.42%，导致 65.69% 的总预测结果发生翻转。 * 标签不对称性： 先验信念的影响具有高度不对称性，根据具体情感标签的不同，影响范围从 4.76% 到 98.20% 不等。这种失效模式被定义为 先验信念污染 (PBC)。

To evaluate this effect, the authors deploy a balanced CREMA-D-Stream counterfactual diagnostic: * The Test: Holding the audio input completely fixed while artificially altering only the injected previous emotion label. * The Finding: Changing the prior label causes current-audio accuracy to plummet from 72.50% down to 30.42%, flipping 65.69% of total predictions. * Label Asymmetry: The pull of prior beliefs is heavily asymmetric, ranging wildly from 4.76% to 98.20% depending on the specific emotion label. This failure mode is defined as Previous-Belief Contamination (PBC).

3. 提出的解决方案：EmoUpdate

为了以无需训练的方式缓解 PBC，作者提出了 EmoUpdate，这是一个通过三个核心组件将当前音频感知与历史状态修正解耦的框架：

  1. 先验盲声学防火墙（Prior-Blind Acoustic Firewall）： 防止历史状态上下文过早泄露到当前的感知流水线中。
  2. 证据收缩因果信念过滤器（Evidence-Shrunk Causal Belief Filter）： 仅在观察形成后才引入历史上下文，仅在有经验证据支持时才保留标签不对称的转换。
  3. 闭式去污染算子（Closed-Form Decontamination Operator）： 从反事实测量中通过数学推导得出，专门为无法使用防火墙机制的部署栈而设计。

To mitigate PBC in a training-free manner, the authors propose EmoUpdate, a framework that decouples current-audio perception from historical state revision via three core components:

  1. Prior-Blind Acoustic Firewall: Prevents historical state contexts from prematurely leaking into the current perception pipeline.
  2. Evidence-Shrunk Causal Belief Filter: Introduces historical context only after observation formation has taken place, retaining label-asymmetric transitions strictly when supported by empirical evidence.
  3. Closed-Form Decontamination Operator: Derived mathematically from counterfactual measurements, designed specifically for deployment stacks where firewalling mechanisms are unavailable.

4. 实验结果

在 四个语音大模型 和 两个流式情感基准测试（涵盖所有八种模型-基准设置）中进行评估，EmoUpdate 实现了最先进的性能： * 将 状态平衡准确率 (S-BAcc) 提升了高达 69.71 个百分点。 * 与最强的受控基线相比，步长准确率 (Step Accuracy) 提升了高达 38.41 个百分点。

Evaluated across four SpeechLMs and two streaming emotion benchmarks (spanning all eight model-benchmark settings), EmoUpdate achieves state-of-the-art performance: * Improves State-Balanced Accuracy (S-BAcc) by up to 69.71 points. * Enhances Step Accuracy by up to 38.41 points over the strongest controlled baselines.

5. 链接与资源