跳转至

少即是多：究竟是何种语音特征驱动了话轮结束检测？

Less can be More: What Aspects of Speech Drive End-of-Turn Detection

arXiv:2609.11066 [eess.AS]
Subjects: Audio and Speech Processing (eess.AS); Artificial Intelligence (cs.AI); Sound (cs.SD)
Authors: Rini Sharon, Manickavela A, Kadri Hacioglu, Andreas Stolcke
Submitted: 10 September 2026
arXiv ID: arXiv:2609.11066 [eess.AS]
DOI: 10.48550/arXiv.2609.11066

文章背景与核心概要

在人机语音交互中，AI 能否准确判断说话人是否“已经说完”（即话轮结束检测，End-of-Turn Detection），是实现自然流畅对话的关键枢纽。过去行业普遍认为，结合文本语义与声学特征的多模态系统效果更佳，但这也带来了显著的计算延迟和语义依赖。本论文通过精细的模态消融实验，深入探究了声学、韵律与文本语义在流式话轮检测中的真实作用，惊人地发现：单纯结合“声学+韵律”特征不仅达到了 0.93 的 F1 分数与仅 400 毫秒的中位数延迟，而且加入文本语义反而会诱发更多抢话误判。这一发现颠覆了传统认知，证明人类日常对话的轮替更多是由语调起伏与停顿模式驱动，无需高昂的文本语义推理即可打造极速且稳健的对话系统。

📌 内容摘要

📌 Summary

在对话式 AI (Conversational AI) 领域，准确检测用户何时结束发言，对于实现自然流畅的轮流交谈（话轮转换，Turn-Taking）至关重要。尽管近年来的系统频繁引入文本语义数据，但不同语音模态各自究竟贡献了多少价值，此前学术界与工业界一直未能厘清。

In conversational AI, accurately detecting when a user has finished speaking is essential for natural, fluid turn-taking. While recent systems frequently incorporate semantic (text) data, the specific contributions of different speech modalities have remained unclear.

本论文基于一个轻量级三模态分类器，通过严谨的受控消融实验，深入探究了声学、韵律与语义信号在其中的具体作用。实验结果表明，声学与韵律的特征组合在准确率与延迟之间实现了最佳平衡——在 400 毫秒的中位数延迟下，取得了 0.93 的语句级 F1 分数，误报率仅为 7.8% 。出人意料的是，额外引入文本语义信号不仅未能改善整体性能，反而增加了过早切入（抢话）的误判几率。特征空间分析进一步证实，韵律特征具备极其出色的类别可分性，而文本表征则存在大面积重叠。这表明人类对话中的话轮转换主要由语调模式和停顿节奏驱动，而非句意在语义层面的完整性。

This paper investigates the role of acoustic, prosodic, and semantic signals using a controlled ablation study with a lightweight trimodal classifier. The findings reveal that acoustic-prosodic combinations achieve the optimal balance of accuracy and latency—matching an utterance F1 score of 0.93 with a 7.8% false alarm rate at a median latency of 400ms. Surprisingly, adding text signals increases premature detections without improving overall performance. Feature space analysis demonstrates that prosodic features offer robust class separability, whereas text representations heavily overlap, suggesting that turn-taking is primarily driven by intonation and silence patterns rather than semantic completeness.

📋 论文摘要

📋 Abstract

在对话式 AI 中，准确判断说话者何时结束发言是实现自然话轮转换的关键所在。尽管近期的研究逐渐融合了语义信息，但各模态之间的相对贡献度依然不够明朗。我们在流式话轮结束检测任务中，使用轻量级三模态分类器，对声学、韵律与语义信号开展了严格受控的消融实验。在完全相同的训练条件下，“声学+韵律”的组合在准确率与低延迟之间达成了最优平衡：在仅 400 毫秒的中位数延迟下，取得了 0.93 的语句级 F1 值与 7.8% 的极低误报率。与之相反，引入文本语义非但没有提升整体表现，反而显著增加了过早触发的误报次数。特征空间分析印证了这一现象：韵律特征展现出了最强的类别区隔能力，而文本表征之间则存在大量重叠。这些研究结论表明，话轮转换的线索主要是通过语调起伏与停顿模式传递的，而非依赖语义层面的完整度；这也为构建摆脱昂贵文本推理开销、反应更灵敏且更可靠的实时语音交互系统指明了新路径。

In conversational AI, detecting when a speaker has finished talking is crucial for natural turn taking. While recent work incorporates semantics, the relative contribution of different modalities remains unclear. We present a controlled ablation of acoustic, prosodic, and semantic signals for streaming end of turn detection using a lightweight trimodal classifier. Under identical training conditions, the acoustic prosodic combination achieves the best balance of accuracy and latency, achieving utterance F1 of 0.93 with 7.8% false alarms at 400ms median latency. Adding text increases premature detections without improving performance. Feature space analysis confirms that prosodic features have the strongest class separability, while text representations overlap substantially. These findings suggest that turn-taking is primarily conveyed through intonation and silence patterns rather than semantic completeness, enabling faster and more reliable systems without expensive text inference.

🔗 资源与相关链接