跳转至

文章背景与核心概要

当前的语音AI应用在处理大语言模型（LLM）对话时，通常将对话上下文视为平铺直叙且不断增长的文本序列，从而忽略了关键的语音特定数据（如语速、背景噪声和丢包情况）。这些语音特征往往隐式地包含在音频中，导致系统在长对话中出现响应错位、环境适应性差以及成本不断攀升的问题。

为了解决这些挑战，本文作者推出了 llmovoice——一种新颖的上下文管理中间件。通过在每个对话轮次中显式建模语音上下文，llmovoice 能够动态构建有界上下文以生成运行时指令。评估结果表明，llmovoice 显著提升了交互质量，将语速对齐误差降低了 52.4%，在丢包情况下将错误打断率从 46.0% 降至 0.9%，并将整体模型使用成本削减了 79.2%。

Scalable Context Orchestration for Serving LLMs Over Voice

Authors: Linyi Jiang, Silvery D. Fu, Yifei Zhu
Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Audio and Speech Processing (eess.AS)
Publication: Accepted for publication in ACM SOSP 2026
arXiv Identifier: arXiv:2609.04288

📌 Summary

Voice AI applications built on Large Language Models (LLMs) often struggle because they treat conversation contexts as flat, growing sequences of text. This ignores crucial voice-specific data—such as speaking rate, background noise, and packet loss—which remains trapped implicitly in the audio. Consequently, these systems suffer from misaligned responses, poor environmental adaptability, and escalating costs during long sessions.

To address these challenges, the authors introduce llmovoice, a novel context-management middleware. By explicitly modeling voice context at every conversational turn, llmovoice dynamically constructs bounded contexts to generate runtime directives. Evaluations show that llmovoice significantly improves interaction quality, reducing speaking-rate alignment errors by 52.4%, lowering false-interruption rates from 46.0% to 0.9% under packet loss, and cutting overall model usage costs by 79.2%.

📖 摘要

Voice AI applications are gaining popularity as advances in large language models (LLMs) enable more natural and accessible spoken interactions. Serving these applications requires accounting not only for what users say, but also for how they speak (e.g., speaking rate) and the conditions under which their audio is captured and transmitted (e.g., background noise and packet loss). However, existing LLM systems represent conversation context as a flat, growing sequence of messages, leaving voice-specific context implicit in the audio. As a result, they can generate responses that are poorly aligned with user preferences, degrade interaction quality under adverse environmental conditions, and incur high costs over long voice sessions.

We present llmovoice, a context-management middleware that explicitly models voice context and orchestrates its use. At each turn, llmovoice constructs a bounded voice context from the current user input, relevant interaction history, and explicit paralinguistic and environmental states. It then uses the serving LLM to reason over this context and generate runtime directives that guide how the system responds. We evaluate llmovoice on real-world voice applications and benchmarks. It reduces speaking-rate alignment error by 52.4%, lowers the false-interruption rate from 46.0% to 0.9% under packet loss, and reduces model usage cost by 79.2%. For long sessions, llmovoice reduces per-turn cost by up to 24.9 times while retaining up to 98.7% of baseline answer quality.

🔗 访问链接与资源

📚 参考文献与引用