跳转至

文章背景与核心概要

随着端侧语言模型在没有服务器端审核的情况下被部署到数亿台设备上，缺乏独立审计带来了显著的可靠性风险。本文对一个开发者可访问的端侧基础模型进行了可复现的审计。研究发现，该模型表现出“任务不对称的校准偏差”——在69%的错误前提问题上胡编乱造，同时拒绝了18%的良性提示词。最令人担忧的是，模型自我报告的置信度无法区分且趋于饱和，这使得用户无法区分“自信且正确”和“自信且错误”的输出。最后，该研究提供了一个与模型无关的审计协议和一个黑盒一致性包装器，在无需访问模型内部权重的情况下显著提高了可靠性。

Confidently Wrong, Silently So: Auditing Undetectable Failures of a Deployed On-Device Language Model

Authors: Shashwat Pandey, Satwik Pandey, Suresh Raghu
Date: August 24, 2026
Subject: Software Engineering (cs.SE); Artificial Intelligence (cs.AI)
arXiv ID: 2608.23663

Summary

随着端侧语言模型在没有服务器端审核的情况下部署到数亿台设备上，缺乏独立审计带来了显著的可靠性风险。本文对一个开发者可访问的端侧基础模型进行了可复现的审计。作者发现该模型表现出“任务不对称的校准偏差”——在 69% 的错误前提（false-premise）问题上胡编乱造，同时拒绝了 18% 的良性提示词。最令人担忧的是，模型自我报告的置信度毫无区分度且已饱和，这使得用户无法区分“自信且正确”与“自信且错误”的输出。研究最后提供了一个与模型无关的审计协议和一个黑盒一致性包装器，在不需要访问模型内部权重的情况下显著提高了可靠性。

As on-device language models are deployed to hundreds of millions of devices without server-side moderation, the lack of independent auditing poses significant reliability risks. This paper presents a reproducible audit of a developer-accessible on-device foundation model. The authors find that the model exhibits "task-asymmetric miscalibration"—confabulating on 69% of false-premise questions while refusing 18% of benign prompts. Most alarmingly, the model’s self-reported confidence is non-discriminative and saturated, making "confident-correct" and "confident-wrong" outputs indistinguishable to users. The study concludes by providing a model-agnostic audit protocol and a black-box consistency wrapper that significantly improves reliability without requiring internal model access.

Key Findings

1. 任务不对称的校准偏差

该模型的护栏（guardrails）在不同类型的输入上表现不一致： * 错误前提： 模型频繁胡编乱造（失败率为 69%）。 * 良性输入： 模型表现出过度拒绝（失败率为 18%）。

1. Task-Asymmetric Miscalibration

The model's guardrails function inconsistently across different types of inputs: * False Premises: The model frequently confabulates (69% failure rate). * Benign Inputs: The model exhibits over-refusal (18% failure rate).

2. “表面不可区分”问题

作者发现，模型内部的置信度分数对于监督监督形同虚设。基于 15 个用户可见特征训练的分类器，其区分正确与错误输出的 AUROC 仅为 0.55，这表明故障在推理时本质上是“悄无声息”且无法检测的。

2. The "Surface-Indistinguishability" Problem

The authors discovered that the model’s internal confidence scores are effectively useless for oversight. A classifier trained on 15 user-visible features could only separate correct from incorrect outputs with an AUROC of 0.55, indicating that failures are essentially "silent" and undetectable at inference time.

3. 提出的缓解方案

研究人员提出了一种黑盒一致性包装器（black-box consistency wrapper），该方法不需要访问模型的内部权重或概率。这一方案带来了显著的改进： * 自信的胡编乱造（Confident Confabulation）： 从 75% 降至 3% 。 * 选择性准确率（Selective Accuracy）： 从 43% 提升至 83% 。

3. Proposed Mitigation

The researchers propose a black-box consistency wrapper that does not require access to the model's internal weights or probabilities. This approach yields significant improvements: * Confident Confabulation: Reduced from 75% to 3%. * Selective Accuracy: Increased from 43% to 83%.

Research Contributions

  • 审计协议： 一个用于评估已部署端侧模型的新型、与模型无关的框架。
  • 表面不可区分性测试： 一种用于确定模型故障是否能被最终用户或开发者检测到的方法。
  • 可复用基础设施： 作者开源了他们的代码以及一组冻结的评估条目，以推动对边缘部署 AI 可靠性的进一步研究。

Research Contributions

  • Audit Protocol: A new, model-agnostic framework for evaluating deployed on-device models.
  • Surface-Indistinguishability Test: A methodology for determining if model failures can be detected by end-users or developers.
  • Reusable Infrastructure: The authors have released their code and a set of frozen evaluation items to facilitate further research into the reliability of edge-deployed AI.

Metadata & Access

Metadata & Access