目标感知的校准数据选择：在量化语言模型中保持不确定性
文章背景与核心概要
大语言模型的量化是实现高效部署的标准做法，但它经常会破坏模型的不确定性行为，例如置信度、边界和拒绝回答能力。传统的量化技术通常侧重于最大化面向准确率的压缩指标，或者事后调整分数输出。
本文引入了疑虑保持量化（Doubt-Preserving Quantization, DPQ），这是一种轻量级的量化前置方法，旨在将校准数据选择视为一个依赖于目标的、不确定性保持的问题。通过利用全精度预测来构建由高疑虑样本和通用锚点组成的目标对齐校准混合集，DPQ 确保模型能够保持下游任务所需的特定不确定性行为。
摘要与元数据详情
(Summary & Metadata Details)
Quantizing large language models (LLMs) is a standard practice for efficient deployment, but it frequently disrupts uncertainty behaviors such as confidence, margins, and abstention. Traditional quantization techniques usually focus on maximizing accuracy-oriented compression metrics or adjusting score outputs post-hoc.
This paper introduces Doubt-Preserving Quantization (DPQ), a lightweight pre-quantization method designed to treat calibration-data selection as a target-dependent uncertainty-preservation problem. By leveraging full-precision predictions to construct target-aligned calibration mixtures of high-doubt examples and generic anchors, DPQ ensures that models preserve the specific uncertainty behaviors required for downstream tasks.
arXiv ID: 2608.21019 Primary Subject: Computation and Language (
cs.CL) Secondary Subject: Artificial Intelligence (cs.AI) Submission Date: August 21, 2026 Conference Acceptance: Accepted to EMNLP Findings 2026 Authors: Zhen Yang, Sizai Hou, Kaiwen Zheng, Yaofang Liu, Liang He, Yixuan Chen, Kangning Cui
核心贡献与方法论
(Key Contributions & Methodology)
- Uncertainty as a Primary Objective: Formalizes uncertainty preservation during quantization using distributional and boundary preservation risks, shifting the paradigm away from purely accuracy-driven compression metrics.
- Mixture-Mismatch Analysis: Explains why a single calibration recipe cannot universally fit all targets, demonstrating that different deployment scenarios emphasize distinct regions of the input distribution.
- Doubt-Preserving Quantization (DPQ): Introduces a pre-quantization recipe family that combines high-doubt examples with generic anchors using full-precision model predictions.
- 以不确定性为首要目标： 利用分布和边界保持风险，将量化过程中的不确定性保持进行了形式化，将研究范式从纯粹由准确率驱动的压缩指标转变开来。
- 混合不匹配分析： 解释了为什么单一的校准方案无法普遍适应所有目标，证明了不同的部署场景侧重于输入分布的不同区域。
- 疑虑保持量化（DPQ）： 引入了一个量化前置方案系列，利用全精度模型预测将高疑虑样本与通用锚点结合起来。
实验洞察
(Experimental Insights)
Evaluated across 8 language models, 9 NLP benchmarks, and 22 comparison methods, the study reveals that the optimal calibration recipe depends entirely on the target preservation goal: * DPQ-r75 excels at preserving answerability boundaries (e.g., on SQuAD2). * Milder or single-signal variants (such as DPQ-r50, confidence-only, and entropy-only metrics) perform better for broad multiple-choice QA tasks.
These findings highlight that calibration data selection should be tailored directly to the specific score behavior a deployment demands, rather than treated as a static detail in the quantization pipeline.
通过对 8 个语言模型、9 个 NLP 基准以及 22 种对比方法的评估，该研究表明最优的校准方案完全取决于目标保持的目标： * DPQ-r75 在保持可回答性边界方面表现出色（例如在 SQuAD2 上）。 * 更温和或单一信号的变体（例如 DPQ-r50、纯置信度和纯熵指标）在广泛的多选题问答任务中表现更好。
这些发现强调，校准数据的选择应该根据部署所需的特定分数行为进行定制，而不应被视为量化流水线中的静态细节。
全文与资源
(Full-Text & Resources)