理解大语言模型免训练低秩压缩中的校准与截断误差传播

文章背景与核心概要

随着大语言模型（LLM）规模的不断扩大，如何在保持任务准确性的前提下高效压缩模型参数已成为研究热点。免训练（Training-free）低秩压缩框架因其无需昂贵的微调过程而备受关注，但现有的主流方法在实际应用中往往面临严重的误差传播问题，导致压缩后的模型性能显著下降。

本文深入分析了导致性能衰减的两个核心机制：校准数据激活失准以及层重要性假设的失效。针对这些问题，作者提出了一种轻量级的免训练方法，通过“逐层压缩与校准修正”以及“迭代压缩与秩分配修正”机制，有效缓解了压缩过程中的误差累积。实验结果表明，该方法在 Llama 和 Qwen3 模型上实现了显著的性能提升，为大模型的轻量化部署提供了新的技术路径。

摘要

Training-free low-rank compression frameworks have been gaining prominence for LLM compression given their effectiveness in reducing model parameter count while maintaining task-level accuracy. However, existing SOTA frameworks share two key limitations:

Residual errors in calibration data activations accumulate across layers during compression, causing misalignment between representations simulated at compression time and those experienced at inference;

The assumption that layer importance distribution is preserved post-compression does not hold.

Together, these two effects introduce misalignment in the compression process in relation to the deployed model. We study these effects and propose a simple, training-free methodology compatible with existing frameworks to mitigate them, comprising: (1) Layer-by-Layer Compression with Calibration Correction; (2) Iterative Compression with Rank Allocation Correction. Implemented atop an existing SOTA decomposition framework, and evaluated on Llama and Qwen3 models across various benchmarks and compression rates, our approach demonstrates up to ~1-2.5 accuracy point improvements over per-weight and joint decomposition baselines on zero-shot tasks.

免训练低秩压缩框架因其在减少模型参数数量的同时保持任务级准确性的有效性，在大语言模型（LLM）压缩领域日益受到重视。然而，现有的最先进（SOTA）框架存在两个关键局限性：

校准数据激活中的残差在压缩过程中逐层累积，导致压缩时模拟的表征与推理时实际观察到的表征之间出现失准；
关于层重要性分布在压缩后保持不变的假设在实践中并不成立。

这两种效应共同导致了压缩过程与实际部署模型之间的偏差。我们研究了这些效应，并提出了一种与现有框架兼容的简单免训练方法来缓解这些问题，包括：（1）逐层压缩与校准修正；（2）迭代压缩与秩分配修正。通过在现有的 SOTA 分解框架上实现，并在 Llama 和 Qwen3 模型上针对各种基准测试和压缩率进行评估，我们的方法在零样本任务中比现有的逐权重和联合分解基线提升了约 1-2.5 个准确率百分点。

拟议方法论

为了对抗误差传播和表征失准，本文提出的方法引入了两个核心机制：

To combat error propagation and representation misalignment, the proposed approach introduces two core mechanisms:

Layer-by-Layer Compression with Calibration Correction: Dynamically updates and corrects calibration activations sequentially across layers to ensure that compression-time simulations closely mirror actual inference dynamics.

Iterative Compression with Rank Allocation Correction: Continuously re-evaluates and adapts the allocation of ranks based on shifting post-compression layer sensitivities, dropping the flawed assumption of static layer importance.

逐层压缩与校准修正： 在各层之间动态更新并顺序校准激活值，以确保压缩时的模拟过程能够紧密反映实际的推理动态。
迭代压缩与秩分配修正： 基于压缩后层敏感度的变化，持续重新评估并调整秩的分配，从而摒弃了层重要性保持不变这一有缺陷的假设。