跳转至

REAL-Q：基于动态梯度下降的端到端大语言模型量化

REAL-Q: E2E LLM Quantization via Dynamic Gradient Descent

arXiv:2609.00049 [cs.LG]
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Authors: Qian Zhang, Yaoming Li, Zhewen Tan, Yanshu Wang, Heng Lu, Kun Su, Zongwei Lv, Wenhan Yu, Yongge Ma, Yinjun Han, Ruikang Liu, Tong Yang
Submitted: 30 Aug 2026; Last revised: 10 Sep 2026 (v2)
DOI / Paper: 10.48550/arXiv.2609.00049

文章背景与核心概要

训练后量化 (Post-Training Quantization, PTQ) 是在严苛算力与显存限制下低成本部署大语言模型 (Large Language Model, LLM) 的关键技术。然而，当前主流的 PTQ 方案为了数学求解上的便利，普遍采用大幅简化的单层二阶求解器，这不仅割裂了通道间的依赖关系，更因冻结黑塞矩阵 (Hessian) 而导致量化误差随网络深层逐级放大，产生严重的“信息失准”现象。

为此，本文提出了 REAL-Q (Real-time E2E-loss Aligned LLM Quantization) 创新量化范式。REAL-Q 突破了传统方法的分析妥协，直接针对与全局端到端损失对齐的代理目标函数进行优化；通过引入细粒度的动态块级梯度下降与跨层滑动窗口机制，在列块量化过程中动态刷新梯度并平滑阻断误差传递。在 LLaMA-3.1 与 Qwen3 系列模型上的 W4A16 精度评测表明，REAL-Q 相比现有顶尖全局引导基准将端到端 KL 散度降低了高达约 49%，实现了更出色的精度保留。

概述

Summary

训练后量化 (PTQ) 是在严苛显存与硬件限制下高效部署大语言模型 (LLM) 不可或缺的关键手段。然而，当前顶尖的 PTQ 方法通常依赖于针对各层的简化封闭形式二阶求解器。这种做法往往迫使算法做出严重的数学妥协，例如忽略跨通道的相关性、将输出行进行分组池化，以及在整层量化过程中始终冻结黑塞矩阵 (Hessian) ，导致其无法感知损失曲面随权重量化而发生的实时变化，我们称之为信息失准 (Information Misalignment) 问题。

Post-training quantization (PTQ) is a crucial technique for deploying large language models (LLMs) efficiently under strict memory and hardware constraints. Traditional state-of-the-art PTQ methods usually rely on simplified second-order solvers for individual layers, which often leads to major analytical compromises such as ignored cross-channel dependencies, grouped output rows, and frozen Hessians that fail to adapt as the loss landscape shifts (a problem termed information misalignment).

为了化解这些局限，本研究提出了 REAL-Q (Real-time E2E-loss Aligned LLM Quantization)。REAL-Q 摒弃了以往为了分析可行性而大幅稀释目标的做法，转而直接优化一个与全局端到端损失紧密对齐的代理目标函数。它在列块之间迭代应用细粒度、动态的块级梯度下降 (Block-wise Gradient Descent) ，并将其与平滑的跨层滑动窗口机制相结合，彻底阻断了量化误差在层间的无序累积。在 LLaMA-3.1 (8B 与 70B) 以及 Qwen3 (0.6B–32B) 模型上的 W4A16 精度评估表明，REAL-Q 相较于现有的全局引导基准方案，实现了高达 ~49% 的端到端 KL 散度降幅 。

To resolve these limitations, this paper introduces REAL-Q (Real-time E2E-loss Aligned LLM Quantization). REAL-Q bypasses the need for heavily diluted objectives by optimizing an end-to-end-aligned surrogate of the global loss function. It applies fine-grained, dynamic Block-wise Gradient Descent iteratively across column blocks and couples this with a smooth cross-layer sliding window mechanism to eliminate error propagation. Evaluated on popular models like LLaMA-3.1 (8B and 70B) and Qwen3 (0.6B–32B) at W4A16 precision, REAL-Q achieves up to a ~49% reduction in end-to-end KL divergence compared to current globally-guided baselines.

论文元数据

Paper Metadata

  • arXiv 编号： arXiv:2609.00049 [cs.LG]
  • 研究方向： 机器学习 (cs.LG)，人工智能 (cs.AI)
  • 提交日期：
  • 首次提交：2026年8月30日 (v1)
  • 最新修订：2026年9月10日 (v2)
  • 作者列表： Qian Zhang, Yaoming Li, Zhewen Tan, Yanshu Wang, Heng Lu, Kun Su, Zongwei Lv, Wenhan Yu, Yongge Ma, Yinjun Han, Ruikang Liu, Tong Yang
  • arXiv ID: arXiv:2609.00049 [cs.LG]
  • Subject Areas: Machine Learning (cs.LG), Artificial Intelligence (cs.AI)
  • Submission Dates:
  • Submitted: August 30, 2026 (v1)
  • Last Revised: September 10, 2026 (v2)
  • Authors: Qian Zhang, Yaoming Li, Zhewen Tan, Yanshu Wang, Heng Lu, Kun Su, Zongwei Lv, Wenhan Yu, Yongge Ma, Yinjun Han, Ruikang Liu, Tong Yang

摘要

Abstract

训练后量化 (PTQ) 对于在严格的资源限制下部署大语言模型 (LLM) 至关重要。目前最先进的 PTQ 方法通常采用单一的封闭形式二阶求解器来量化每一层：为了保证数学推导的可行性，它们对全局损失进行了大幅近似（舍弃了跨通道耦合、将输出行进行分组池化），并在整个网络层量化过程中彻底冻结求得的黑塞矩阵 (Hessian) ，致使模型无法随着列权重的逐列改变而动态刷新损失曲面，这一现象被称为“信息失准”。

Post-training quantization (PTQ) is essential for deploying large language models (LLMs) under strict resource constraints. State-of-the-art PTQ methods quantize each layer with a single closed-form second-order solver: to remain analytically tractable, they heavily approximate the global loss (dropping cross-channel coupling, pooling output rows into groups), and they then freeze the resulting Hessian across the entire layer, with no way to refresh it as the loss landscape shifts column by column--a phenomenon we call information misalignment.

我们提出了 REAL-Q (Real-time E2E-loss Aligned LLM Quantization)，这是一种打破这一妥协的新型 PTQ 范式：REAL-Q 不再为了解析上的易解性而稀释目标，而是瞄准全局损失的端到端对齐代理目标，并在每个列块（128 列）处理完毕后，通过细粒度、动态的块级梯度下降进行修正优化。通过将这种细粒度校正与实现平滑跨层过渡的滑动窗口机制相结合，REAL-Q 有效抑制了误差在全网络范围内的扩散传播。在 W4A16 精度的 LLaMA-3.1 (8B 和 70B) 以及 Qwen3 (0.6B-32B) 上，REAL-Q 相比最先进的全局引导方法，将端到端 KL 散度降低了高达约 49%。

We propose REAL-Q (Real-time E2E-loss Aligned LLM Quantization), a novel PTQ paradigm that breaks this compromise: instead of diluting the objective for the sake of analytic tractability, REAL-Q targets an end-to-end-aligned surrogate of the global loss and refines it via fine-grained, dynamic Block-wise Gradient Descent applied after every column block (128 columns). By coupling this fine-grained correction with a sliding window mechanism for smooth cross-layer transitions, REAL-Q effectively mitigates error propagation across the network. On LLaMA-3.1 (8B and 70B) and Qwen3 (0.6B-32B) at W4A16, REAL-Q reduces end-to-end KL divergence by up to ~49% relative to state-of-the-art globally-guided methods.

资源与访问链接

Access & Resources