跳转至

文章背景与核心概要

随着大语言模型规模的不断膨胀，高效的模型量化技术对于降低部署成本至关重要。传统的 GPTQ 类自适应舍入算法通常仅依赖单侧输入激活信息，而利用克罗内克因子化黑塞矩阵（Kronecker-factored Hessians）的双侧方法能够捕捉输出坐标之间的相关性，但直接在向量化权重域中应用时计算复杂度极高。

为了克服这一计算瓶颈，本文介绍了 BaKron 求解器。通过将反对角线并行化（anti-diagonal parallelism）与递归分治结构相结合，BaKron 成功将 \(m \times n\) 权重矩阵的总体计算复杂度从 \(O(m^2 n^2)\) 降低至 \(O(mn(m+n))\)。这不仅匹配了标准 GPTQ 的立方扩展性，还充分利用了更丰富的曲率信息，且该算法对基础量化器和黑塞矩阵估计器均具有高度的模块化。

BaKron: Efficient Quantization with Kronecker-Factored Hessians

BaKron: Efficient Quantization with Kronecker-Factored Hessians

Summary

Summary

BaKron is an efficient solver designed to accelerate neural network quantization algorithms that leverage Kronecker-factored Hessian approximations (such as BoA and YAQA). While traditional GPTQ-style adaptive rounding relies solely on one-sided input activation information, two-sided Kronecker-factored Hessians capture correlations across output coordinates—though applying them directly in the vectorized weight domain is usually computationally prohibitive.

BaKron is an efficient solver designed to accelerate neural network quantization algorithms that leverage Kronecker-factored Hessian approximations (such as BoA and YAQA). While traditional GPTQ-style adaptive rounding relies solely on one-sided input activation information, two-sided Kronecker-factored Hessians capture correlations across output coordinates—though applying them directly in the vectorized weight domain is usually computationally prohibitive.

BaKron solves this by combining anti-diagonal parallelism with a recursive divide-and-conquer construction. For an \(m \times n\) weight matrix, BaKron reduces the total computational complexity from \(O(m^2 n^2)\) down to \(O(mn(m+n))\), matching the cubic scaling of standard GPTQ while utilizing richer curvature information. Additionally, the algorithm is fully modular with respect to both the base quantizer and the Hessian estimator.

BaKron solves this by combining anti-diagonal parallelism with a recursive divide-and-conquer construction. For an \(m \times n\) weight matrix, BaKron reduces the total computational complexity from \(O(m^2 n^2)\) down to \(O(mn(m+n))\), matching the cubic scaling of standard GPTQ while utilizing richer curvature information. Additionally, the algorithm is fully modular with respect to both the base quantizer and the Hessian estimator.

Article Details

Article Details

Abstract

Abstract

We accelerate a family of algorithms for neural network quantization whose geometry is informed by any Kronecker-factored approximation of the Hessian. GPTQ-style adaptive rounding typically uses one-sided information derived from input activations. Two-sided Kronecker-factored Hessian approximations can additionally capture correlations across output coordinates, but applying GPTQ directly in the vectorized weight domain is computationally expensive. Building on the two-sided adaptive-rounding formulation used by BoA and YAQA, we introduce BaKron, an efficient solver that combines anti-diagonal parallelism with a recursive divide-and-conquer construction. For an \(m \times n\) weight matrix, BaKron uses \(O(m+n)\) sequential steps while reducing the total work from \(O(m^2n^2)\) to \(O(mn(m+n))\). Thus, it matches the cubic scaling of GPTQ while exploiting richer curvature information. Moreover, BaKron is modular with respect to both the base quantizer and the Hessian estimator. We also provide practical benchmarks, consider a range of Hessians that BaKron can be called with, find an efficient technique to compute these Hessians, and evaluate the algorithm experimentally.

We accelerate a family of algorithms for neural network quantization whose geometry is informed by any Kronecker-factored approximation of the Hessian. GPTQ-style adaptive rounding typically uses one-sided information derived from input activations. Two-sided Kronecker-factored Hessian approximations can additionally capture correlations across output coordinates, but applying GPTQ directly in the vectorized weight domain is computationally expensive. Building on the two-sided adaptive-rounding formulation used by BoA and YAQA, we introduce BaKron, an efficient solver that combines anti-diagonal parallelism with a recursive divide-and-conquer construction. For an \(m \times n\) weight matrix, BaKron uses \(O(m+n)\) sequential steps while reducing the total work from \(O(m^2n^2)\) to \(O(mn(m+n))\). Thus, it matches the cubic scaling of GPTQ while exploiting richer curvature information. Moreover, BaKron is modular with respect to both the base quantizer and the Hessian estimator. We also provide practical benchmarks, consider a range of Hessians that BaKron can be called with, find an efficient technique to compute these Hessians, and evaluate the algorithm experimentally.

Full-Text & Resources

Full-Text & Resources