论文精选

BaKron:基于Kronecker分解Hessian的高效量化

BaKron: Efficient Quantization with Kronecker-Factored Hessians

精选理由

BaKron用Kronecker曲率做量化,速度和GPTQ同级,能换量化器和Hessian估计器。

AI 摘要

BaKron 是一种新的神经网络量化求解器,利用 Kronecker 分解的 Hessian 信息做自适应取整。它在 m×n 权重矩阵上把顺序步骤数降到 O(m+n),总工作量从 O(m²n²) 降到 O(mn(m+n)),与 GPTQ 的立方复杂度同级。BaKron 基于 BoA 和 YAQA 的两侧自适应取整框架,并兼容不同基础量化器和 Hessian 估计器。论文给出了多种 Hessian 估计器下的量化效果评测。

原文 · arXiv cs.AI

BaKron: Efficient Quantization with Kronecker-Factored Hessians

We accelerate a family of algorithms for neural network quantization whose geometry is informed by any Kronecker-factored approximation of the Hessian. GPTQ-style adaptive rounding typically uses one-sided information derived from input activations. Two-sided Kronecker-factored Hessian approximations can additionally capture correlations across output coordinates, but applying GPTQ directly in the vectorized weight domain is computationally expensive. Building on the two-sided adaptive-rounding formulation used by BoA and YAQA, we introduce BaKron, an efficient solver that combines anti-diagonal parallelism with a recursive divide-and-conquer construction. For an $m\times n$ weight matrix, BaKron uses $O(m+n)$ sequential steps while reducing the total work from $O(m^2n^2)$ to $O(mn(m+n))$. Thus, it matches the cubic scaling of GPTQ while exploiting richer curvature information. Moreover, BaKron is modular with respect to both the base quantizer and the Hessian estimator. We also provide practical benchmarks, consider a range of Hessians that BaKron can be called with, find an efficient technique to compute these Hessians, and evaluate the algorithm experimentally.