想给图像或视频扩散模型做低位量化?OrbitQuant不需要每换一个模型或任务就重跑校准,FLUX.1、Wan 2.1上表现都很能打,W2A4也有可用效果。
OrbitQuant提出一种数据无关的权重量化方案,通过随机置换块Hadamard(RPBH)旋转将激活值变换到归一化旋转基,使各坐标分布固定无需重新拟合校准数据。在FLUX.1、Z-Image-Turbo、Wan 2.1、CogVideoX四个模型上,该方法在多个低位宽设定下达到后训练量化(PTQ)最佳效果,并将图像扩散Transformer的PTQ推进到W2A4可用质量。同一量化器可直接从图像迁移到视频,无需针对每种模态调整。
OrbitQuant: Data-Agnostic Quantization for Image and Video Diffusion Transformers
Diffusion transformers (DiTs) achieve state-of-the-art image and video generation, but their multi-step sampling and growing parameter count make inference expensive. Post-training quantization (PTQ) is the natural remedy, yet DiT activations shift across timesteps, prompts, and guidance branches, forcing prior methods to re-fit calibration data for every new checkpoint or modality. We present OrbitQuant, a data-agnostic weight-activation quantizer that bypasses range estimation by quantizing in a normalized, rotated basis. In this basis, a randomized permuted block-Hadamard (RPBH) rotation concentrates each coordinate around one fixed, known marginal regardless of the input, so a single Lloyd-Max codebook serves all timesteps, prompts, and layers of a given input dimension. We extend the same quantizer to weight rows offline, absorbing the rotation into the weights so that it cancels inside each linear layer and only a forward rotation on the activations remains at runtime. The same recipe transfers from image to video with no per-modality tuning. Across FLUX.1, Z-Image-Turbo, Wan 2.1, and CogVideoX, it sets the state of the art for PTQ at several low-bit settings. It also pushes PTQ of image diffusion transformers to W2A4 with usable generation quality.