这篇论文把PTQTP量化器约束成九级格式,在DeepSeek-V4上做到和官方API几乎一样准,还快6.7%、文件小9%,适合搞MoE推理优化的朋友看。
PTQTP将LLM权重矩阵分解为两个三元平面,本研究首次将尺度比固定为3的约束引入其求解器,使分解坍缩为统一九级量化器。两个三元平面无损折叠为4位码平面,作为持久服务表示,磁盘字节、专家缓存和内核输入均为4.0625位/权重块。该方法应用于DeepSeek-V4-Flash-0731的284B-A13B MoE模型,在64GB笔记本上从MXFP4权重一次性量化并流式加载专家。与4.5位Q4_K基线相比,在5/5测试中匹配官方API(Q4_K为4/5),MMLU子集得分86对84,解码速度快6.7%,文件小9%。尽管权重重建误差和困惑度更高,但参考保真度无显著差异。
Tied Trit-Planes: Constraining PTQTP to a Uniform Nine-Level Quantizer, with a Persistent Folded Format for Disk-Streamed Mixture-of-Experts Serving
PTQTP decomposes LLM weight matrices into two ternary (trit) planes with two free per-group scales. Tying the scales to a fixed ratio of three collapses the decomposition into a single uniform nine-level quantizer, a known balanced-ternary identity. To our knowledge, at the time of writing, this work is the first to impose that identity as a constraint inside PTQTP's solver. The two trit planes then fold losslessly into one 4-bit code plane that we make the persistent serving representation: disk bytes, expert-cache bytes, and kernel input are the same 4.0625-bits/weight blocks, consumed in one integer dot pass. For this conjunction (ratio-3 nine-level code, CPU-SIMD kernels, SSD expert streaming, identical persistent bytes) we likewise found no precedent. We apply this to the routed experts of DeepSeek-V4-Flash-0731, a 284B-A13B mixture-of-experts model, quantizing in one shot from the released MXFP4 expert weights and streaming experts from SSD on a 64 GB laptop. Against a 4.5-bit Q4_K baseline, measured one process per fixture with an expert-lossless anchor arm as reference control, the tied model matches the official serving API on 5/5 fixtures at step 0 (Q4_K: 4/5) and 12/14 captured continuation steps (11/14), scores 86 vs. 84 on a 100-item MMLU subset, decodes 6.7% faster in decode phase, and ships 9% smaller files: no detected fidelity difference at these small evaluation sizes, and every fixture-level difference between the arms traces to a single measured near-tie cell. The tied fit nevertheless shows higher weight-reconstruction error and worse perplexity, a measured dissociation between proxy metrics and reference fidelity. A cumulative trunk-ternarization ladder and bitwise-pinned aarch64/x86-64 kernels complete the report. All code, formats, and evaluation artifacts are open source in the fucina inference stack.