QSelect 为太空机器人机载语言模型提供低比特压缩方案
Compressing Onboard Language Models for Space Robots under a Behavioral-Fidelity Requirement
arXiv 上这篇论文教你怎么把 7B/8B 模型压到 3-4 比特还给太空机器人用，混搭三种量化器比单用一种效果更好，做端侧部署的可以看看。
论文提出 QSelect，把 AWQ、GPTQ 和 AutoRound 三种量化检查点按模块组合成一个静态低比特模型，用于太空机器人在轨运行语言模型。在 DeepSeek-R1-Distill-Qwen-7B、Llama-3.1-8B 和 Llama-3.2-3B 的 3 与 4 比特设置下，组合模型在全部 6 个场景的困惑度均低于任一单一量化家族模型。默认保真模式无需标注数据，仅在前向 KL 散度低于 BF16 参考时接受替换；目标适配模式在两个 Llama 模型的 3 比特合并准确率上比最佳单一家族模型提升 4.3 和 4.5 个百分点。所有评估基于地面 GPU 与通用域基准，太空任务与飞行硬件验证尚未进行。
Compressing Onboard Language Models for Space Robots under a Behavioral-Fidelity Requirement
Language models are being explored as the interface through which space robots interpret operator commands and select among verified skills. Running them onboard calls for low-bit compression, and flight use adds requirements that generic post-training quantization leaves implicit: a fixed memory budget and storage format, a static artifact under configuration control, and behavior close to the full-precision model validated on the ground. We treat compression as a ground-segment step and study QSelect, which composes one static low-bit model from compatible AWQ, GPTQ, and AutoRound checkpoints by assigning each linear module to one quantizer family and scoring every substitution at the final next-token distribution. The default fidelity mode needs no labels or tuned thresholds and accepts a substitution only when the forward KL divergence from the BF16 reference decreases; an optional target-adapted mode adds answer likelihood on a small labeled set. On DeepSeek-R1-Distill-Qwen-7B, Llama-3.1-8B, and Llama-3.2-3B at 3 and 4 bits, composed models reach lower mean held-out perplexity than every single-family model in all six settings, and the fidelity mode lowers perplexity and KL further. In a matched scaffold, final-output scoring gives lower held-out KL than representation-similarity scoring in all 12 paired runs. With labels from training or development splits of the evaluated task families, the target-adapted mode raises 3-bit pooled accuracy on the two Llama models by 4.3 and 4.5 points over the best single-family model. At 3 bits, unquantized embedding and output layers make up 43-46% of the packed artifact. All evaluations use general-domain benchmarks on a ground GPU; space-domain tasks and flight hardware are left to future work.
