FIT-GGUF builds mixed-precision MiniCPM5-2B GGUF variants
Mixed-precision quantization with a size target lets you hit an exact memory budget rather than settling for whatever Q4/Q5 yields, which matters on edge devices where every few hundred MB counts. Builders running local inference should compare these 1.14–1.46 GiB MiniCPM5-2B builds against their current preset and try FIT-GGUF on their own models. The models are on Hugging Face for direct download.
Developer @Scorp1o_117 used FIT-GGUF to create GGUF quantized variants of MiniCPM5-2B with controllable file sizes.
Instead of fixed presets like Q4 or Q5, users set a target file size or fidelity tier, and FIT-GGUF allocates precision per tensor, then predicts, generates and verifies the output.
Four MiniCPM5-2B builds were produced, ranging from about 1.14 GiB to 1.
46 GiB, with Quality, Balanced, Compact and Mini presets.Quality was checked with KL Divergence and Same-top evaluation, and generated file sizes matched the predicted targets.
🚀 FIT-GGUF brings controllable-size mixed-precision quantization to MiniCPM5-2B Developer @Scorp1o_117 used FIT-GGUF to build MiniCPM5-2B GGUF variants around specific size and quality targets. Instead of choosing a fixed quantization preset, you can set a target file size or fidelity tier, and FIT-GGUF automatically decides how much precision to allocate to different tensors—then predicts, generates, and verifies the final GGUF.
✨ What’s included 🧠 Tensor-level mixed-precision quantization 📦 Four MiniCPM5-2B builds from ~1.14 GiB to ~1.46 GiB 🎯 Quality / Balanced / Compact / Mini presets 📊 KL Divergence and Same-top evaluation ✅ Generated file sizes matched the predicted targets
A nice example of how MiniCPM5-2B can be tuned for different memory and deployment constraints, without being locked into a single Q4/Q5-style quantization preset.
Check out FIT-GGUF and try building a MiniCPM5-2B variant that fits your own device budget.
🤗Model: https://t.co/WL3o7jLA1M https://t.co/FZOMTZh3tS
This report is categorized under AITOP's "Products" desk, focusing on China's AI industry and domestic foundation model ecosystem. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include MiniCPM5-2B, FIT-GGUF, quantization, open-source models.
Source attribution: curated from OpenBMB (@OpenBMB).
