Primary sourceOpenBMB
View sourceEvent ·Multi-source
社区推出 MiniCPM5-2B 的 EXL3 4-bit 量化版,权重仅 1.61 GB
社区为 MiniCPM5-2B 制作了一个 EXL3 4-bit(4.0 bpw)量化版本,量化后模型权重只有 1.61 GB。在 NVIDIA Tesla T4 上实测推理速度约 68–70 tokens/s。该版本支持通过 ExLlamaV3 和 TabbyAPI 进行本地推理,适合低资源设备的本地部署。
Current read
MiniCPM5-2B 出了 4-bit 量化版,权重才 1.61 GB,T4 上能跑 70 tokens/s,想在自己机器上跑小模型可以试试。
11 sources73° AI heatLast updated 2026/9/28 13:00:03
Evidence
11 sourcesRelated source 1Decoder
View sourceRelated source 2Artificial Analysis
View sourceRelated source 3Aravind Srinivas
View sourceRelated source 4Gary Marcus
View sourceRelated source 5The Information
View sourceRelated source 6PyTorch
View sourceRelated source 7Yuchen Jin
View sourceRelated source 8arXiv cs.AI
View sourceRelated source 9NVIDIA AI
View sourceRelated source 10@koltregaskes
View sourceConflict check
Dedup data only shows that these sources discuss the same event, not that they agree. There is no structured supporting or refuting evidence yet, so no conflict is inferred automatically.