Event ·Multi-source

社区推出 MiniCPM5-2B 的 EXL3 4-bit 量化版,权重仅 1.61 GB

社区为 MiniCPM5-2B 制作了一个 EXL3 4-bit(4.0 bpw)量化版本,量化后模型权重只有 1.61 GB。在 NVIDIA Tesla T4 上实测推理速度约 68–70 tokens/s。该版本支持通过 ExLlamaV3 和 TabbyAPI 进行本地推理,适合低资源设备的本地部署。

Current read

MiniCPM5-2B 出了 4-bit 量化版,权重才 1.61 GB,T4 上能跑 70 tokens/s,想在自己机器上跑小模型可以试试。

11 sources73° AI heatLast updated 2026/9/28 13:00:03

Evidence

11 sources
Related source 2Artificial Analysis
View source
Related source 3Aravind Srinivas
View source
Related source 4Gary Marcus
View source
Related source 5The Information
View source
Related source 7Yuchen Jin
View source
Related source 8arXiv cs.AI
View source
Related source 10@koltregaskes
View source

Conflict check

Dedup data only shows that these sources discuss the same event, not that they agree. There is no structured supporting or refuting evidence yet, so no conflict is inferred automatically.

MiniCPM5-2B 4-bit EXL3 quantization shrinks weights to 1.61 GB · 事件专题 | AITOP