MiniCPM-V 4.7 runs 35B MoE on phones via LES
The notable point is that a 35B-class MoE can now run at usable speeds on consumer phones without altering the model itself, since the router still decides which experts execute. Mobile and edge developers can test the setup via the released GitHub code and LES links. The benchmark figure on an iPhone 18 Pro Max gives a concrete comparison point against on-demand expert loading baselines.
OpenBMB introduced Lookahead Expert Scheduling (LES), an inference system that runs the MiniCPM-V 4.7 35B-A3B MoE model in real time on edge devices. On an iPhone 18 Pro Max, the full system reaches 20.51 tokens per second, a 6.07× speedup over on-demand expert loading. LES combines prediction-based expert prefetching, contiguous expert layouts with mixed-precision quantization, and DSpark speculative decoding. The quantization cuts the weight file by 26.7% with only a 0.16-point drop in the five-benchmark average, while model architecture and routing stay unchanged.
📱 35B parameters. Real-time inference. On a phone.
Meet Lookahead Expert Scheduling (LES), the inference system powering MiniCPM-V 4.7’s 35B-A3B MoE model on edge devices.
🚀 On an iPhone 18 Pro Max, the full system reaches 20.51 tokens per second, a 6.07× speedup over loading expert weights on demand.
How? Three coordinated optimizations: - 🔮 Scheduling: Predict and prefetch the next layer’s experts while the current layer computes. - 💾 Storage: Combine contiguous expert layouts with mixed-precision quantization to reduce loading overhead. - ⚡ Compute: Use DSpark speculative decoding to verify multiple candidate tokens in one pass.
Predictions guide prefetching; the model’s actual router still determines which experts execute. The architecture, expert counts, and routing remain intact. Mixed-precision quantization reduces the weight file by 26.7%, with just a 0.16-point drop in the five-benchmark average.
LES shows how better inference systems can bring larger, more capable models to everyday devices. 🌍
👉 LES: https://t.co/EVYiHJ7eC5 🔗 GitHub: https://t.co/msjm2QlrvX
#MiniCPM #OnDeviceAI #MoE #EdgeAI
