ReMoE解决了内存受限设备上MoE模型推理的I/O瓶颈，做边缘部署或大模型推理优化的开发者可以直接试，开源代码让复现门槛很低。
细粒度混合专家（MoE）模型在推理时只激活部分专家，但内存受限场景下只能缓存少量专家，未缓存的专家需从慢速外部存储获取，导致频繁换入换出和I/O开销。ReMoE提出一种路由器微调框架，通过偏向近期选中的专家来产生时间上稳定的路由，从而提升专家复用率，减少存储访问。实验表明，在DeepSeek和Qwen模型上，ReMoE将专家复用率提升26%，同时保持下游任务性能。在vLLM GPU-CPU专家卸载场景下，输出吞吐量提升8.4%；在Jetson Orin NX上使用llama.cpp时，TPOT降低43.6-49.8%，解码速度提升1.77-1.99倍。代码和模型已开源。
ReMoE: Boosting Expert Reuse through Router Fine-Tuning in Memory-Constrained MoE LLM Inference
Fine-grained Mixture-of-Experts (MoE) models sparsely activate only a subset of experts per token, reducing activated computation while maintaining high model capacity. However, in memory-constrained inference scenarios, only a small set of experts can be cached. Experts not in the cache must be fetched from slow external storage (e.g., UFS), leading to frequent evictions and substantial I/O overhead. We propose ReMoE, a router fine-tuning framework designed to boost token-wise expert reuse. ReMoE biases the router toward recently selected experts, producing temporally stable routing that better matches cache locality constraints. By increasing short-horizon expert reuse, ReMoE reduces expert fetches from storage without adding inference-time computation. Experiments on DeepSeek and Qwen models show that ReMoE improves expert reuse by 26% while maintaining downstream task performance. Real-system evaluations further confirm these benefits, improving output throughput by 8.4% under vLLM GPU-CPU expert offloading and reducing TPOT by 43.6-49.8% under llama.cpp on Jetson Orin NX, corresponding to a 1.77-1.99$\times$ decode speedup across diverse workloads. Checkpoints and usage instructions are available at https://github.com/BUAA-OSCAR/ReMoE.