模拟存算一体是突破大模型内存墙的关键路线，但硬件噪声对MoE架构的破坏一直未被正视。做硬件-算法协同设计的团队，这篇论文给出了可直接复用的校准方案，值得细读。
混合专家（MoE）大模型通过稀疏激活专家实现高效扩展，但频繁切换专家造成内存带宽瓶颈，而模拟存算一体（CIM）架构可缓解此问题。然而，模拟CIM的硬件缺陷会扰动存储权重，其对MoE模型的影响此前未被系统研究。本文首次基于真实芯片噪声校准，发现硬件噪声会破坏专家负载均衡，使训练好的路由决策失效。为此提出ROMER后训练校准框架：替换低激活专家为高频专家以恢复负载均衡，并通过百分位数归一化重新校准路由器logits。在DeepSeek-MoE、Qwen-MoE和OLMoE上，ROMER在真实芯片噪声下将困惑度分别降低58.6%、58.8%和59.8%，验证了其有效性和跨架构泛化能力。
ROMER: Expert Replacement and Router Calibration for Robust MoE LLMs on Analog Compute-in-Memory Systems
Large language models (LLMs) with mixture-of-experts (MoE) architectures achieve remarkable scalability by sparsely activating a subset of experts per token, yet their frequent expert switching creates memory bandwidth bottlenecks that compute-in-memory (CIM) architectures are well-suited to mitigate. However, analog CIM systems suffer from inherent hardware imperfections that perturb stored weights, and its negative impact on MoE-based LLMs in noisy CIM environments remains unexplored. In this work, we present the first systematic investigation of MoE-based LLMs under noise model calibrated with real chip measurements, revealing that hardware noise critically disrupts expert load balance and renders clean-trained routing decisions consistently suboptimal. Based on these findings, we propose ROMER, a post-training calibration framework that (1) replaces underactivated experts with high-frequency ones to restore load balance, and (2) recalibrates router logits via percentile-based normalization to stabilize routing under noise. Extensive experiments across multiple benchmarks demonstrate that ROMER achieves up to 58.6\%, 58.8\%, and 59.8\% reduction in perplexity under real-chip noise conditions for DeepSeek-MoE, Qwen-MoE, and OLMoE, respectively, establishing its effectiveness and generalizability across diverse MoE architectures.