ISER-MoE 通过 NVLink 交换机复制热点专家，提升 MoE 推理吞吐
ISER-MoE: Dynamic In-Switch Replication of Hot Experts for Load Balancing in Expert-Parallel
MoE 推理总被几个热点专家拖慢，这个方案直接借 NVLink 交换机做复制，DeepSeek-V4-Pro 实测数学任务吞吐涨了 17.4%。
ISER-MoE 是一个构建在 TensorRT-LLM 中的系统，通过 NVLink 交换机将热点专家组播到固定 GPU 分组，让每个字节只从拥有者发送一次。GPU 调度器按层分配热点专家到分组，并用多端口广播把注入负载分散到分组端口。在 trace 重放中，调度器消除了 98-99% 的超额负载；在 DeepSeek-V4-Pro 的 prefill 服务中，数学基准吞吐提升 17.4%，agentic 提示吞吐提升 7.2%。
ISER-MoE: Dynamic In-Switch Replication of Hot Experts for Load Balancing in Expert-Parallel
Expert-parallel mixture-of-experts (MoE) layers run at the pace of the most loaded GPU, and a few hot experts carry most of the excess load. Replicating them at every MoE layer balances the load only if their weights arrive within the layer, and broadcasting from the owner is the bottleneck: point-to-point copies overload it, and baseline broadcasts pay latency or bandwidth overheads. We present ISER-MoE (In-Switch Expert Replication for MoE), built in TensorRT-LLM, which multicasts hot experts through the NVLink switch to fixed GPU groups, so each byte leaves its owner once. A GPU scheduler assigns hot experts to groups per layer, and a multi-port broadcast spreads the injection over the group's ports, both overlapped with the shared expert. In trace replay, the scheduler removed 98-99% of the excess load; in prefill serving of DeepSeek-V4-Pro, throughput rose 17.4% on a math benchmark and 7.2% on agentic prompts.