GC-MoE Operator Accelerates MoE Inference with CUDA Green Contexts
The technique shows that partitioning a single GPU with CUDA Green Contexts removes the resource contention between dispatch warps and GEMM warps that limits fused MoE operators like Mega MoE. Practitioners serving DeepSeek-V4.1-Flash or GLM-5.2 on Hopper or Blackwell clusters can adopt the open-source operator to cut decode latency without extra hardware. The results on H20 GPUs, a China-market chip, make it especially relevant for deployments where H100-class hardware is unavailable.