KOSMOS Self-Scheduling Kernels Speed Up Training on AMD GPUs
Fixed-role fused kernels leave idle windows because a predetermined communication-compute split cannot track actual data readiness; KOSMOS replaces static schedules with demand-ordered queues that adapt at runtime. Engineers training large MoE models on AMD Instinct hardware can apply it directly through Megatron-LM, where it outperforms DeepEP by up to 1.83x on DeepSeek-V3 and Qwen3-235B proxies.