LMSYS Miles Trains Qwen3-30B-A3B on AMD MI355X
The engineering details on keeping long agent trajectories and hundreds of CPU sandboxes from stalling RL training are the real payload here, not the headline score. Teams building agentic RL pipelines can study how SGLang, Megatron-LM and Harbor were combined on AMD hardware as an alternative to NVIDIA-only stacks. It also shows Qwen3-30B-A3B reaching 27.5% on SWE-bench Verified with open training recipes.
LMSYS and AMD trained Qwen3-30B-A3B with fully asynchronous agentic RL using SGLang, Megatron-LM and Harbor on AMD MI355X GPUs. SWE-bench Verified pass@1 rose from 7.3% to 27.5% after SFT plus RL, measured on a 100-task stratified subset with 4 attempts per task. RL steps became 3x faster, dropping from 850s to 284s. Rollout throughput improved roughly 4x, from about 860 to about 3,480 trajectories per hour, with 384 agent sandboxes running concurrently.
🚀 Training SWE Agents with Miles on AMD MI355X
The @AIatAMD and Miles teams trained Qwen3-30B-A3B with fully asynchronous agentic RL using SGLang, Megatron-LM, and Harbor. Keeping long trajectories and hundreds of CPU sandboxes from stalling training took some engineering.
Highlights: - SWE-bench Verified pass@1: 7.3% → 27.5% after SFT + RL (100-task stratified subset, 4 attempts per task) - 3× faster RL steps: 850s → 284s - ~4× rollout throughput (~860 → ~3,480 trajectories/h), with 384 agent sandboxes running at once.
Full report: https://t.co/BbZBN7nhQW
