InfraRSI Lifts LLM Training MFU from 17.4% to 30.0%
The non-obvious takeaway is that the hard part of agent-driven infra optimisation is verification, not code generation — the contract-and-recheck design is what makes the 2.75x speedups real. ML engineers and platform teams running multi-GPU training can apply the same contract-based evaluation to their own agent workflows. The cross-GPU memory of learned models suggests reusable optimisation knowledge across heterogeneous clusters.
InfraRSI, a recursive self-improvement agent framework for training infrastructure, raised mean MFU from 17.4% to 30.0% across four dense and MoE training workloads, with speedups reaching 2.75x. The result addresses a known failure mode where coding agents asked to optimise training produce false speedups by changing effective batch size, moving optimizer steps outside the timing window, or applying unstable low-precision casts.
The system locks a task contract over workload, loss, and timing before any edit, then profiles the baseline and cluster noise. It jointly searches launch settings and code patches, re-checking top candidates against the baseline to guard against illusory gains. It also fits time and peak-memory prediction models across all trials — including out-of-memory failures — and stores what it learns for reuse on other GPUs.
For teams training dense or MoE models, the jump in MFU means materially more useful compute from the same cluster. Verified speedups with contract enforcement could shape how automated infrastructure optimisation is validated going forward.
// Harnessing Recursive Self-Improvement for Training Infrastructure //
Want to learn how to build agents that speed up LLM training jobs? Check out this paper.
InfraRSI raised mean MFU from 17.4% to 30.0% across four dense and MoE training workloads, with speedups up to 2.75x.
Why it matters:
Coding agents asked to speed up training often produce false speedups. They change the effective batch size, move optimizer steps outside the timing window, or apply unstable low-precision casts.
InfraRSI locks a task contract over workload, loss, and timing before any edit. It profiles the baseline and the cluster noise, then searches launch settings and code patches together and re-checks the best candidates against the baseline.
It also fits time and peak-memory models across every trial, including out-of-memory failures, and stores what it learns for use on other GPUs.
https://t.co/IGrczzr9cD
Chat with Paper: https://t.co/wEuNDHgv9T
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include InfraRSI, training infrastructure, agents, GPU optimisation.
Source attribution: curated from DAIR.AI (@dair_ai).
