Ling team cuts agent eval batch times with trajectory scheduling
The non-obvious insight is that batch completion time is governed by the longest trajectory, so ordering by expected length can free capacity without any hardware spend. Teams running RL rollouts or large agent benchmark suites get faster cycles at identical scores, which directly lowers eval cost. Compared with simply provisioning more GPUs or sandboxes, this scheduler-level fix is cheaper and already proven in production for the Ling-3.x models.
The Ling team released a practical method for speeding up agent evaluation and reinforcement learning rollouts by improving scheduling between model inference and environment execution. On BFCL, SWE-bench Multilingual, WideSearch, and MCP-Atlas, batch completion time dropped by 10.4% to 65.0% while evaluation scores stayed the same. The methods now run in both training and evaluation for the Ling-3.x models.
Agent workloads alternate between model inference and environment steps, which means GPUs and sandboxes repeatedly wait on each other. The team's scheduler keeps trajectories on the workers that already hold their KV cache, avoiding reload overhead, and prioritises the longest trajectories when environment slots are limited. Because a batch finishes only when its slowest trajectory completes, running the longest chains first shortens the critical path and reduces idle time across the whole batch.
The result matters for teams running large-scale agent evals or RL rollouts, where tail latency from long trajectories is a common bottleneck. The takeaway for practitioners: if eval runs end with a few long trajectories while other workers sit idle, improving scheduling can beat buying more GPUs.
Great practical paper on speeding up agent evals and RL rollouts.
For agent batches, they suggest running the longest trajectories first, because they determine when the whole batch finishes.
Agent tasks alternate between model inference and environment steps, so GPUs and sandboxes wait on each other. The Ling team schedules both together. Trajectories stay on the workers that hold their KV cache, and when environment slots are limited, the longest chains get priority.
On BFCL, SWE-bench Multilingual, WideSearch, and MCP-Atlas, batch completion time drops by 10.4% to 65.0% with the same evaluation scores. The methods now run in training and evaluation for the Ling-3.x models.
Takeaway: If your eval runs end with a few long trajectories while the other workers sit idle, look at scheduling before buying more GPUs.
https://t.co/36rd5LB9Ix
Chat with Paper: https://t.co/mE2qcyw1EO
This report is categorized under AITOP's "Papers" desk, focusing on China's AI industry and domestic foundation model ecosystem. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Ling, RL, BFCL, SWE-bench.
Source attribution: curated from DAIR.AI (@dair_ai).