Tencent Hunyuan extends critical-batch-size theory to LLM RL
Rollout generation dominates RL training cost, and this work shows batch-size scaling plus a retuned learning rate can cut wall-clock time without new hardware. Teams running GRPO or PPO fine-tuning can apply the retuning rule directly to squeeze more throughput from existing GPU clusters. The numbers come from Tencent's own infrastructure, so results may shift with different model sizes and interconnects.
Tencent Hunyuan published research extending classical critical-batch-size theory to online LLM reinforcement learning, where models generate their own training data and rollout generation and training scale differently.
Testing across GRPO and PPO, the team found that learning-rate retuning preserves learning per response over a bounded range of batch sizes.
On fixed hardware, scaling up batch size improved PPO generation-stage throughput by up to 2.29×, and the best measured GRPO configuration reached the same validation target in 29% less time.
The full paper is available on the Hunyuan research site.
⚡️ As LLM reinforcement learning scales to larger GPU clusters and more training data, training efficiency becomes a first-order concern. Our new research revisits classical critical-batch-size theory and extends it to online LLM RL, where the model generates its own training data and rollout generation and training scale differently. Across GRPO and PPO, we find that learning-rate retuning can preserve learning per response over a bounded range of batch sizes. On fixed hardware, scaling up the batch size improves PPO generation-stage throughput by up to 2.29×, while our best measured GRPO configuration reaches the same validation target in 29% less time. 🚀 Read the full research: hy-ai.woa.com/research/10011… 5📊 3 ⚡
This report is categorized under AITOP's "Papers" desk, focusing on China's AI industry and domestic foundation model ecosystem. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Tencent Hunyuan, GRPO, PPO, reinforcement learning.
Source attribution: curated from Hunyuan (@TXhunyuan).
