Cognition分享了他们怎么跨三大洲四个数据中心做RL训练，用紧耦合trainer加分布式异步推理，还靠压缩权重差异同步，做到稳定rollout。做大规模RL的值得看看。
Cognition团队分享了其生产级强化学习训练基础设施的设计，核心将训练分为两部分：紧耦合通信的trainer和分布式异步rollout推理。他们的RL训练横跨三大洲四个数据中心，结合自有GPU集群和Fireworks AI等推理提供商的算力。rollout推理引擎通过压缩权重差异在对象存储中同步，实现分布式扩展。这套基础设施支撑了SWE-1.7模型的训练，并确保跨数据中心的可靠rollout。
This is what production RL infra looks like. ICYMI: RL training at scale separates into two distinc...
This is what production RL infra looks like. ICYMI: RL training at scale separates into two distinct problems. - Tight collective comms for the trainer. - Distributed async inference for rollout. Kudos to the @cognition team on this. Their trainer is the secret sauce behind SWE-1.7 and reliable RL rollouts across four datacenters on three continents is easier said than done! We're excited to continue partnering with AI leaders on their path to specialized intelligence. Cognition @cognition Our RL training spans four datacenters across three continents, combining our own GPUs across multiple clusters with additional compute from inference providers like @FireworksAI_HQ . Only the trainer needs tight collective communication. Rollout inference was distributed, with engines syncing through compressed weight diffs in object storage. 🔗 View Quoted Tweet 💬 1 🔄 0 ❤️ 1 👀 394 📊 1 ⚡