Nathan Lambert 转了一条讲闭源实验室 RL 训练细节的推文串，OpenAI、Anthropic 内部怎么做强化学习，这里能挖到不少线索。
强化学习研究者 Nathan Lambert 在 X 上转发并评论一条推文串，内容是闭源实验室（如 OpenAI、Anthropic）使用的 RL 训练细节正在逐渐外泄。推文串梳理了这些实验室在 RLHF/RLVR 等环节的做法。Lambert 评价称这类信息此前很少公开，如今正缓慢流出。
Nice thread, some RL details used by closed labs slowly leaking out.
Ownership of the resulting model weights is the differentiator here, since most hosted fine-tuning services leave users with an API endpoint rather than a portable asset. Teams building proprietary capabilities on open models can bring their own reward functions and environments rather than accepting a fixed RL recipe. It positions Together as an alternative to platform-locked tuning offerings from larger cloud providers.
The discount makes B200-based SFT and RL runs cheaper for teams that want full control of their training loop rather than a managed pipeline. Startups fine-tuning open-source models should compare this against renting B200 capacity on clouds like AWS or CoreWeave before the eight-week window ends. Pricing is in US dollars and available to customers globally, including Southeast Asia.
Verified-by-construction environments sidestep the usual reward-hacking problem in RL for open-ended analysis tasks, a reusable recipe for teams training agents beyond fixed-answer benchmarks. The diagnostic finding that frontier models compute correctly but conclude wrongly matters for anyone building data-analysis agents who should audit the reasoning step, not just the arithmetic. The paper is available on arXiv (2610.21656) with a Chat with Paper interface at academy.dair.ai.
有人质疑 V4.1 Pro 可能跟 Flash 差距不大，理由是 RL 预算和蒸馏数据卡了上限，观点挺有参考性。
有消息说智谱的 GLM 5.5 快发布了，靠用户数据做 RL 后训练，写代码有望追上前沿闭源模型，开源党可以关注下。
