Hugging Face turns coding harnesses into RL environments
Training in a scaffold nobody ships has been a quiet flaw in agentic RL; this proxy approach removes the need to reimplement harnesses as environments. Practitioners building agentic models can replicate the full pipeline on Hugging Face's guide and shape rewards for behaviors like tool efficiency. The results also show why harness-specific benchmark scores overstate real-world capability.
Hugging Face released an open-source method for training models with reinforcement learning inside real coding harnesses such as Claude Code, Codex, Mini-SWE-Agent, Hermes, Pi and OpenCode, without modifying either the harnesses or the training code.
A capture proxy speaks the four formats coding agents use (OpenAI Chat Completions, OpenAI Responses, Anthropic Messages, Gemini), forwards requests to vLLM, records token IDs and logprobs, and feeds sequences to TRL for training.
Testing on Liquid AI's LFM2.5-2.
6B showed the same weights score 62% under Mini-SWE-Agent but 33% under Claude Code, and training across 4 harnesses at once lifted scores from 42% to 54% in all four.
Adding a reward bonus for fewer tool calls cut tool usage by 31% on already-solved tasks.All 7 trained models, the SFT data (3,189 rollouts from Qwen3.
8-27B), tasks and training code are public.
We turned Claude Code, Codex, Hermes, Pi, @opencode and other coding harnesses into RL environments. No changes to the harnesses, no changes to the training code. Any open model, any task set, fully open source my friends! Same model, same weights: 62% under Mini-SWE-Agent, 33% under Claude Code. But training inside a real harness normally means reimplementing it as an environment, so most models get trained in a scaffold nobody actually ships. The fix is a proxy, not a rewrite. The harness thinks it's talking to a model API. It's actually talking to a capture proxy that speaks the 4 formats coding agents use (OpenAI Chat Completions, OpenAI Responses, Anthropic Messages, Gemini), forwards to @vllm_project , records the exact token IDs and logprobs vLLM sampled, and hands TRL sequences it can train on. The harness becomes the environment. 10 harnesses run through it today, none modified. And because you control the reward, you can shape behavior the harness never asked for. We added a small bonus for solving a task in fewer tool calls: on tasks it already solved, the model now uses 31% fewer calls, in every harness, and about half under Codex. Tested on LFM2.5-2.6B from @liquidai : → Train in one harness: better mostly in that harness (OpenCode 34% → 58%). → Train in 4 at once: better in all 4 (42% → 54%). → SFT on 3,189 rollouts from Qwen3.8-27B instead: plateaus at 47.5%, below both RL runs. Everything is open and reproducible: the capture proxy in OpenEnv, the trainer in TRL, the tasks, the SFT data, the training code and all 7 trained models. Bigger models and bigger runs next. Full guide: huggingface.co/spaces/FineEnv…📊 19 ⚡
This report is categorized under AITOP's "Models" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include Claude Code, Codex, Hugging Face, vLLM.
Source attribution: curated from Clement Delangue (@ClementDelangue). Cross-source corroboration: 11 independent sources have covered this development (including lmarena.ai, 宝玉, Tibor Blaho), confirming industry-wide signal strength.