Papers

自出题+解释器标注:Qwen3-4B 在新考题上准确率从 8.3% 升至 16.7%

The Frontier Must Grow - Self-proposed, interpreter-labelled training roughly doubles a 4B model's accuracy on a fresh exam as its verified frontier grows

Why we picked it

这篇论文最难得的是把失败实验也全写出来了。Qwen3-4B 自己写 Lua 程序、解释器当裁判来造训练数据,新考题准确率翻倍到 16.7%,但单次训练也能到 14.6%,多轮的优势其实说不准。做数据合成或自训练的人值得对照它的预注册方法。

arXiv 论文(编号 2611.20049)提出一种自我出题的训练流程:Qwen3-4B 生成 Lua 5.1 程序,凡在 4 次采样中至少猜错一次的题目进入训练池,由解释器直接给出答案标签,每轮新增题量为上一轮的两倍(累计 300、900、2100 题),每轮从基座模型重新训练一个 adapter。

Body

在预注册的原始考试上,该方法通过了逐题检验标准;在事后抽出的 2340 题新考试上,准确率从 8.3% 升至 16.7%。不过第 1 轮没有增益,第 2、3 轮平均只提升 5.0 和 2.9 个百分点。用相同规模基座出题数据做单次训练得到 14.6%,与多轮方法的 2.1 点差距不具统计确定性。

作者同时报告了负面结果:更早的三个预注册实验序列均未达标,同样的循环在仓库级任务和 27B agent 配置下没有收益。论文的价值在于完整公开了预注册标准和失败记录,可作为自我生成训练数据方向的可复现参照。

Original · arXiv cs.AI

The Frontier Must Grow - Self-proposed, interpreter-labelled training roughly doubles a 4B model's accuracy on a fresh exam as its verified frontier grows

A model can learn from problems it poses itself when an executable oracle supplies the answers. Qwen3-4B proposed Lua 5.1 programs; those it predicted wrong at least once in four samples joined a pool labelled by the interpreter, each round adding twice as many programs as the last (300, 900, 2100 in total), and each round trained a fresh adapter from the base on the pool. On the original exam the procedure met its registered criterion, an item-level test of each adapter against its predecessor. On a 2340-item exam drawn afterwards, accuracy rose from 8.3% to 16.7%; that follow-up's registered criteria were not met because round 1 gained nothing, while rounds 2 and 3 added 5.0 and 2.9 points on average, both intervals above zero. One fit on the same amount of base-proposed frontier data reached 14.6%, so the rounds' 2.1-point advantage is inconclusive. Three earlier registered series met no per-lineage criterion, and the same loop showed no gain on repository tasks or for a 27B agent in those configurations.

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk, focusing on China's AI industry and domestic foundation model ecosystem. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Qwen3-4B, Lua, open-source models.

Source attribution: curated from arXiv cs.AI (@Martin Schroder).