研究人员为小模型对话游戏代理设计了一套三步训练法,显著提升了特定任务表现,同时保持通用能力。
研究团队针对2B开放权重模型在LM Playschool Challenge中的表现,发现模型失败不仅是知识不足,还有局部决策问题。他们提出三步训练方案:监督微调获取广泛游戏参与能力,使用回合局部偏好对修复特定对话游戏家族中的可验证失败,并保留对话游戏外的通用能力。该方法将公开clemscore从10.67提升至38.92,领域内分数从13.41提升至41.17,同时保持静态性能基本不变。
Acquire, Repair, Preserve: A Diagnosis-Guided Post-Training Recipe for Small-Model Dialogue Game Agents
Interactive dialogue games test a capability that static benchmarks largely leave implicit: a model must carry state across turns, interpret feedback, and choose valid actions under changing constraints. We study this setting in the LM Playschool Challenge with a 2B open-weight model, and find that many failures are not only broad knowledge failures but also local decision failures: repeated guesses, malformed actions, and violations of feedback that the model has just seen. These diagnostics motivate a training recipe organized around three steps: acquire broad game participation through supervised fine-tuning, repair mechanically verifiable failures within one targeted dialogue-game family using turn-local preference pairs, and preserve general capabilities beyond these dialogue games. In the official final evaluation, our submission improves public clemscore from 10.67 to 38.92 and closed in-domain score from 13.41 to 41.17, while approximately preserving aggregate static performance (44.14 vs. 44.24 for the baseline). Out-of-domain clemscore remains low at 7.88, with the largest gains concentrated in unseen variants of the targeted family. Our results suggest that broad SFT brings most of the model's capability improvement; turn-local supervision can be effective when failure detection is precise, with observed transfer concentrated primarily within-family.