PIHF把强化学习的迭代改进搬到了上下文学习里，用语言模型当执行器，专家审阅改进，罕见病诊断上Recall@1提升明显，值得一看。
PIHF（Policy Iteration with Human Feedback）提出了一种利用预训练语言模型作为执行基底，通过版本化的自然语言策略和工具集进行持续改进的方法。该方法结合了语言模型评论家和临床专家的审查，以定位重复失败并形成候选修订。在超罕见疾病基准测试中，PIHF派生的策略在多个执行器上提升了Recall@1，其中GPT-5.4提升32.7个百分点，Qwen3.6-35B提升31.1个百分点。结果表明，固定权重的预训练语言模型可用于专家引导的策略开发。
Policy Iteration with Human Feedback: Bringing Post-Training RL to In-context Learning
Generative pretraining established reusable task representations; later work on language-based task conditioning and in-context learning showed that a fixed model could adapt its behavior from instructions and demonstrations. Policy Iteration with Human Feedback (PIHF) builds on this development and the recurrent evaluate-and-improve structure of generalized policy iteration. PIHF uses a pretrained language model as its execution substrate and moves persistent revision to a versioned natural-language policy and tool set. A language-model critic and clinical expert review complete-panel reasoning and tool-use trajectories to localize recurrent failures and form candidate revisions; the expert may reinterpret the evidence and retains authority over admission and rollback, while Recall@1 and Recall@5 validate outcomes after candidate execution. Across cumulative ablations and ultra-rare-disease benchmarks, a PIHF-derived policy improved Recall@1 in one proprietary executor and three open-weight executors spanning 3 to 49 billion active parameters. Gains were 32.7 percentage points for GPT-5.4 and 31.1 points for Qwen3.6-35B, a difference of 1.7 points. These results support the feasibility of using pretrained language models as fixed-weight execution substrates for expert-guided policy development in rare-disease diagnosis.