这篇论文把Self-Refine、Reflexion这些热门方法拉出来做了严格对比，发现相同token预算下自己反思不如多采样几次取多数，想省成本做推理的可以看看。
该论文对七种自我反思方法进行对照实验，使用1.5B、3B和7B三种开源模型，在两个数学基准上各150道题。所有方法按实际生成token数对齐后，与重复采样取多数答案的基线比较，共36组配对检验。结果显示没有任何方法在相等成本下稳定优于重复采样，其中10组可靠更差，全部18组涉及模型自我检查的比较均为负面。Self-Refine和强制的Reflexion在7B模型上仍落后基线3.6至10.1个百分点，而最佳选择在1.5B时取多数答案比模型自选高8.0和11.3点，到7B时差距缩小至不显著。
Sample More, Reflect Less: Self-Refine and Reflexion Lose to Repeated Sampling at Equal Token Cost, from 1.5B to 7B
Methods that make a language model plan, criticise and rewrite its own answer, reflect on mistakes, pick the best of several attempts, or debate with copies of itself nearly all make it generate far more text than a single chain of thought. Because generating more text raises accuracy by itself, a gain over one chain of thought does not show the method's idea is what helped. Wang et al. (2024) reported that a simple baseline, sampling the same question repeatedly and keeping the most common answer, often wins once budgets are comparable, but gave point estimates with no confidence intervals or significance tests. We rerun that comparison as a designed experiment: seven methods, open models of 1.5B, 3B and 7B parameters, two mathematics benchmarks, 150 questions each. We count every generated token, including those spent on critiques, reflections, debate turns and checking, and compare each method against repeated sampling at its own measured cost. All 36 comparisons are paired by question, with bootstrap intervals and multiplicity correction. No method is reliably better than repeated sampling at equal cost anywhere. Ten are reliably worse, all of them methods where the model inspects its own output, and all 18 self-inspection comparisons are negative. The two kinds of self-inspection part company as models grow. Choosing stops hurting: taking Best-of-N's eight samples and just counting the most common answer beats letting the model pick by 8.0 and 11.3 points at 1.5B, but only 2.0 and 1.3 at 7B, no longer distinguishable from zero. Rewriting does not recover: Self-Refine and a forced Reflexion stay 3.6 to 10.1 points below baseline at 7B. Reflexion as published never triggered its own retry on the smallest model. It judged itself correct every time and silently became a single chain of thought. We release code, prompts, all generations, and our verification scripts.