论文显示:长任务中模型准确率的小差距会累积成巨大结果差异
Mollick 转了篇论文:长任务里便宜模型的错误会滚雪球,强模型当编排器才划算,做 agent 工作流的值得看看。
Ethan Mollick 分享了一篇论文,核心结论是前沿模型与更便宜模型之间准确率和判断力的细微差距,会在长时程任务中不断复利式放大,最终造成结果的巨大差异。他因此对'前沿模型可以被廉价模型轻易替代'的观点持怀疑态度,并建议用最强的模型来充当编排器(orchestrator),把便宜模型放在下游。这条推文没有给出具体模型名或基准数据,观点指向多智能体工作流中的模型选型策略。
This paper shows why I am skeptical of the idea that frontier models can be easily replaced by cheaper ones (and why you should use the best models as orchestrators): Small improvements in accuracy & judgement compound over long-horizon tasks, creating huge gains in outcomes. https://t.co/2P4Eq6RkcN