NVIDIA 论文研究多模型系统中的模型选择策略
Banger paper from NVIDIA. It's on the topic of choosing which models go into a multi-agent system. ...
NVIDIA 的这篇论文研究得挺有意思，讲的是怎么选模型组成多模型系统，比如多数投票和 LLM 作为裁判的效果，以及不同模型组合的优劣对比。
该论文比较了八种模型选择策略，基于规模、准确率、答案多样性和错误多样性，在路由、多数投票和 LLM 作为裁判的设置下，在硬科学基准上测试。使用多个相同模型的副本效果更好。多数投票将 HLE 准确率从 29.4% 提升至 32.2%，但混合模型组大多表现下降。从单一模型家族中选择候选模型，比所有八种策略中任何一种单独模型都带来了更大的提升。
Banger paper from NVIDIA. It's on the topic of choosing which models go into a multi-agent system. ...
Banger paper from NVIDIA. It's on the topic of choosing which models go into a multi-agent system. The team compared eight selection strategies, based on size, accuracy, answer diversity and error diversity, across routing, majority vote and LLM-as-judge setups on hard science benchmarks. Larger pools of different open models raised the theoretical best-case accuracy. Achieved accuracy often fell below the single best model in the pool. Using several copies of one model worked better. Majority vote over the best single model raised HLE accuracy from 29.4% to 32.2%, while nearly every mixed-model group declined. Choosing candidates from a single model family gave the largest improvement over a standalone model of all eight strategies. Before adding another model to a router or ensemble, measure what it adds. Paper: arxiv.org/abs/2609.17306 Chat with Paper: academy.dair.ai/papers/mo-mode… 💬 2 🔄 1 ❤️ 6 👀 1143 📊 4 ⚡
