做优化算法选型或自动化机器学习的研究者值得关注——这项研究直接点出了学术基准与真实场景的鸿沟,看完能帮你避开模型部署的坑。
该研究系统评估了算法选择(AS)模型在合成与真实优化场景间的泛化能力。研究使用了BBOB和CEC两个学术基准套件,以及机器人轨迹优化和无人机路径规划两个真实问题集。通过跨基准测试发现,AS模型在学术基准间表现良好,但迁移到真实领域时泛化能力显著下降。研究揭示了当前AS方法在领域特定应用中的鲁棒性挑战,为开发更可靠的实用AS系统提供了方向。
Evaluating Real-World Generalizability of Algorithm Selection Models
Algorithm Selection (AS) aims to automatically identify the most suitable optimization algorithm for a given problem instance by leveraging measurable problem characteristics and historical performance data. In this study, we investigate the generalization ability of AS models across both synthetic and real-world optimization landscapes. We consider two widely used academic benchmark suites (BBOB and CEC) and two real-world problem sets (robotics trajectory optimization tasks and unmanned aerial vehicle path-planning problems). Through a systematic cross-benchmark evaluation, we analyze how AS models transfer between domains, identify where generalization succeeds or breaks down, and highlight the challenges that arise when applying AS in realistic, domain-specific contexts. Our findings provide insights into the robustness of current AS approaches and inform the development of more reliable, broadly applicable AS systems for real-world optimization.