论文研究自主研究在电信工单检索任务中的应用
Autonomous Research for Open-Ended Problems: A Case Study on Telecom Ticket Retrieval
朋友，这篇论文讲的是用AI自己研究解决一个实际问题的方法，挺有意思的，比如电信工单检索，AI自己研究比人类快10倍还便宜。
这篇论文通过电信工单检索案例，探讨了如何将自主研究方法应用于开放性问题。研究发现，使用商业和开源代理的自主研究在超参数优化方面表现良好，但在需要人类直觉和创造力的任务上存在局限。在10周内，自主研究达到了90%的SOTA性能（0.34 vs. 0.38 Recall@1），比人类工作快10倍，成本仅约200美元。
Autonomous Research for Open-Ended Problems: A Case Study on Telecom Ticket Retrieval
Recent breakthroughs in LLM-based systems and their abilities in problem solving and coding have allowed progress in the AI for Science paradigm, potentially replacing human roles in machine learning (ML) research. However, while several frameworks of fully autonomous end-to-end ML research have been proposed, successful implementations of them are often limited to problems with narrow search spaces, like language modeling or biomedical ML benchmarks. In this paper, we explore how autonomous research can be adapted to solve open-ended, industry-grade ML problems, by considering a case study: telecom ticket retrieval, an open-ended task with degrees of freedom in representation, architecture, and training data generation. We discover that autonomous research for open-ended problems with commercial and open-source agents shows both promise and limitations: while autonomous research can excel in narrow hyperparameter optimization, it lacks human-like intuition and creativity and requires operational overhead. Even with minimal human supervision, autonomous research can reach $90\%$ of state-of-the-art performance (0.34 vs. 0.38 Recall@1) in a much shorter time period (10 weeks vs. 10 months of human work) at a modest cost (up to \$200 per Cursor campaign). Our empirical evidence recommends that human researchers and autonomous research frameworks work together for best results in ML research.
