论文精选73°

韩国开放API基准与多步工具调用数据合成

Multi-Step Tool-Calling over Korean Open Public APIs: A Benchmark and a Data-Synthesis Recipe

精选理由

韩国团队推出首个多步工具调用基准,用动态图方法合成数据,小模型也能接近大模型效果。

研究人员发布KOPA-Bench基准,包含145个真实世界任务,测试多步工具调用能力。他们提出EDGE方法,通过动态图构建和实时执行验证来合成多步轨迹数据。使用GRPO微调的90亿参数模型在KOPA-Bench和BFCL基准上表现接近未调优的270亿参数模型。

原文 · arXiv cs.AI

Multi-Step Tool-Calling over Korean Open Public APIs: A Benchmark and a Data-Synthesis Recipe

Data-sovereignty regulations increasingly require public institutions to deploy open-source, on-premise LLM agents that chain multiple tool-calls across live government APIs. However, open-source models consistently underperform in this multi-step setting, and no existing benchmark measures the gap. We introduce the Korean Open Public API Benchmark (KOPA-Bench), comprising 145 real-world tasks. To close this gap, we present EDGE, an Execution-grounded Dynamic Graph for tool-calling data synthEsis driven by live execution. EDGE builds a graph of how each tool's output can feed another's input, keeps only the links that succeed when actually called against the live APIs, and traverses these verified links to synthesize executable multi-step trajectories. Fine-tuned via GRPO on the resulting dataset, our 9B model nearly matches the untuned 27B model from the same family, improving substantially not only on KOPA-Bench but also on the BFCL benchmark.