论文精选

CoDyControlBench:复杂动力系统反馈控制器设计的LLM基准与推理蒸馏

Benchmarking and Reasoning Distillation of Large Language Models for Feedback Controller Design in Complex Dynamical Systems

精选理由

论文搞了个新基准CoDyControlBench,测了6款模型,GPT成功率最高94.8%,还蒸馏出1.5B小模型能上机械臂干活。

AI 摘要

论文提出CoDyControlBench基准,包含132个系统配置,覆盖自由度、系统类型、耦合度等五个维度。评估了GPT、Gemini、Claude、GLM、DeepSeek、Qwen六款模型,GPT设计成功率最高达94.8%,Qwen最低为50.0%。分析显示自由度与控制器类型对成功率影响最大,差距主要来自增益选择与瞬态限制机制。通过推理蒸馏得到1.5B参数模型,在基准上胜过答案蒸馏模型,并在气动人工肌肉机械臂的三次物理试验中全部成功跟踪目标。

原文 · arXiv: DeepSeek

Benchmarking and Reasoning Distillation of Large Language Models for Feedback Controller Design in Complex Dynamical Systems

Although remarkable capabilities have been demonstrated by Large Language Models (LLMs) across scientific domains, feedback controller design remains underexplored. Existing benchmarks focus mainly on linear single-Degree-of-Freedom (DoF) systems and large API-hosted models, leaving performance on complex controller-design tasks and feasibility for edge deployment unclear. To address these limitations, we introduce the Complex Dynamics-to-Control Benchmark for Large Language Models (CoDyControlBench), comprising 132 system configurations across five evaluation dimensions: number of DoF, system type, coupling level, damping regime, and controller type. Six state-of-the-art LLMs were evaluated over three independent runs, including three commercial models (GPT, Gemini, and Claude) and three open-source models (GLM, DeepSeek, and Qwen). GPT achieved the highest design success rate at 94.8\%, whereas Qwen showed the lowest rate at 50.0\%. Across the benchmark dimensions, DoF and controller type exhibited the largest model-averaged variations in design success, with success-rate ranges of 36.3\% and 17.6\%, respectively, exceeding those associated with system type, coupling level, and damping regime. Comparison of GPT and Qwen showed that their performance gap arose mainly from the control-design knowledge, particularly gain selection and the use of transient-limiting mechanisms. For edge deployment, a specialized 1.5B-parameter model was developed through reasoning distillation. The reasoning-distilled model outperformed the answer-distilled and base model on CoDyControlBench, maintained stable performance across 1-6 DoFs, and achieved successful traget tracking in all three physical trials on a pneumatic-artificial-muscle-driven robotic arm. These results establish a benchmark baseline and highlight the potential of lightweight, edge-deployable controller-design models.