EffiSkel:显式提取效率骨架提升LLM代码生成性能

Chiseling Out Efficiency: Structured Skeleton Supervision for Efficient Code Generation

精选理由

做代码生成或LLM推理优化的开发者,EffiSkel直接解决了生成代码跑得慢的痛点——不用等后优化,训练时就注入效率骨架,值得关注其开源实现。

AI 摘要

大型语言模型生成的代码虽语法正确,但运行速度通常远慢于人类优化代码。现有方法通过后迭代优化或微调模型来提升效率,但未能显式编码高效代码的结构模式。为此,研究者提出EffiSkel框架,通过三种互补策略提取并学习效率骨架(抽象的可复用结构模式),并采用多任务学习联合优化代码生成与骨架预测。实验表明,在Mercury基准上,基于DeepSeek-Coder (7B)的EffiSkel相比EffiCoder和CodeDPO,效率比分别提升11.11%和3.71%,平均加速比分别提升0.36和0.22。该工作为提升LLM生成代码的运行时性能提供了新思路。

原文 · arXiv: DeepSeek

Chiseling Out Efficiency: Structured Skeleton Supervision for Efficient Code Generation

Large Language Models (LLMs) are capable of generating syntactically correct and functionally complete programs, greatly streamlining software development. However, recent studies reveal that these programs typically execute substantially slower than human-optimized counterparts. Existing approaches to bridging this efficiency gap typically involve either iteratively optimizing code after generation or fine-tuning models on corpora of efficient code. Yet, these methods expose the model to efficiency signals only by mimicking complete, optimized solutions, without explicitly encoding the structural code patterns essential for achieving high runtime performance. Addressing this gap presents two core challenges: (1) extracting and representing latent, efficiency-oriented structural patterns embedded within complex syntax and control flows, and (2) effectively learning these patterns without destabilizing the semantic training of LLMs. To tackle these challenges, we propose EffiSkel, an efficiency skeleton-guided framework that explicitly extracts and learns efficiency skeletons-abstract, reusable structural patterns underpinning efficient code-by leveraging three complementary strategies. These skeletons are integrated into a multi-task learning regime that jointly optimizes code generation and skeleton prediction. Experiments across multiple programming languages and benchmarks demonstrate that EffiSkel significantly enhances both functional correctness and efficiency, resulting on Mercury with DeepSeek-Coder (7B) a +11.11% (vs. EffiCoder) and +3.71% (vs. CodeDPO) higher Efficiency Ratio (ER), and a +0.36 (vs. EffiCoder) and +0.22 (vs. CodeDPO) increase in Average Speedup (AS). These results highlight the effectiveness of explicitly modeling efficiency skeletons in improving the runtime performance of code generated by LLMs.

EffiSkel:显式提取效率骨架提升LLM代码生成性能 · AI 热点