这篇论文为Solidity智能合约代码生成建了个新基准（5470个合约）和专用评分指标，测试了多个主流代码模型的各种方法，结论明确：靠谱的领域数据+微调最管用。
该论文提出了SolidityBench，包含5,470个存储库级Solidity智能合约及其自然语言描述。同时提出SolidityScore，一种关注安全性修饰符、合约声明等域关键结构的语义度量。研究评估了Qwen2.5-Coder、DeepSeek-Coder和CodeLlama等模型在零样本、思维链、上下文学习、检索增强生成和监督微调五种方法上的表现。结果显示，通用模型在存储库级Solidity生成中存在结构性缺陷；在非参数方法中，检索增强生成效果最佳，而上下文学习在超过两个示例后因上下文饱和而性能下降；监督微调通过将Solidity特定约束内化到模型参数中实现了最大改进。
Repository-Level Solidity Code Generation with Large Language Models: From Prompting to Fine-Tuning
Large Language Models (LLMs) have shown strong capabilities in general-purpose code generation, but their effectiveness in specialized software domains remains underexplored. Solidity smart contracts represent a high-stakes domain where generated code must satisfy strict language-level, security, and software-engineering constraints. Existing benchmarks and metrics remain insufficient for repository-level Solidity generation, where models must synthesize complete contracts from natural language requirements. To address this gap, we introduce SolidityBench, a benchmark of 5,470 repository-level Solidity smart contracts paired with natural language descriptions. We also propose SolidityScore, a Solidity-aware semantic metric that emphasizes domain-critical constructs such as security modifiers, contract declarations, and Solidity-specific keywords. Using this benchmark, we evaluate representative code LLMs, including Qwen2.5-Coder, DeepSeek-Coder, and CodeLlama, across zero-shot prompting, Chain-of-Thought reasoning, in-context learning, retrieval-augmented generation, and supervised fine-tuning. The results show that general-purpose models exhibit systematic structural deficiencies in repository-level Solidity generation. Among non-parametric methods, retrieval-augmented generation performs best, while in-context learning degrades beyond two examples due to context saturation. Supervised fine-tuning achieves the largest improvement by internalizing Solidity-specific constraints into model parameters. Overall, our study provides a comprehensive benchmark for repository-level Solidity code generation and shows that high-quality domain data combined with supervised fine-tuning is the most effective strategy for improving the reliability of LLM-generated smart contracts.