多智能体代码生成团队终于有了低成本的质量评估工具——FASE用0.3%的计算成本实现更优的代码正确性预测,做自动化软件开发的工程师可以直接集成到工作流中。
多智能体代码生成系统常因LLM幻觉和错误传播而可靠性不足。现有语义熵方法虽能量化不确定性,但依赖昂贵的LLM等价性检查。新提出的FASE指标通过结构/语义差异图的最小生成树近似功能正确性,无需LLM参与。在HumanEval和BigCodeBench上,FASE相比传统语义熵在Spearman相关性上平均提升25%,ROCAUC提升19%,而计算成本仅为传统方法的0.3%。这使得FASE成为多智能体工作流中实用且经济的质量评估方案。
FASE: Fast Adaptive Semantic Entropy for Code Quality
Multi-agent code generation offers a promising paradigm for autonomous software development by simulating the human software engineering lifecycle. However, system reliability remains hindered by LLM hallucinations and error propagation across interacting agents. While semantic entropy provides a principled way to quantify uncertainty without ground-truth answers, current methods often rely on costly LLM-driven equivalence checks. In this work, we introduce Fast Adaptive Semantic Entropy (FASE), a novel metric that approximates functional correctness based on the minimum spanning tree of structural and semantic dissimilarity graphs. Evaluations on HumanEval and BigCodeBench demonstrate that FASE outperforms state-of-the-art semantic entropy by LLM entailment, achieving a 25% average improvement in Spearman correlation and a 19% increase in ROCAUC score against Pass@1 from ground-truth test cases when using the Qwen3-Embedding-8B model. Furthermore, by eliminating costly LLM-driven equivalence evaluation, FASE incurs negligible computational overhead, requiring only approximately 0.3% of the runtime cost of traditional semantic entropy approaches. These results position FASE as a practical, cost-effective solution for optimizing uncertainty quantification in real-world multi-agent workflows.