做工程设计自动化或LLM多智能体系统的开发者，这个基准能帮你精准定位模型在条件分支、RAG和HPC编排上的短板，建议直接参考EngiAI框架来测试自己的方案。
EngiAI 是一个针对大型语言模型（LLM）在工程设计任务中应用的多智能体框架与基准套件。该基准包含三个评估维度：工作流基准（7种提示风格，涵盖直接工具使用、语义消歧、条件分支等）、检索增强生成（RAG）基准（通过门控评分隔离检索对参数选择的贡献）以及高性能计算（HPC）基准（评估SLURM集群上的端到端ML训练编排）。EngiAI 参考实现基于LangGraph，通过监督架构协调7个专业智能体，统一拓扑优化、文档检索、HPC作业编排和3D打印机控制。在Beams2D问题上，专有模型平均任务完成率达96-97%，而开源4B参数模型为55-78%，条件分支任务最具挑战性（Photonics2D上完成率降至20-53%）。RAG门控验证了检索增强评分接近完美（≈1.0），而无检索时接近零，HPC编排中一个模型100%完成所有步骤，另一个仅50%，揭示了多步骤指令遵循在长工作流中会退化。
EngiAI: A Multi-Agent Framework and Benchmark Suite for LLM-Driven Engineering Design
Large Language Model (LLM) agents are increasingly applied to engineering design tasks, yet existing evaluation frameworks do not adequately address multi-agent systems that combine simulation, retrieval, and manufacturing preparation. We introduce a benchmark suite with three evaluation dimensions: (1) a workflow benchmark with seven prompt styles targeting distinct cognitive demands-including direct tool use, semantic disambiguation, conditional branching, and working-memory tasks; (2) a Retrieval-Augmented Generation (RAG) benchmark with gated scoring isolating retrieval contributions to parameter selection; and (3) an High Performance Computing (HPC) benchmark evaluating end-to-end ML training orchestration on a SLURM cluster. Alongside the benchmark we present EngiAI, a Multi-Agent System (MAS) reference implementation built on LangGraph that operationalizes the benchmark by coordinating seven specialized agents through a supervisor architecture, unifying topology optimization, document retrieval, HPC job orchestration, and 3D printer control. Across four LLM backends and two EngiBench problems, proprietary models achieve 96-97% average task completion on Beams2D, while open-source 4B-parameter models reach 55-78%, with clear generational improvement. Conditional branching proves most challenging, with task completion dropping to 20-53% for the conditional style on Photonics2D. RAG gating confirms near-perfect retrieval-augmented scores ($\approx 1.0$) versus near-zero without retrieval, validating the evaluation design. On HPC orchestration, one model completes all pipeline steps in 100% of runs while another drops to 50%, revealing that multi-step instruction following degrades over long-running workflows.