想看你用的LLM在核工程上有多靠谱？NuclearQAv2用1240道硬核题测出模型的定量推理短板，比通用基准更实在。
NuclearQAv2是一个专为核工程领域设计的基准测试，包含约1240个问答对，涵盖布尔、数值和文字三类问题。该基准采用混合流程，结合专家编写、现有数据集和LLM辅助生成。评估多种LLM发现，模型在事实性问答上表现良好，但在定量推理和概念理解上存在明显短板。NuclearQAv2提供了一种可扩展的方案，用于系统衡量大语言模型在技术领域的实际能力。
NuclearQAv2: A Structured Benchmark for Evaluating Domain-Science Competence in Large Language Models
Large language models (LLMs) have demonstrated strong performance across a wide range of tasks, but ensuring their reliability in highly technical domains remains a significant challenge. In nuclear engineering, problem solving often requires not only factual knowledge but also quantitative reasoning and conceptual understanding. To address the need for systematic evaluation in this domain, we introduce NuclearQAv2, a benchmark for assessing LLMs on nuclear engineering knowledge. The benchmark comprises approximately 1,240 question-answer pairs spanning three categories: boolean, numeric, and verbal. NuclearQAv2 is constructed using a hybrid pipeline that combines expert-authored questions, existing datasets, and LLM-assisted generation from domain-specific technical corpora. By leveraging structured prompting for both automated question generation and response evaluation, the proposed framework enables scalable benchmark construction and evaluation. We evaluate a diverse set of LLMs using NuclearQAv2 and observe substantial performance differences across task types. While the models generally perform well on factual questions, quantitative reasoning and conceptual understanding remain considerably more challenging. These results highlight the importance of multi-faceted evaluation frameworks and establish NuclearQAv2 as a scalable benchmark for assessing LLM capabilities in technical domains.