CQ4OE：面向 LLM 本体生成评估的基准发布
CQ4OE: A benchmark for assessing LLM-assisted ontology generation from competency questions
ontology 工程师可以看看这个新基准，测了 9 个 LLM 从 competency questions 生成本体的能力，连失败模式都拆得很细。
arXiv 论文提出 CQ4OE 基准，用于系统评估大模型从 Competency Questions（CQ）生成本体的能力。基准为每个本体构建带明确来源标注的 OWL 金标本体，将每条 CQ 关联到对应的类、属性和公理。评估包含两个任务：CQ2Term 覆盖 99 条 CQ 的词条级预测，CQ2Onto 覆盖 118 条 CQ 的本体级评估，涉及层级、属性建模和公理结构。作者用 9 个 LLM 在零样本、迭代和多智能体三种策略下实验，结果显示 LLM 生成显式词条比构建完整本体更可靠，短板集中在属性建模、层级构建和公理生成。
CQ4OE: A benchmark for assessing LLM-assisted ontology generation from competency questions
Ontology generation from Competency Questions (CQs) is a central yet labor-intensive phase of Ontology Engineering. While large language models (LLMs) offer promising automation capabilities, current evaluations remain fragmented. Task formulations are heterogeneous, gold standards often lack fine-grained CQ provenance, metrics conflate lexical overlap with structural and logical adequacy, and reference ontologies are not always explicitly designed around the evaluation CQs. Here, we address these limitations with CQ4OE, a benchmark for the systematic and reproducible evaluation of LLM-based ontology generation from CQs. For each ontology in the benchmark, we build a CQ-driven gold OWL ontology with explicit provenance linking each CQ to the classes, properties, and axioms required to answer it. From this resource, we define two complementary evaluation tasks. CQ2Term supports term-level evaluation of CQ-specific class and property prediction over 99 CQs, and CQ2Onto supports ontology-level evaluation over 118 CQs, including hierarchy, property modeling, and axiom-level structure. We demonstrate CQ4OE with experiments using nine LLMs under zero-shot, iterative, and multi-agent generation strategies, showing that LLMs recover explicit vocabulary terms more reliably than creating ontologies, particularly in property modeling, hierarchy construction, and axiom generation.