VirtualSet:用类型化本体世界作为LLM生成目标实现有根基的查询和受控决策

VirtualSet: Typed Ontology Worlds as an LLM Generation Target for Grounded Queries and Guarded Decisions

精选理由

用VirtualSet替代SQL,让LLM查询出错时提前报类型错误而不是给假结果,BIRD上还比SQL高4个点,很适合做安全决策。

AI 摘要

论文提出VirtualSet,一种接收者类型化的本体世界接口,替代SQL作为LLM的生成目标。在BIRD基准测试的1072题子集上,VirtualSet配合deepseek-reasoner达到67.5%准确率,比带修复和投票的SQL高4.0个百分点(p=0.00117)。系统通过通用约束投影(GCP)在表达式执行前检查,并利用集合链保留具体接收者类型将无效字段转为类型错误。在30个受控决策用例中,写入链拦截了20/20的幻觉动作体且无假阳性。

原文 · arXiv: DeepSeek

VirtualSet: Typed Ontology Worlds as an LLM Generation Target for Grounded Queries and Guarded Decisions

Large language models increasingly read and act on enterprise data, but SQL gives a late error signal: hallucinated fields or relations can execute and return plausible wrong answers, while incorrect writes cannot be safely assessed after execution. We present VirtualSet, a live, receiver-typed ontology-world interface and generation target for LLMs. Instead of SQL, the model emits set expressions over entity-edge worlds. Generic Constraint Projection (GCP) checks expressions before execution, while future this preserves concrete receiver types through collection chains, turning invalid fields, edges, receivers, and actions into token-anchored type errors. Type-clean reads use a SQL fast path or bounded stream interpretation, with a parity oracle checking both paths over the exercised operator space. The same substrate supports guarded decisions: actions run first in a simulated world, and world-change events require external approval before actualization. On BIRD, we lift relational schemas into typed worlds and compare VirtualSet with direct SQL while holding the model, evidence, values, zero-shot setting, timeout, glossary, repair/voting, and grader constant where possible. On a frozen 1,072-question split, VirtualSet achieves 67.5% accuracy versus 63.5% for glossary-matched direct SQL with repair and voting (+4.0 points; McNemar exact p = 0.00117) using deepseek-reasoner. Full-corpus analysis finds no engine mis-computation of a type-clean expression; remaining errors arise from model semantics or gold defects. In a 30-body guard corpus, the write chain intercepts 20/20 hallucinated action bodies with zero false positives. VirtualSet thus remains competitive on SQL's home benchmark while providing pre-execution semantics for guarded decisions.

VirtualSet:用类型化本体世界作为LLM生成目标实现有根基的查询和受控决策 · AI 热点