VirtualSet让LLM不用冒SQL幻觉的风险,改天用类型安全的set表达式,BIRD上准确率反超4个百分点,写操作还能拦截瞎编的数据输入。
VirtualSet提出将LLM的生成目标从SQL替换为类型化的set expressions,通过Generic Constraint Projection在执行前检测类型错误。在BIRD基准的1,072题测试集上,使用deepseek-reasoner模型,VirtualSet达到67.5%准确率,高于直接SQL的63.5%(McNemar exact p=0.00117)。在一个30个动作的防护测试中,VirtualSet成功拦截全部20个幻觉动作且零误报。该方法在保持SQL基准竞争力的同时,提供了写操作前的语义决策保障。
VirtualSet: Typed Ontology Worlds as an LLM Generation Target for Grounded Queries and Guarded Decisions
Large language models increasingly read and act on enterprise data, but SQL gives a late error signal: hallucinated fields or relations can execute and return plausible wrong answers, while incorrect writes cannot be safely assessed after execution. We present VirtualSet, a live, receiver-typed ontology-world interface and generation target for LLMs. Instead of SQL, the model emits set expressions over entity-edge worlds. Generic Constraint Projection (GCP) checks expressions before execution, while future this preserves concrete receiver types through collection chains, turning invalid fields, edges, receivers, and actions into token-anchored type errors. Type-clean reads use a SQL fast path or bounded stream interpretation, with a parity oracle checking both paths over the exercised operator space. The same substrate supports guarded decisions: actions run first in a simulated world, and world-change events require external approval before actualization. On BIRD, we lift relational schemas into typed worlds and compare VirtualSet with direct SQL while holding the model, evidence, values, zero-shot setting, timeout, glossary, repair/voting, and grader constant where possible. On a frozen 1,072-question split, VirtualSet achieves 67.5% accuracy versus 63.5% for glossary-matched direct SQL with repair and voting (+4.0 points; McNemar exact p = 0.00117) using deepseek-reasoner. Full-corpus analysis finds no engine mis-computation of a type-clean expression; remaining errors arise from model semantics or gold defects. In a 30-body guard corpus, the write chain intercepts 20/20 hallucinated action bodies with zero false positives. VirtualSet thus remains competitive on SQL's home benchmark while providing pre-execution semantics for guarded decisions.
- marktechpost07-19 22:20原文