零样本GPT-4o当教育顾问爱多管闲事？实测误报率43%，换成Decision Transformer或XGBoost就能准到几乎零误报。
研究在Open University Learning Analytics数据集（N=800，4个时间截点）上发现干预偏差：第56天，oracle标记70.1%学生无需干预，但零样本GPT-4o对73%学生推荐行动，误报率43个百分点。商业RAG和SQL增强检索校准同样差，在10000学生规模下每周期约4300次不必要接触。监督学习方法（ONNX Decision Transformer和XGBoost）校准误差接近零，Decision Transformer达到macro-F1 0.79、macro-recall 0.85，决策延迟低于5毫秒。LLM-as-judge评估（DeepEval G-Eval）无法捕捉干预偏差，反而奖励流畅的过度干预。
Deterministic Decisions for High-Stakes AI. A Zero-Egress Pipeline with the Deployability of RAG and the Accuracy of Machine Learning
We identify intervention bias as a previously unquantified failure mode of zero-shot large-language-model (LLM) educational advisory agents: without task-specific training, they recommend action when a hindsight-optimal oracle policy mandates inaction. In a six-arm ablation on the Open University Learning Analytics Dataset (N=800 students, four temporal cutoffs), at day 56 -- when the oracle designates 70.1% of students as needing no intervention -- zero-shot GPT-4o recommends action for 73%, a 43 percentage-point false-positive rate. Commercial RAG and SQL-augmented retrieval are comparably miscalibrated; at 10,000 students this implies about 4,300 unnecessary advisor contacts per cycle. Supervised policy learning eliminates this bias: a trajectory-conditioned ONNX Decision Transformer (DT) and a snapshot XGBoost classifier, trained on the same oracle-labelled trajectories under strict prefix-only features, both achieve near-zero calibration error. The DT reaches macro-F1 0.79 (macro-recall 0.85) across all five action classes, predicting even the rare load-reduction action without collapsing, at a 0% action flip rate and sub-5 ms CPU decision latency. The two supervised arms are on par; the DT's edge over XGBoost at the final cutoff is indicative only (unpaired across cohorts). Scope: we validate Stage-2 decision-making (EAV state vector to supervised policy) under controlled oracle input from structured OULAD data; high fidelity reflects feature-oracle alignment, not general high-stakes-AI capability. The most robust finding is the intervention-bias contrast, not the absolute accuracies. We also show an Evaluation Gap: LLM-as-judge scoring (DeepEval G-Eval) is blind to intervention bias, rewarding fluent over-prescription rather than decision quality.