FLY-EVAL++为飞行预测任务提供新评估标准，发现LLM在安全约束下存在显著差异，比单纯看准确率更重要。
FLY-EVAL++是一种证据驱动的评估协议，专为安全关键环境设计。该协议结合确定性验证和固定评分指南，将模型行为分为合规性、物理可行性和安全性三个维度。研究评估了66个大型语言模型，发现安全性是最具区分度的维度，预测性能相近的模型安全评分差异超过28分。研究还观察到在物理合理预测中存在安全违规问题，以及多步预测中的不稳定性。
FLY-EVAL++: An Evidence-Driven Evaluation Protocol for Safety-Constrained Flight Prediction with Large Language Models
Evaluating large language models (LLMs) in safety-critical, physics-governed environments requires more than accuracy-based metrics, because predictions that are numerically close to the ground truth can still violate operational constraints, combine fields in physically inconsistent ways, or fail to produce usable structured outputs. Existing evaluation protocols do not measure these failure modes reliably. We propose FLY-EVAL++, an evidence-driven evaluation protocol that combines deterministic verification of protocol compliance, physical feasibility, and safety constraints with fixed rubric-guided aggregation into interpretable multi-dimensional scores. We instantiate FLY-EVAL++ for Flight Trajectory and Attitude Prediction (FTAP) by extending the PilotBench setting with history-conditioned and multi-step prediction tasks. Across 66 LLMs, safety compliance is the most discriminative dimension of model behavior: models with comparable predictive performance differ by more than 28 points in safety score, and we observe recurrent failures including safety violations under physically plausible predictions and instability in multi-step rollouts. These results show that evaluation in safety-critical domains should measure constraint satisfaction and structured validity explicitly rather than rely on accuracy-centric reporting alone.