这篇论文为做强化学习或机器人部署的团队提供了一个严谨的框架，帮你判断什么时候该相信模拟器、什么时候该做真实实验。做仿真到真实迁移的开发者可以直接参考其Fisher-SEP算法来优化实验预算。
这篇论文研究了在模拟器与真实实验之间如何权衡的问题。模拟器成本低但存在校准偏差，真实实验无偏但成本高。作者提出了一个扩展的模拟引理，将模拟器的价值误差分解为可识别的校准-部署偏移和不可减少的参数残差。他们还分析了模拟器最优策略与真实最优策略之间的价值差距，分为局部和可达性两部分。最后，提出了Fisher-SEP算法，通过最小化目标策略价值的后验预测方差来指导实验设计，并在自动售货机供应链和HIV移动测试两个案例中验证了其有效性。
Suppose a planner has a pre-trained simulator of a sequential decision problem and the option to run real experiments in the field. The simulator is cheap to query but inherits confounding and drift from its calibration data. Experimentation is unbiased but consumes one real unit per trial. We study when, and how, the planner should supplement the simulator with experiments. We give three results. First, an extended simulation lemma decomposes the simulator's value error into a calibration--deployment shift that randomization can identify and a parametric residual that no further interaction can reduce. Second, the value gap between the simulator-optimal policy and the optimum splits into a local component, on states the deployed policy already visits, and a reachability component, on states it does not. The reachability component stays bounded away from zero at any horizon under purely passive learning. Third, we propose Fisher-SEP, a simulation-aided experimental policy (SEP) that minimizes the posterior predictive variance of a target policy's value, with reward-only and transition-only specializations. Two case studies illustrate the regimes. In a vending-machine supply chain, front-loaded experimentation overtakes posterior updating once the horizon is long enough to amortize the pilot. In an HIV mobile-testing example with a corridor that separates a well-surveilled region from a poorly-surveilled one, only designed exploration reaches the poorly-surveilled region.