这篇论文用置信序列代替传统联合界方法，让MDP采样效率平均提升50倍，做在线统计模型检验的人可以看看。
本文提出用于Markov决策过程（MDP）在线统计模型检验的置信序列方法。传统方法依赖已知转移概率或通过子优采样策略，而新方法利用在线置信序列避免了经典的联合界（union-bound）风格。作者实现了所有方法并证明其比先前最先进技术平均减少50倍样本量。实验表明，新方法在保证统计保证的同时显著提升采样效率。
Markov decision processes (MDPs) are a classic model of decision making under uncertainty, exhibiting both non-deterministic choice as well as probabilistic uncertainty. Traditionally, exact knowledge of the underlying probabilities is assumed. However, this often is unrealistic, e.g.\ when modelling cyber-physical systems or biological processes. Here, statistical methods provide a way towards obtaining meaningful guarantees. The classical approach is to gather samples in the MDP, use these to draw statistical conclusions about the transition probabilities, and from there obtain bounds on the true value; then, if these bounds are too broad, repeat. However, existing implementations of this approach are either subtly incorrect or sub-optimal, and quite often both. We present several \emph{confidence sequences}, which are specifically designed for such \enquote{online} settings, implement all of them in an efficient tool, and show their practical applicability. In particular, we show that they outperform classical \enquote{union-bound} style approaches, and overall our implementation requires 50x less samples on average than previous state of the art.