这篇论文把Q-learning的Bellman目标正则性研究透了，还给出了DeepONet的近似界限，适合搞理论强化学习的人细读。
该论文研究了连续时间随机控制中Q-learning的算子理论核心，在均匀椭圆性和Hölder正则系数条件下，证明了Bellman更新将有界输入映射到各向异性正则类，状态变量被平滑而动作变量仅保持Lipschitz依赖。论文提出了适应混合正则性的张量积DeepONet架构，并给出了显式近似和资源界限以及时间步δ→0时的刚度-复杂度权衡。作者未声称对带探索、经验回放和随机梯度更新的实际采样Q-learning有完整的收敛定理。
Deep Q-Learning on Hölder Spaces
We study the operator-theoretic core of Q-learning in continuous-time stochastic control with continuous states and actions. In value-based reinforcement learning, each Q-learning or DQN update is built from a Bellman optimality target; our analysis isolates this target in a diffusion setting and studies its regularity and approximation complexity. Under uniform ellipticity and Hölder-regular coefficients, we show that a Bellman update maps bounded inputs into an anisotropic regularity class, smoothing the state variable while leaving only Lipschitz dependence on the action variable. This yields a compact family of Bellman iterates and motivates a tensor-product DeepONet architecture adapted to the mixed regularity of the problem. We then derive explicit approximation and resource bounds, together with a stiffness--complexity trade-off as the time step $δ\to 0$. The resulting theory makes a direct contribution to Q-learning theory at the level of Bellman target regularity and approximation in continuous stochastic control. At the same time, we do not claim a full convergence theorem for practical sampled Q-learning with exploration, replay, and stochastic gradient updates.