研究者用决策树“拆解”了黑箱TD3,在倒立摆任务上性能不减,还解析了控制规律,适合安全关键AI参考。
在倒立摆基准上,TD3教师模型(Twin Delayed DDPG)的策略被蒸馏至浅层决策树学生。通过自定义物理感知特征和噪声Oracle Rollouts生成数据集,学生模型性能与教师相当。控制理论分析显示从连续控制转向离散规则控制导致高频Bang-Bang执行和稳定双模态极限环。BIBO稳定性得到维持,同时提供了全局和局部可解释性。
Explainable Reinforcement Learning via Physics-Aware Policy Distillation
In safety-critical sectors such as robotics and automotive engineering, the deployment of Deep Reinforcement Learning (DRL) is often hindered by the black-box nature of deep neural networks. This lack of transparency poses significant challenges for regulatory compliance and human-agent trust. This paper presents an experimental study aimed at making high-performance continuous control DRL systems interpretable. A policy distillation framework is implemented using the classic Inverted Pendulum benchmark. A high-performance Twin Delayed DDPG (TD3) agent serves as an opaque, continuous teacher model, whose policy is distilled into an interpretable student surrogate based on a shallow Decision Tree. By leveraging a custom physics-aware feature and "Noisy Oracle Rollouts" for dataset generation, the distillation process achieves performance equivalent to the expert teacher. Furthermore, comparative control theory analysis reveals a fundamental trade-off: transitioning from continuous to discrete rule-based control induces high-frequency Bang-Bang actuation and a stable bimodal limit cycle. Simulation results indicate that Bounded-Input Bounded-Output (BIBO) stability is maintained while providing both global and local interpretability for safe autonomous systems.