分阶段混合:通过知识蒸馏实现视觉量子强化学习

Staged Hybridisation for Visual Quantum Reinforcement Learning via Knowledge Distillation

精选理由

这篇论文用蒸馏方法让量子电路也能搞定视觉强化学习,实验做得扎实,省去从头训练的麻烦。

AI 摘要

论文提出分阶段知识蒸馏(KD)策略,用于视觉量子强化学习(QRL)。先训练经典视觉老师模型,冻结编码器作为特征接口,将老师策略行为蒸馏到紧凑下游头中。下游头可以是经典或变分量子电路(VQC)的。在CartPole Pixels和Acrobot Pixels环境上评估,结果显示浅层VQC头能在直接像素训练困难的情况下获得非平凡视觉控制行为。角度编码VQC头保持接近老师性能(约90%成功率),而幅度编码头更紧凑(参数减少70%)但更脆弱。

原文 · arXiv cs.LG

Staged Hybridisation for Visual Quantum Reinforcement Learning via Knowledge Distillation

Visual environments are a demanding setting for quantum reinforcement learning (QRL): high-dimensional observations, unstable RL optimisation, and constrained variational quantum circuits (VQCs) are difficult to train jointly. This paper studies knowledge distillation (KD) as a staged hybridisation strategy for visual QRL. Instead of training a hybrid visual agent end-to-end from pixels, we first train a classical visual teacher, freeze its encoder as a feature interface, and distil the teacher's policy behaviour into compact downstream heads. These heads can be classical or VQC-based, enabling small quantum-compatible students to be evaluated under the same frozen representation as compact classical controls. We evaluate the pipeline on CartPole Pixels and Acrobot Pixels. The results show that staged KD enables shallow VQC heads to acquire non-trivial visual-control behaviour in settings where direct pixel-based training would be substantially more difficult. Angle-encoded VQC heads retain near-teacher performance, while amplitude-encoded heads push compactness to an extreme regime, at the cost of greater fragility, stronger budget sensitivity, and higher simulation time. Overall, staged KD reframes visual QRL as a compact-head learning problem, opening a practical route for training small quantum-compatible policies outside the standard end-to-end RL loop.