这篇论文给预训练的动作分块变换器加了强化学习后训练，在工业精密操作任务上峰值力降了46倍，适合做实时控制的人看。
研究提出PAC-ACT，一种针对预训练动作分块变换器（ACT）的强化学习后训练框架。该方法在分块级别重新定义策略优化，构建ACT迁移的演员-评论家架构，并引入混合行为先验约束以在在线微调中保持预训练动作分布。在工业精密接触基准上，PAC-ACT提升了任务成功率、接触稳定性与力安全性。在Contour任务中，峰值接触力显著降低，60N以上力读数比例减少46倍。稀疏奖励实验表明行为先验约束能在随机初始位姿下实现有效探索。
PAC-ACT: Post-training Actor-Critic for Action Chunking Transformers
Precision industrial contact manipulation requires reliable robot policies under pose perturbations and contact-force constraints. Vision-language-action models offer broad generalization but often introduce high inference latency and GPU-memory cost, while vision-action chunking policies are more suitable for real-time industrial control. However, these policies are usually trained by behavior cloning and suffer from distribution shift in contact-rich tasks. This paper proposes PAC-ACT, a reinforcement-learning post-training framework for pretrained Action Chunking Transformer policies. PAC-ACT reformulates policy optimization at the chunk level, constructs an ACT-transferred actor-critic architecture, and introduces a hybrid behavior-prior constraint to preserve the pretrained action distribution during online fine-tuning. Experiments on industrial precision-contact benchmarks show that PAC-ACT improves task success, contact stability, and force safety while retaining low latency and low GPU-memory usage. On the Contour task, PAC-ACT significantly reduces peak contact force and decreases the proportion of force readings above 60 N by 46 times. Sparse-reward ablations further show that the proposed behavior-prior constraint enables effective exploration under randomized initial poses.