Lighthouse RL用巧妙的‘灯塔’重置点大幅提升电路优化效率，样本快1.72倍，成功率拉满到100%。搞硬件优化的千万别错过。
本文提出Lighthouse RL，一种样本高效的强化学习方法，用于模拟电路尺寸优化。传统方法难以泛化到不同性能目标，标准RL则在无前途区域浪费资源。该方法通过从训练中发现的高性能配置（称为'lighthouses'）初始化回合，引导探索到有前景区域。在2D基准问题和两个模拟电路上，相比RL和贝叶斯优化，样本效率提升高达1.72倍，成功率从0-87%提升至100%，泛化成功率从0-50%提升至75%。该重置策略可作为任何基于RL的优化方法的即插即用增强。
Lighthouse RL: Sample-Efficient Circuit Optimization via Strategic Reset Points
In this paper, we introduce Lighthouse RL, a sample-efficient reinforcement learning (RL) approach for analog circuit sizing. Traditional methods lack generalization across different performance targets, while standard RL approaches waste resources exploring unpromising regions. Our method addresses these inefficiencies through a strategic reset strategy that initializes episodes from high-performing configurations discovered during training, called "lighthouses". These states, which are closer to the target objectives, guide exploration toward promising regions. When compared to RL and Bayesian optimization methods from the literature, we demonstrate the effectiveness of our approach on a 2D benchmark problem and on two analog circuits, showing significant improvements in sample efficiency (up to 1.72x faster), optimization performance (100% vs. 0-87% success rate), generalization (75% vs. 0-50% extrapolation success), and objective maximization. This efficiency is particularly valuable for computationally expensive black-box optimization problems, and our reset strategy can be used as a plug-and-play enhancement for any RL-based optimization approach.