这篇论文用表格Q学习做分布式无人机轨迹规划，在200辆车场景下效果逼近集中式深度强化学习，还省带宽省电，值得一看。
该论文提出了TRUAV框架，一种基于独立表格Q学习的分布式多智能体强化学习方法，用于无人机辅助车联网中的联合轨迹规划和路由增强。每个无人机使用本地可观测信息（车辆密度、数据包队列状态、邻居无人机位置）进行决策，无需全局状态交换。在包含200辆移动车辆的大规模城市区域仿真中，TRUAV实现了与集中式深度强化学习方法相当的网络覆盖率和数据包投递率，同时降低了中继延迟并提升了能效。论文还讨论了下分布式多智能体无人机辅助物联网系统的挑战和未来方向。
TRUAV: Distributed Multi-Agent Reinforcement Learning for Trajectory Planning and Routing Enhancement in UAV-Aided IoT-Enabled VANETs
Unmanned aerial vehicles (UAVs) have emerged as a key enabler of next-generation Internet of Things (IoT) ecosystems, offering flexible aerial relaying to extend connectivity across dynamic vehicular ad hoc networks (VANETs) in smart city environments. However, conventional centralized approaches for UAV trajectory planning require continuous global network state aggregation, making them impractical under bandwidth and energy constraints typical of dense urban deployments. In this article, we present TRUAV, a distributed multi-agent reinforcement learning framework based on independent tabular Q-learning for joint UAV trajectory planning and routing enhancement in UAV-aided VANETs. Each UAV is equipped with a local Q-learning agent that operates purely on locally observable information, including vehicle density, packet queue states, and neighbor UAV positions, thereby eliminating the need for global state exchange. A potential-game-inspired reward design encourages spatial diversity and routing-aware UAV positioning among interacting agents while accounting for energy consumption. Numerical simulations over a large urban area with 200 mobile vehicles show that the proposed TRUAV framework achieves network coverage and packet delivery ratios comparable to centralized deep reinforcement learning methods, while also improving relay delay and energy efficiency. Finally, we discuss emerging challenges and future research directions for distributed multi-agent UAV-assisted IoT systems.