DoorDash 用离线强化学习解决调度权重调整难题，做物流调度或平台经济的团队可以借鉴其安全部署思路。
DoorDash 部署了一套基于离线强化学习的系统，通过延迟的市场反馈（如配送速度、骑手利用率、商家拥堵）来动态调整配送调度目标的权重。该系统不替换原有的组合优化调度器，而是在门店层面学习一个策略，选择离散乘数来调整调度器在配送质量与批处理效率之间的权衡。通过集中式离线数据和分散式门店执行训练共享价值函数，并采用 Double Q-learning 和保守正则化减少过估计。生产环境切换实验表明，该策略在不降低客户配送质量的前提下，提高了批处理效率并减少了骑手时间成本。这项工作展示了如何利用真实经济物流系统的反馈安全地在线调整决策策略。
Multi-Agent Reinforcement Learning from Delayed Marketplace Feedback for Objective-Weight Adaptation in Three-Sided Dispatch
Dispatch in three-sided marketplaces provides a natural setting for reinforcement learning from world feedback: decisions are evaluated by delayed operational outcomes such as delivery speed, courier utilization, and merchant congestion. We present a deployed reinforcement learning system at DoorDash that adapts dispatch objective weights in a large-scale food-delivery marketplace using delayed signals. Rather than replacing the combinatorial assignment optimizer, a store-level policy learned from logged marketplace data selects a discrete multiplier that shifts the dispatch optimizer's tradeoff between delivery quality and batching efficiency. This interface enables offline policy learning under noisy, delayed, and coupled feedback while preserving production feasibility constraints and operational safeguards. We train a shared value function using centralized offline data and decentralized store-level execution, with Double Q-learning targets and a conservative regularizer to reduce out-of-distribution value overestimation. In a production switchback experiment, the offline-trained policy increases batching and reduces courier-side time costs without degrading customer-facing delivery quality. Results illustrate how world feedback from a live economic and logistics system can be used to safely adapt decision policies online.