Roblox 提出搜索感知强化学习框架优化查询理解
Search-Aware Reinforcement Learning for Multi-Component Query Understanding in Roblox Game Search
Roblox 把游戏搜索的查询理解拆成多组件，用搜索引擎实时反馈做奖励来训练 LLM，NDCG@20 提了 8.9 点，做法挺特别。
Roblox 发表论文，针对游戏搜索中的查询理解（QU）任务提出搜索感知强化学习（RL）框架，采用先蒸馏后 RL 的训练范式。先用师生监督微调（SFT）得到符合 schema 的策略初始化，再用来自搜索引擎实时交互的奖励分别优化每个 QU 组件。在 Roblox 搜索上的实验显示，NDCG@20 比 SFT 策略提升 8.9 点，比单一端到端奖励训练提升 3.5 点。
Search-Aware Reinforcement Learning for Multi-Component Query Understanding in Roblox Game Search
Query understanding (QU) plays a critical role in production search systems, translating raw user queries into search execution plans that drive downstream retrieval and ranking. While large language models (LLMs) have enabled QU to be framed as a structured multi-task generation problem (e.g., intent classification, query expansion), optimizing such models to produce search-engine-coupled outputs remains challenging: static, label-based supervision fails to capture how each component actually interacts with the underlying search pipeline to affect downstream performance. We present a search-aware reinforcement learning (RL) framework for QU based on a distill-then-RL paradigm. Teacher-student supervised fine-tuning (SFT) first yields a well-formed, schema-compliant policy initialization. The RL stage then optimizes each QU component with rewards derived from live interaction with the search engine, tailored to that component's operational role, rather than a single reward tied to the final search outcome. Experiments on Roblox search show that this component-specific optimization improves both per-component utility and downstream search quality, raising NDCG@20 by 8.9 points over the SFT policy and by 3.5 points over training with a single end-to-end reward.
