SimRec 用强化学习训练推荐智能体的完整交互策略
SimRec: Learning to Search, Ask, and Recommend from Simulated Interaction
推荐系统开发者值得看看这篇：浙大团队开源了 SimRec，用 GRPO 端到端训练会搜目录、会追问用户的推荐智能体。4B 的 Qwen3 打赢了更大规模的免训练方案，代码可复现，做电商搜索推荐的团队可以直接拿来参考。
浙大 ZJU-REAL 团队在 arXiv 发布 SimRec，一个用强化学习训练工具集成推荐智能体完整交互策略的框架，代码已在 GitHub 开源。现有方法多依赖手工设计的工作流，或只优化查询改写、列表排序等孤立环节，SimRec 则把搜索、追问、修正被拒推荐等行为作为整体策略来训练。
框架包含两个核心组件：任务锚定的用户模拟器基于当前任务需求（而非用户画像）对每次推荐给出反馈，可由外部 LLM 担任，也可在自博弈中由推荐器自身扮演；任务对齐的奖励用 GRPO 优化整条轨迹，将检索质量与有效工具调用纳入打分，并通过仅在训练阶段注入目标来缓解大目录下命中稀疏的问题。
实验在 ESCI 和 Amazon C4 两个数据集上进行：Qwen3-4B 智能体的 NDCG@100 从 32.23% 提升到 35.42%，从 22.94% 提升到 24.72%，超过更大参数量、未经训练的智能体和任务对齐基线。分析显示训练后的智能体更频繁地验证候选商品、利用用户偏好并从被拒推荐中恢复。
SimRec: Learning to Search, Ask, and Recommend from Simulated Interaction
LLM-based recommendation agents can search catalogs, inspect products, ask users, and revise rejected recommendations. However, these behaviors are often governed by hand-designed workflows, while existing training methods optimize isolated components such as query rewriting or list ranking rather than the interaction policy as a whole. We present SimRec, a reinforcement learning framework that trains the full interaction policy of a tool-integrated recommendation agent. SimRec introduces two components. A task-anchored user simulator provides feedback on every recommendation: conditioned on the current task need rather than on user profile, and it can be an external LLM or the recommender itself under self-play. A task-aligned reward optimizes the whole trajectory with GRPO, crediting natural retrieval quality and valid tool use alongside the final hit, with training-only target injection to handle sparse hits over large catalogs. On ESCI and Amazon C4, SimRec improves the NDCG@100 of a Qwen3-4B agent from 32.23% to 35.42% and from 22.94% to 24.72%}, outperforming training-free agents on far larger backbones and task-aligned baselines on NDCG. Analysis shows that the trained agent verifies candidates, uses preferences, and recovers from rejected recommendations far more often than its untrained counterpart. Our code is available at https://github.com/ZJU-REAL/SimRec .
This report is categorized under AITOP's "Papers" desk, focusing on China's AI industry and domestic foundation model ecosystem. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include SimRec, Qwen3, GRPO, 推荐系统.
Source attribution: curated from arXiv cs.AI (@Yaxin Zhang, Xingyu Wu, Yuchen Yan, Yuqin Dai, Hong Zhao, Lixing Shen, Zhan Chen, Jian Shao, Yongliang Shen).