快手发布 SEEK 框架：技能路由让 LLM 自动评估搜索质量
SEEK: Skill-Routed Evaluation with Evolvable Knowledge for Industrial Search
快手把搜索评估标准拆成技能库，按需路由、不重训就能更新规则，做搜索评估的可以看看这篇。
SEEK 是一个搜索质量自动评估框架，将评估标准外置为技能库，按查询-结果对动态路由相关技能，再用 listwise 评估器给出页面级判断和失败模式归因。两阶段训练使评估器对齐人类偏好，重放门控的技能库可在不重训模型的情况下补充评估知识。在快手短视频搜索的实验中，SEEK 提升了 listwise 质量评估准确率和归因诊断表现，已部署在日活超 4 亿的平台上。
SEEK: Skill-Routed Evaluation with Evolvable Knowledge for Industrial Search
Search quality evaluation provides essential supervision and diagnostic signals for the development and iteration of industrial search systems. Although large language models (LLMs) offer a scalable alternative to manual assessment, reliable automatic evaluation remains challenging: users experience search results at the page level, while the applicable evaluation criteria are multi-dimensional and continuously evolving. Packing all evaluation criteria into a unified prompt introduces irrelevant context and potential criterion interference, whereas internalizing them through post-training tightly couples rule updates with costly model retraining cycles. To address these issues, we propose Skill-routed Evaluation with Evolvable Knowledge (SEEK). Specifically, SEEK externalizes specific search evaluation criteria into a skill bank, dynamically routes relevant skills for each query-result list pair, and employs a task-adapted listwise evaluator to produce page-level judgments and failure mode attribution. A two-stage training pipeline teaches the evaluator to align evaluation criteria with human preferences, while a replay-gated skill bank allows recurring evaluation knowledge gaps to be incorporated without model retraining. Experiments on industrial short-video search show that SEEK improves listwise quality evaluation accuracy and achieves significant progress in attribution diagnosis. SEEK has been deployed at Kuaishou, a short-video platform with over 400 million daily active users, significantly improving the scale and quality of online search evaluation.
