RubricsTree用4000条真实查询构建100多条可验证规则，评估健康AI比LLM裁判更准，还能当训练奖励，让Gemini等模型性能飙升66%。
RubricsTree是一个专家对齐的分层评估框架，包含超过100个可临床验证的原子布尔规则，这些规则从4000个真实用户查询中通过迭代人机协作提炼而成。框架使用上下文自适应路由器为每个查询激活相关子集，实现可扩展且与专家质量对齐的评估。在元评估中，RubricsTree在专家对齐上显著超过强基线，且可靠惩罚上下文退化的响应。作为结构化指令、文本反馈或训练奖励用于性能优化时，RubricsTree在HealthBench上为Gemini、GPT和Qwen系列模型带来高达约66%的相对提升。
RubricsTree: Scalable and Evolving Open-Ended Evaluation of Personal Health Agents across Health Memory and Medical Skills
The LLM-empowered personal health agents with user health (sensor) metrics have offered a promising pathway to alleviate global disparities in healthcare access. However, large-scale clinical deployment remains constrained by an open-ended evaluation bottleneck: physician annotation is reliable but costly and unscalable, while LLM-as-a-judge evaluators are scalable but subjective, inconsistent, and sometimes clinically misaligned. We introduce RubricsTree, a scalable evaluation framework with an expert-aligned hierarchical taxonomy of over 100 atomic, clinically-verifiable Boolean rubrics, evolving from the insights of 4,000 real user queries through an iterative human-in-the-loop curation protocol with an expertise panel led by an experienced physician. A context-aware adaptive router activates only the relevant auto-weighted rubric subset per query, providing the throughput needed for scalable evaluation with expert-aligned quality. Through a systematic meta-evaluation, we show that RubricsTree (i) substantially exceeds a strong large-scale evaluation baseline in expert alignment on challenging open-ended queries; (ii) reliably penalizes contextually degraded responses; and (iii) when used as structured instructions, text feedback, or training rewards for performance optimization, yields up to ~66% relative gains on HealthBench for Gemini, GPT, and Qwen model families. RubricsTree thus provides a scalable, auditable, and evolving evaluation infrastructure required for the continuous optimization of product-level personal healthcare AI.