这篇论文提出了EYT-Bench，能更真实地测出LLM在多轮对话中的意图追踪能力。用16/17模型发现推理模式大幅提升长上下文准确率，值得关注。
EYT-Bench是一个评估LLM多轮对话能力的基准，采用三部分解耦设计：基于人工语料库（Nemotron-Personas-USA和PersonaMem-v2）的特定角色用户模拟器、分离意图感知与回复生成的目标模型、以及独立第三方LLM评判器。在17个目标模型×200段对话的评估中，闭源与开源模型在主观维度（共情/人格化等）差异不超过0.3，但客观意图追踪差距可达9倍。开启推理模式后，Gemma-4的长上下文潜在意图准确率提升0.47-0.50。交叉评判消融实验通过DeepSeek-v4-pro验证了目标排名和最终意图满意度的一致性。
Enjoy Your Talk: A Human-Centered Benchmark for Multi-Turn Dialogue with Decoupled User Simulation, Target Modeling, and Judging
Evaluating large language models (LLMs) as multi-turn conversational partners requires probing capabilities that single-turn benchmarks miss: persona consistency, evolving intent tracking, emotional dynamics, and goal completion. We introduce EYT-Bench, a human-centered benchmark built around a three-party decoupled design: a persona-grounded user simulator, a target model that separates intent perception from response generation, and an independent third-party LLM judge with optional multi-judge ensembling. Personas are sampled from public human-curated corpora, Nemotron-Personas-USA and PersonaMem-v2, rather than synthesized, reducing LLM-induced persona bias. EYT-Bench also introduces two trajectory-level metrics: embedding-based intent drift and final-intent completion rate (FICR), inspired by tau-bench. In a 17-target x 200-dialogue evaluation, EYT-Bench reveals four findings: (i) state-of-the-art closed- and open-source models are statistically close on subjective dimensions (empathy / persona / anthropomorphism vary within <= 0.3), but differ by up to 9x on objective intent tracking; (ii) reasoning ("thinking on") sharply improves objective tracking on long-context personas (+0.47-0.50 latent-intent accuracy on Gemma-4) while leaving subjective scores nearly unchanged; (iii) persona format dominates trajectory spread, with FICR saturating above 0.95 on Nemotron-USA but spreading from 0.53 to 0.88 on PersonaMem-v2; and (iv) the warm-up effect is robust on 16/17 models (one outlier, GPT-5.5, reverses the effect), with stable rankings across alpha in [0.05, 0.15]. A cross-judge ablation using deepseek-v4-pro confirms that target rankings and final-intent satisfaction are preserved across judges.