论文搞了个8B的小模型FATE,专门给AI老师打分,比蒸馏前涨了22个点。还顺便测了Gemini、ChatGPT等,Gemini 2.5 Flash得分最高,挺有意思。
该论文提出FATE(FLC AI Tutor Evaluator),一个8B参数的语言模型,专门用于评估AI导师的教学质量。FATE基于BEA 2025共享任务的四个核心评估轨道(Mistake Identification、Mistake Location、Guidance、Actionability)进行对齐。通过从前沿LLM进行知识蒸馏生成额外监督数据,FATE在评估性能上最高提升22.63个百分点。论文还使用FATE对ChatGPT、Claude、Gemini、DeepSeek等主流商业模型的教学响应进行基准测试:Gemini 2.5 Flash表现最好(82.88%),其次是ChatGPT 5.5 Instant(80.75%)、DeepSeek V4 Flash(80.13%)和Claude Sonnet 4.6(74.00%)。
Knowledge Distillation for Automated AI Tutor Evaluation
The rapid integration of Large Language Models (LLMs) into K-12 and higher education has outpaced the development of reliable methods for evaluating their pedagogical quality. As the research community starts to explore the space of automating evaluation of AI tutors, we introduce FATE (FLC AI Tutor Evaluator), a specialized 8B-parameter language model designed to evaluate AI tutors. Aligned with the four core evaluation tracks from the BEA 2025 Shared Task, our model assesses pedagogical ability across Mistake Identification, Mistake Location, Guidance, and Actionability. Because pedagogical evaluation is a specialized task with limited labeled data, we leverage knowledge distillation from a frontier LLM to generate additional supervision, yielding absolute performance gains up to 22.63 percentage points. Finally, we demonstrate FATE's utility as an automated evaluator by benchmarking instructional responses generated by popular commercial models, including ChatGPT, Claude, Gemini, and DeepSeek. On average, we have found that Gemini 2.5 Flash perfomed best (82.88%), then ChatGPT 5.5 Instant (80.75%), followed by DeepSeek V4 Flash (80.13%) and Claude Sonnet 4.6 (74.00%).