Themis和Prometheus评分机制被解析，微调如何改变LLM评估流程有新发现
Themis(Llama-3-8B)和Prometheus(Mistral-7B)评分模型采用两阶段评估流程。低于15层注意力执行局部错误比较，高于15层MLP级联整合信号并写入评分。在Themis中决策形成于第26层，Prometheus在第25层。基础模型(Llama-3-8B)重现路由架构和决策形成，但不具备阶段分离，表明微调塑造了现有结构而非从头构建。
Beyond Scores: Understanding LLM-as-a-Judge Mechanisms in Summarization Evaluation
LLM-based evaluators of natural language generation (NLG) quality are widely deployed as scoring tools and as automated training signals, yet the internal procedure by which they assign a rating remains poorly understood. We investigate this procedure mechanistically through an eight-attack perturbation taxonomy across the Readability and Adequacy dimensions of NLG quality, a generation pipeline that produces paired clean and corrupt summaries with controlled error intensity and explicit token-level modification maps, and a four-experiment battery of causal tracing, logit-lens vocabulary projection, and attention-head knockout applied to Themis (Llama-3-8B) and Prometheus (Mistral-7B). Both evaluators implement a structured, coherent evaluation pipeline operating in two stages: below layer 15, attention performs local error comparison and routes the result to the final input position; above it, the MLP cascade integrates the signal and writes the rating, with the decision crystallizing in the residual stream at a sharp late layer (L = 26 on Themis, L = 25 on Prometheus). Furthermore, a base-model control at the same scale (Llama-3-8B) reproduces the routing architecture and crystallization but not the stage separation, isolating the two mechanisms that fine-tuning specifically installs, suppression of below-L15 MLP contribution at the last position and a two-layer advance of the crystallization depth, indicating that fine-tuning sculpts an existing substrate rather than building the pipeline from scratch. We release the source code and data at https://github.com/himil-v/judge-mech