Apple Paper Tests LLMs for Machine Translation Quality Annotation
MT quality evaluation is often treated as a solved problem once an LLM is in the loop, but this study shows agreement between models and humans is shaky on both sides. Teams building localisation pipelines or multilingual products should benchmark their chosen LLM per language pair and scheme before trusting automated MQM or ESA scores. The comparison with established evaluation frameworks like MQM gives practitioners a concrete reference point.