Study Finds MLLM Judges Show Cultural Bias Under Ambiguity
Scoring bias in judge models flows directly into training signal and leaderboard rankings, so skewed calibration quietly shapes which outputs get rewarded. Teams using MLLM-as-a-judge pipelines should audit scale usage and split alignment reports by reference culture rather than averaging. Anyone building evaluation or RLHF infrastructure for products serving both Western and Chinese audiences has a concrete methodology to copy here.
