想搞3D生成质量自动评估？这篇论文告诉你：光选模型不够，整个评估管道都得调。六视图RGB就够用，省成本。
3D-DefectBench是一个用于系统分析基于视觉语言模型（VLM）的3D缺陷检测管道的基准和框架。它涵盖9种细粒度二分类缺陷（几何、纹理、提示遵循），通过平衡因子设计，在84种推理设计和约320万评分缺陷决策上变化4个管道因子（VLM、相机协议、视觉输入、提示方案）。模型选择是与人标注一致性最大的决定因素，但其他因子也影响性能并与模型选择交互。紧凑的六视图RGB协议与更密集的多视图设置性能相当。最佳VLM法官仍落后于经过训练的人类标注者。
3D-DefectBench: A Controlled Factorial Study of Vision-Language Model Evaluation Pipelines for Fine-Grained 3D Generation Defects
Automated evaluation is essential for scaling generative 3D systems, where exhaustive human review is costly and slow. However, the reliability of an automated judge depends on the entire evaluation pipeline, not only the underlying vision-language model (VLM), but also how assets are rendered, what visual evidence is provided, how the task is specified, and how human reference labels are constructed. We introduce 3D-DefectBench, a benchmark and framework for systematic analysis of VLM-based 3D defect detection pipelines. It complements holistic ratings and pairwise preferences with nine fine-grained binary defects spanning geometry, texture, and prompt adherence, providing actionable diagnostics for generator development and judge evaluation. Using a balanced factorial design, we vary four pipeline factors, VLM, camera protocol, visual input, and prompt schema, across 84 inference designs and approximately 3.2 million scored defect decisions, followed by staged validation on a broader set of frontier models. Model choice is the largest determinant of agreement with human labels, but the remaining factors also affect performance, interact with model selection, and can change the best configuration. Within the evaluated design space, a compact six-view RGB protocol performs comparably to denser multi-view settings and inputs augmented with depth or surface normals, making it a strong cost-effective default. Under this standardized pipeline, the best of 12 VLM judges still lag behind trained human labelers, while texture agreement drops sharply when expert-consensus labels are replaced by noisier silver labels. These findings show that automated judges should be evaluated as complete pipelines and calibrated across human reference regimes, rather than benchmarked only as standalone models. We release labels, prompts, predictions, and Croissant metadata on Hugging Face.