MCR-GRPO:结构化视觉感知的逐框边际贡献分配

Credit the Right Box: Marginal Contribution Assignment for Structured Visual Perception

精选理由

GRPO以前只看整体得分,现在MCR-GRPO能逐个框算功劳,定位和分割都更准了。

AI 摘要

现有GRPO等组相对强化学习方法只提供响应级监督,与结构化多目标预测存在粒度失配。MCR-GRPO通过留一比较估计每个预测框的边际贡献,并利用连续匹配集值评估器改进归一化与定位。在REC、DOD、分割和计数基准上,该方法超越了现有GRPO基线。

原文 · arXiv cs.AI

Credit the Right Box: Marginal Contribution Assignment for Structured Visual Perception

Multimodal Large Language Models (MLLMs) are increasingly expected to solve structured perception tasks that require visual recognition, language-to-object binding, object cardinality preservation, and precisely localized grounding and segmentation outputs. However, existing group-relative reinforcement learning methods provide only response-level supervision, creating a granularity mismatch for structured multi-object prediction: a single advantage is broadcast to all tokens in a response, without distinguishing individual box contributions. To address this mismatch, we propose MCR-GRPO, a marginal contribution assignment framework that derives box-level credit directly from each sampled response. Specifically, Marginal Contribution Reward (MCR) estimates each predicted box's contribution through a leave-one-out comparison, measuring how the matched set value changes when the box is removed from the response. After within-response normalization, records that improve the set value receive positive credit, while redundant or harmful ones are suppressed. To make marginal attribution stable and informative, we further introduce a Continuous Matched Set Value Evaluator that integrates permutation-invariant matching, count-aware normalization, and graded localization. MCR-GRPO maps normalized box-level marginal advantages to the token spans that generated each box, preserving GRPO's response-level comparison while enabling box-aware optimization of structured multi-object grounding. Experiments across REC, DOD, segmentation, and counting benchmarks show state-of-the-art performance over prior GRPO-based baselines.