RaRe：用 MLLM 思维链推理改进开放词汇场景图生成
Reasoning-Aware Relational Representation Learning for Open-Vocabulary Scene Graph Generation
用 MLLM 的思维链直接当训练信号来学关系表示，这思路跟只拿 CoT 做中间推理的方法不一样。做视觉关系检测、场景图或开源多模态研究的人可以看看，VG150 和 PSG 上有具体对比数字。
一项发表于 arXiv 的论文提出 RaRe 框架，针对开放词汇场景图生成（OV-SGG）任务中稀有关系识别难、泛化到未见谓词能力弱的问题，将 MLLM 生成的思维链（CoT）推理文本转化为结构化的关系嵌入，而不只是把 CoT 当作最终预测的中间痕迹。
技术上 RaRe 分两步：先通过自监督对比学习把推理嵌入与文本化的关系描述对齐，再用面向具体性的强化学习目标优化推理生成，促使模型产出与目标物体对绑定的语义证据，提升关系间可区分性。在 VG150 和 PSG 两个基准上，RaRe 在类别均衡识别指标上超过现有基线，但作者也明确指出增益随数据集与候选框设置不同而有所波动。
Reasoning-Aware Relational Representation Learning for Open-Vocabulary Scene Graph Generation
Open-Vocabulary Scene Graph Generation (OV-SGG) requires models to recognize visual relationships beyond the training vocabulary, yet existing methods often rely on dataset-specific object--predicate co-occurrence patterns. Under long-tailed distributions and noisy object localization, such reliance leads to biased, entangled relation representations, limiting both rare-relation recognition and generalization to unseen predicates. To address these challenges, we propose RaRe, a reasoning-aware relational representation learning framework that transforms MLLM-generated chain-of-thought (CoT) rationales into structured relation embeddings. Rather than using CoT only as an intermediate reasoning trace for final prediction, RaRe directly optimizes the rationale-derived representation space to improve inter-relation discriminability. Specifically, RaRe first aligns rationale embeddings with textualized relation descriptions through self-supervised contrastive learning, and then refines rationale generation with a specificity-oriented reinforcement learning objective that encourages semantically object-pair-specific relational evidence. Experiments on VG150 and PSG demonstrate improved category-balanced recognition over existing baselines, while the additional gains from learning vary across datasets and proposal settings. Code available at: https://artanic30.github.io/project_pages/RaRe.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include RaRe, MLLM, 场景图生成, 对比学习.
Source attribution: curated from arXiv cs.AI (@Jiawei Xiao, Boya Wang, Longtian Qiu, Shan Ning, Xuming He).