做多模态模型训练的团队终于有了一个能同时提升理解和生成的后训练方法——SGT用分割任务对齐表征空间，比解耦训练更高效，做视觉AI的开发者可以直接参考代码实现。
统一多模态模型（UMMs）试图在单一架构中整合视觉理解与视觉生成，但现有训练范式将两者解耦，导致表征空间错位。本文首次系统研究生成式后训练，发现高层语义任务（如图像分割）可作为最优代理，通过提供结构语义来增强视觉感知和生成布局保真度。作者提出语义生成调优（SGT）范式，利用分割作为生成代理对齐多模态能力。机制分析表明SGT改善了特征线性可分性和视觉-文本注意力分配。实验证明SGT在主流基准上持续提升多模态理解和生成保真度。
Semantic Generative Tuning for Unified Multimodal Models
Unified multimodal models (UMMs) strive to consolidate visual understanding and visual generation within a single architecture. However, prevailing training paradigms independently optimize understanding via sparse text signals and generation through dense pixel objectives. Such a decoupled strategy yields misaligned representation spaces, isolating visual understanding from generation and hindering their mutual reinforcement. This work presents the first systematic investigation into generative post-training, where we formulate hierarchical visual tasks as generative proxies to bridge the isolation in UMMs. Our empirical investigation reveals that high-level semantic tasks, particularly image segmentation, serve as optimal proxies. Unlike low-level tasks that distract models with texture details, segmentation provides structural semantics that significantly enhance both vision-centric perception and generative layout fidelity. Building upon these insights, we introduce Semantic Generative Tuning (SGT), a novel paradigm that leverages segmentation as a generative proxy to align and synergize multimodal capabilities. Mechanistic analyses further demonstrate that SGT fundamentally improves feature linear separability and optimizes visual-textual attention allocation pattern. Extensive evaluations show that SGT consistently improves both multimodal comprehension and generative fidelity across mainstream benchmarks. Our code is available on the https://song2yu.github.io/SGT/.