DecoEdit 提出 3D 感知统一图像编辑框架,几何精度超越 Seedream 5.0
DecoEdit: Beyond Flat Control for 3D-Aware Unified Image Editing
做图像编辑或 3D 内容生成的朋友可以看看这篇:SAM-3D 管几何、另一条路保纹理,解耦设计挺聪明。开源思路配 160 万样本数据集,对闭源的 Seedream 5.0 都能在姿态控制上占优,做研究或搭编辑工具的都值得跟进。
论文提出 DecoEdit,一个整合 1D 文本、2D 空间与 3D 体积输入的统一图像编辑框架,解决传统方法只能在文本提示、边界框或掩码等平面 2D 控件下操作、难以进行精确 3D 几何操控的问题。为训练该模型,团队构建了包含 160 万样本的 DecoDataset,覆盖多维度任务及其复杂组合,并发布 DecoBench 基准系统评估多任务编辑表现。
技术核心是一条解耦管线:利用 SAM-3D 提供的结构引导锚定目标的几何、尺度和空间朝向,同时通过细节保留路径锁定高保真纹理,避免复杂空间变换与身份保持之间的取舍,也无需昂贵的逐实例微调。在 3D 空间姿态与朝向控制任务上,DecoEdit 在几何精度和身份一致性两方面显著超过 Seedream 5.0 等闭源模型。
DecoEdit: Beyond Flat Control for 3D-Aware Unified Image Editing
While recent advances in multimodal image editing have achieved impressive results, current frameworks remain largely confined to ``flat" 2D controls (e.g., text prompts, bounding boxes, or masks). Precise 3D-aware geometric manipulation -- such as coordinating an object's location, physical scale, and 3D orientation while strictly preserving its fine-grained identity -- remains an elusive challenge, typically suffering from severe detail degradation or requiring expensive per-instance fine-tuning. In this paper, we present DecoEdit, an all-in-one image editing framework that goes beyond flat control by seamlessly integrating 1D textual, 2D spatial, and 3D volumetric inputs under a unified architecture. First, to power this unified model, we establish a data curation pipeline and curate a large-scale dataset, namely DecoDataset, containing 1.6 million samples, encompassing diverse multi-dimensional (1D/2D/3D) tasks and their complex compositions. Furthermore, we establish a comprehensive image editing benchmark named DecoBench to systematically evaluate multi-task editing performance. Finally, to address the fundamental trade-off between complex spatial transformations and identity preservation, we propose a novel decoupled pipeline: DecoEdit factorizes the editing process by utilizing structural guidance from SAM-3D to anchor the target geometry, scale, and spatial orientation, while concurrently leveraging a detail-preservation path to lock in high-fidelity textures. Extensive experiments demonstrate that DecoEdit achieves state-of-the-art results. Notably, on the challenging 3D spatial pose and orientation control tasks, DecoEdit significantly outperforms leading closed-source models such as Seedream 5.0 in both geometric accuracy and identity consistency.
This report is categorized under AITOP's "Papers" desk, focusing on China's AI industry and domestic foundation model ecosystem. Within our cross-regional analytical framework, it directly maps to US-China foundation model benchmarks and commercialization comparison and AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include DecoEdit, SAM-3D, Seedream 5.0, image editing.
Source attribution: curated from arXiv cs.AI (@Xiaotang Gai, Jinbo Xing, Zijie Meng, Jiayi Yi, Jimin Xiao, Yixuan Qian, Yu Liu, Chaojie Mao, Zuozhu Liu).