Papers

DecoEdit 提出 3D 感知统一图像编辑框架,几何精度超越 Seedream 5.0

DecoEdit: Beyond Flat Control for 3D-Aware Unified Image Editing

Why we picked it

做图像编辑或 3D 内容生成的朋友可以看看这篇:SAM-3D 管几何、另一条路保纹理,解耦设计挺聪明。开源思路配 160 万样本数据集,对闭源的 Seedream 5.0 都能在姿态控制上占优,做研究或搭编辑工具的都值得跟进。

论文提出 DecoEdit,一个整合 1D 文本、2D 空间与 3D 体积输入的统一图像编辑框架,解决传统方法只能在文本提示、边界框或掩码等平面 2D 控件下操作、难以进行精确 3D 几何操控的问题。为训练该模型,团队构建了包含 160 万样本的 DecoDataset,覆盖多维度任务及其复杂组合,并发布 DecoBench 基准系统评估多任务编辑表现。

Body

技术核心是一条解耦管线:利用 SAM-3D 提供的结构引导锚定目标的几何、尺度和空间朝向,同时通过细节保留路径锁定高保真纹理,避免复杂空间变换与身份保持之间的取舍,也无需昂贵的逐实例微调。在 3D 空间姿态与朝向控制任务上,DecoEdit 在几何精度和身份一致性两方面显著超过 Seedream 5.0 等闭源模型。

Original · arXiv cs.AI

DecoEdit: Beyond Flat Control for 3D-Aware Unified Image Editing

While recent advances in multimodal image editing have achieved impressive results, current frameworks remain largely confined to ``flat" 2D controls (e.g., text prompts, bounding boxes, or masks). Precise 3D-aware geometric manipulation -- such as coordinating an object's location, physical scale, and 3D orientation while strictly preserving its fine-grained identity -- remains an elusive challenge, typically suffering from severe detail degradation or requiring expensive per-instance fine-tuning. In this paper, we present DecoEdit, an all-in-one image editing framework that goes beyond flat control by seamlessly integrating 1D textual, 2D spatial, and 3D volumetric inputs under a unified architecture. First, to power this unified model, we establish a data curation pipeline and curate a large-scale dataset, namely DecoDataset, containing 1.6 million samples, encompassing diverse multi-dimensional (1D/2D/3D) tasks and their complex compositions. Furthermore, we establish a comprehensive image editing benchmark named DecoBench to systematically evaluate multi-task editing performance. Finally, to address the fundamental trade-off between complex spatial transformations and identity preservation, we propose a novel decoupled pipeline: DecoEdit factorizes the editing process by utilizing structural guidance from SAM-3D to anchor the target geometry, scale, and spatial orientation, while concurrently leveraging a detail-preservation path to lock in high-fidelity textures. Extensive experiments demonstrate that DecoEdit achieves state-of-the-art results. Notably, on the challenging 3D spatial pose and orientation control tasks, DecoEdit significantly outperforms leading closed-source models such as Seedream 5.0 in both geometric accuracy and identity consistency.

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk, focusing on China's AI industry and domestic foundation model ecosystem. Within our cross-regional analytical framework, it directly maps to US-China foundation model benchmarks and commercialization comparison and AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include DecoEdit, SAM-3D, Seedream 5.0, image editing.

Source attribution: curated from arXiv cs.AI (@Xiaotang Gai, Jinbo Xing, Zijie Meng, Jiayi Yi, Jimin Xiao, Yixuan Qian, Yu Liu, Chaojie Mao, Zuozhu Liu).