FLUX3D:扩散对齐稀疏表示实现高保真3D高斯生成

FLUX3D: High-Fidelity 3D Gaussian Generation with Diffusion-Aligned Sparse Representation

精选理由

想从单张图生成高质量3D模型?FLUX3D用扩散对齐稀疏表示解决了细节丢失问题,效果比现有方法好一截,值得搞3D生成的看看。

AI 摘要

FLUX3D 提出了一种从单张图像生成高保真 3D 高斯泼溅(3DGS)的新方法。它通过设计 Diffusion-Aligned Structured Latents (DA-SLAT) 改进稀疏体素表示,并用 Sparse-structure Multimodal Diffusion Transformer (SMDiT) 和 Modal-Aware Rotary Positional Embedding (MARoPE) 解决二维图像令牌与三维体素潜在的对齐问题。在多个基准测试中,FLUX3D 在外观保真度上显著优于现有最先进方法,生成的 3DGS 资产质量更高。

原文 · arXiv cs.AI

FLUX3D: High-Fidelity 3D Gaussian Generation with Diffusion-Aligned Sparse Representation

Sparse voxel representation has emerged as a scalable foundation for image-to-3D Gaussian Splatting (3DGS) generation, yet current methods struggle to preserve high-frequency visual details of input images due to two structural bottlenecks. First, they adopt discriminative 2D features optimized for semantic abstraction to construct sparse voxel latents, which suppress reconstructive cues and induce a representation bottleneck. Second, in the generation stage, standard diffusion transformers lack effective mechanisms to align dense 2D image tokens with sparse 3D voxel latents, resulting in a cross-modal correspondence bottleneck. To address these issues, we propose FLUX3D, a scalable image-to-3DGS framework that boosts both representation learning and cross-modal alignment during generation. We first revisit 2D feature selection for sparse-voxel-based 3D representation learning, propose Diffusion-Aligned Structured Latents (DA-SLAT) and couple it with a decoder-only architecture to improve 3DGS reconstruction fidelity. We also design a sparse-structure-aware diffusion framework, which integrates the Sparse-structure Multimodal Diffusion Transformer (SMDiT) and Modal-Aware Rotary Positional Embedding (MARoPE) to achieve geometry-agnostic 2D-3D alignment. Extensive benchmark experiments demonstrate that FLUX3D yields substantial improvements in appearance fidelity and significantly outperforms all state-of-the-art (SOTA) methods in generating high-quality 3DGS assets.