论文精选

Pose-ICL:3D感知上下文学习实现可控姿态主体定制

Pose-ICL: 3D-Aware In-Context Learning for Pose-Controllable Subject Customization

精选理由

做图像生成和主体定制的团队终于有了一个能精准控制姿态的方案——Pose-ICL 用3D感知解决了2D模型的老大难问题,做定制化生成的开发者可以直接试试。

AI 摘要

Pose-ICL 是一种无需微调的框架,通过3D感知的上下文学习(ICL)实现姿态可控的主体定制。其核心机制 Surface-Anchored Position Embedding (SAPE) 将图像标记锚定到体积边界框的表面坐标,赋予模型显式的3D感知能力。该方法解决了现有2D模型在主体定制中姿态不准确和跨姿态外观不一致的问题。实验表明,Pose-ICL 在3D资产和真实主体上均显著优于现有方法,在姿态准确性和身份一致性上表现突出。该框架与现有DiT模型兼容,可直接应用。

原文 · arXiv cs.AI

Pose-ICL: 3D-Aware In-Context Learning for Pose-Controllable Subject Customization

Subject Customization is a foundational task in modern image generation. By providing a few reference images and a text prompt, users can generate images of a specific object in any desired scene. However, existing methods still struggle to achieve effective pose control for customized subjects. In practice, they often exhibit inaccurate poses or inconsistent cross-pose appearances. These limitations suggest that understanding objects in a volumetric manner remains a significant challenge for 2D-native backbones. To address this challenge, we propose Pose-ICL, a tuning-free framework that leverages 3D-aware In-Context Learning (ICL) to directly adapt to new subjects through multiple paired image-pose references. Its core mechanism,Surface-Anchored Position Embedding (SAPE), equips the model with explicit 3D awareness by anchoring image tokens to the surface coordinates of a volumetric bounding box. Dedicated refinements ensure its seamless compatibility with existing DiT models. Extensive evaluations on both 3D assets and real-world subjects demonstrate that Pose-ICL significantly outperforms current methods in both pose accuracy and identity consistency.