LUNA:无需线性混合蒙皮的通用3D人体动画模型

LUNA: Learning Universal 3D Human Animation Beyond Skinning

精选理由

LUNA 不用传统的 LBS 蒙皮,直接用图片或关键点驱动3D角色,效果真实还能自动适配新角色,动画师和研究者可以试试。

AI 摘要

LUNA 提出了一种无 LBS 的通用神经动画模型,直接从2D输入(图像、关键点、草图)预测3D高斯变形,无需显式人体拟合。其核心是一个基于 transformer 的运动回归器,分离全局刚体运动与局部非刚性细节。通过混合监督策略,模型从 LBS 教师蒸馏结构先验,并利用大量无标签视频数据进行训练。实验表明,LUNA 在视觉保真度上媲美 LBS 方法,同时实现零样本跨身份泛化,支持多种驱动模态。该工作是首个端到端支持隐式2D驱动的3D可动画化模型。

原文 · arXiv cs.AI

LUNA: Learning Universal 3D Human Animation Beyond Skinning

Creating photorealistic, animatable 3D human avatars from monocular images still largely depends on Linear Blend Skinning (LBS) and parametric body models, which constrain expressivity and often introduce artifacts due to imperfect fitting. We propose LUNA, an LBS-free universal neural animation model that directly maps multiple 2D controls like images, keypoints, sketches, and unseen characters into 3D Gaussian deformations, bypassing explicit body fitting. At its core, a transformer-based motion regressor disentangles global rigid motion from fine-grained local dynamics to capture both coherent movement and subtle non-rigid effects. To resolve the inherent ambiguity of 2D-to-3D lifting while scaling beyond fitted datasets, we introduce hybrid supervision that distills soft structural priors from an LBS teacher and a loss that supports training on both limited fitted data and large in-the-wild unlabeled videos. Extensive experiments show LUNA achieves competitive visual fidelity compared to LBS-based approaches, while delivering realistic human motion and zero-shot cross-identity generalization across diverse driving modalities. To the best of our knowledge, LUNA is the first end-to-end 3D animatable model that supports implicit 2D driving.