做3D动画和内容生成的团队终于有了解决姿态错位痛点的方案——R-DMesh 能自动矫正网格姿态对齐视频,避免手动调整的繁琐,建议做动态资产制作的开发者点开看看。
R-DMesh 提出了一种解决视频引导3D动画中姿态错位问题的统一框架。传统方法在用户提供的静态网格与参考视频起始帧姿态不匹配时,会导致几何扭曲或动画失败。R-DMesh 通过引入新型 VAE 将输入解耦为条件基础网格、相对运动轨迹和矫正跳跃偏移,并利用 Triflow Attention 机制确保物理一致性和局部刚性。该方法基于 Rectified Flow 的扩散 Transformer 从预训练视频中迁移时空先验,并构建了包含50万动态网格序列的 Video-RDMesh 数据集。实验表明,R-DMesh 有效解决了姿态对齐问题,并支持姿态重定向和整体4D生成等下游应用。
R-DMesh: Video-Guided 3D Animation via Rectified Dynamic Mesh Flow
Video-guided 3D animation holds immense potential for content creation, offering intuitive and precise control over dynamic assets. However, practical deployment faces a critical yet frequently overlooked hurdle: the pose misalignment dilemma. In real-world scenarios, the initial pose of a user-provided static mesh rarely aligns with the starting frame of a reference video. Naively forcing a mesh to follow a mismatched trajectory inevitably leads to severe geometric distortion or animation failure. To address this, we present Rectified Dynamic Mesh (R-DMesh), a unified framework designed to generate high-fidelity 4D meshes that are ``rectified'' to align with video context. Unlike standard motion transfer approaches, our method introduces a novel VAE that explicitly disentangles the input into a conditional base mesh, relative motion trajectories, and a crucial rectification jump offset. This offset is learned to automatically transform the arbitrary pose of the input mesh to match the video's initial state before animation begins. We process these components via a Triflow Attention mechanism, which leverages vertex-wise geometric features to modulate the three orthogonal flows, ensuring physical consistency and local rigidity during the rectification and animation process. For generation, we employ a Rectified Flow-based Diffusion Transformer conditioned on pre-trained video latents, effectively transferring rich spatio-temporal priors to the 3D domain. To support this task, we construct Video-RDMesh, a large-scale dataset of over 500k dynamic mesh sequences specifically curated to simulate pose misalignment. Extensive experiments demonstrate that R-DMesh not only solves the alignment problem but also enables robust downstream applications, including pose retargeting and holistic 4D generation.