不用多相机，单视频就能重建多个物体怎么动，还能人工纠正错误。HAT-4D这个框架开源了，做Embodied AI的数据生成很方便。
HAT-4D是首个用于从单目视频中重建多个物体3D几何、时间动态和物理交互的智能体框架。它结合视觉语言模型（VLM）和多级人工反馈机制，解决深度模糊和遮挡问题，无需多相机设备。利用HAT-4D构建了MVOIK-4D基准，包含新的多维评估协议，侧重物理合理性和时间一致性。在大多数评估指标上达到SOTA，同时保持语义对齐。少量人工反馈即可显著改善交互重建质量。
HAT-4D: Lifting Monocular Video for 4D Multi-Object Interactions via Human-Agent Collaboration
Extracting dynamic 4D object interactions from massive, in-the-wild monocular videos offers a highly efficient data collection pathway for scaling Embodied AI and training VLAs. However, existing monocular 4D reconstruction methods primarily focus on isolated objects, often failing under the severe occlusions and complex dynamics inherent in multi-object interactions. To bridge this gap, we propose HAT-4D, the first agentic framework designed to reconstruct the 3D geometry, temporal dynamics, and physical interactions of multiple objects from a single video. By integrating VLMs with a multi-level human-in-the-loop feedback mechanism, HAT-4D efficiently resolves depth ambiguities and interaction-induced occlusions during 3D generation and 4D propagation, yielding physically plausible assets without relying on expensive multicamera rigs. As a scalable data engine, HAT-4D facilitates the creation of MVOIK-4D, an open-world benchmark for monocular 4D interaction reconstruction, accompanied by a novel multi-dimensional evaluation protocol focused on physical plausibility and temporal consistency. Extensive experiments demonstrate that HAT-4D achieves SOTA performance on most evaluation metrics, while maintaining competitive semantic alignment. Ablation studies show that introducing a small amount of human feedback improves interaction reconstruction. Moreover, the data produced by HAT-4D effectively improves baseline performance when used for fine-tuning. Our data and code are available at https://lijiaxin0111.github.io/HAT4D/