Motion 3-to-4:面向 4D 合成的三维运动重建
Motion 3-to-4: 3D Motion Reconstruction for 4D Synthesis
将 4D 合成分解为静态 3D 形状生成与运动重建,用紧凑运动潜变量加逐帧 Transformer 预测顶点轨迹,为从单目视频恢复时序一致几何提供了可复用的表示与架构思路。
原文依据 learns a compact motion latent representation and predicts per-frame vertex trajectories
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论本文提出 Motion 3-to-4,一个前馈框架,可从单段单目视频和可选的 3D 参考网格合成高质量的 4D 动态物体。尽管近期进展显著提升了 2D、视频与 3D 内容生成,4D 合成仍因训练数据有限以及从单目视角恢复几何与运动所固有的歧义性而困难重重。Motion 3-to-4 通过将 4D 合成分解为静态 3D 形状生成与运动重建来应对这些挑战。借助规范参考网格,本模型学习紧凑的运动潜变量表示,并预测逐帧顶点轨迹,以恢复完整且时序一致的几何。可扩展的逐帧 Transformer 进一步使其对变化的序列长度具备鲁棒性。在标准基准以及一个具有精确真值几何的新数据集上的评估表明,与先前工作相比,Motion 3-to-4 具有更优的保真度与空间一致性。项目主页见 https://motion3-to-4.github.io/。
英文原文摘要 1058 字符
We present Motion 3-to-4, a feed-forward framework for synthesising high-quality 4D dynamic objects from a single monocular video and an optional 3D reference mesh. While recent advances have significantly improved 2D, video, and 3D content generation, 4D synthesis remains difficult due to limited training data and the inherent ambiguity of recovering geometry and motion from a monocular viewpoint. Motion 3-to-4 addresses these challenges by decomposing 4D synthesis into static 3D shape generation and motion reconstruction. Using a canonical reference mesh, our model learns a compact motion latent representation and predicts per-frame vertex trajectories to recover complete, temporally coherent geometry. A scalable frame-wise transformer further enables robustness to varying sequence lengths. Evaluations on both standard benchmarks and a new dataset with accurate ground-truth geometry show that Motion 3-to-4 delivers superior fidelity and spatial consistency compared to prior work. Project page is available at https://motion3-to-4.github.io/.