MotionCrafter：基于 4D 变分自编码器的稠密几何与运动重建
MotionCrafter: Dense Geometry and Motion Reconstruction with a 4D VAE
把稠密 3D 点图与 3D 场景流放进同一坐标系联合表示，并训练专用 4D VAE 来承载扩散先验，可直接复用到单目视频的 4D 重建与稠密运动估计；其「不必强行对齐 RGB VAE 潜空间、改用数据归一化与训练策略」的结论，对任何想把视频生成先验迁移到几何/运动回归任务的工作都是可借鉴的设计经验。
原文依据 we show that such alignment is unnecessary and can hurt performance
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论本文提出 MotionCrafter，一个利用视频生成器从单目视频中联合重建 4D 几何并估计稠密运动的框架。其核心思想是在共享坐标系中对稠密 3D 点图与 3D 场景流进行联合表示，并配合一个专为该表示学习而设计的 4D 变分自编码器。与先前工作不同——那些工作尽管 3D 数值与潜变量同 RGB 变分自编码器潜变量的分布存在本质差异，仍严格地将二者对齐——本文表明这种对齐并无必要，且可能损害性能。为此，本文提出一种新的数据归一化与变分自编码器训练策略，能够更好地迁移扩散先验并大幅提升重建质量。在多个数据集上的大量实验表明，MotionCrafter 在几何重建与稠密场景流估计两方面均达到最优性能，几何与运动重建分别提升 38.64% 与 25.0%，且无需任何后优化。
英文原文摘要 999 字符
We present MotionCrafter, a framework that leverages video generators to jointly reconstruct 4D geometry and estimate dense motion from a monocular video. The key idea is a joint representation of dense 3D point maps and 3D scene flows in a shared coordinate system, together with a 4D VAE tailored to learn this representation effectively. Unlike prior work that strictly aligns 3D values and latents with RGB VAE latents--despite their fundamentally different distributions--we show that such alignment is unnecessary and can hurt performance. Instead, we propose a new data normalization and VAE training strategy that better transfers diffusion priors and greatly improves reconstruction quality. Extensive experiments on multiple datasets show that MotionCrafter achieves state-of-the-art performance in both geometry reconstruction and dense scene flow estimation, delivering 38.64% and 25.0% improvements in geometry and motion reconstruction, respectively, all without any post-optimization.