Mesh4D：从单目视频重建与跟踪四维网格
Mesh4D: 4D Mesh Reconstruction and Tracking from Monocular Video
用自编码器把整段动画压进一个紧凑潜空间、再用潜扩散模型一次性预测完整形变场，这种「序列级潜表示 + 一次性生成」的范式可迁移到动态人体/手部/物体的单目四维重建与新视角合成；训练时用骨骼先验约束形变、推理时不需要骨骼，也提供了一个把结构化先验蒸馏进潜空间的可复用思路。
原文依据 we train a latent diffusion model that, conditioned on the input video and the mesh reconstructed from the first frame, predicts the full animation in one shot
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论我们提出 Mesh4D，一个用于单目四维网格重建的前馈模型。给定一段动态物体的单目视频，该模型重建出物体的完整三维形状与运动，并以形变场的形式表示。我们的核心贡献是一个紧凑的潜空间，它编码了整段动画序列。该潜空间由一个自编码器学习得到，在训练过程中该自编码器以训练物体的骨骼结构作为引导，从而对合理的形变提供强先验。关键在于，推理阶段并不需要骨骼信息。编码器采用时空注意力，从而得到对物体整体形变更稳定的表示。基于这一表示，我们训练了一个潜扩散模型，它以输入视频和由首帧重建出的网格为条件，一次性预测完整的动画。我们在重建与新视角合成基准上评估 Mesh4D，在恢复准确的三维形状与形变方面优于已有方法。
英文原文摘要 1031 字符
We propose Mesh4D, a feed-forward model for monocular 4D mesh reconstruction. Given a monocular video of a dynamic object, our model reconstructs the object's complete 3D shape and motion, represented as a deformation field. Our key contribution is a compact latent space that encodes the entire animation sequence. This latent space is learned by an autoencoder that, during training, is guided by the skeletal structure of the training objects, providing strong priors on plausible deformations. Crucially, skeletal information is not required at inference time. The encoder employs spatio-temporal attention, yielding a more stable representation of the object's overall deformation. Building on this representation, we train a latent diffusion model that, conditioned on the input video and the mesh reconstructed from the first frame, predicts the full animation in one shot. We evaluate Mesh4D on reconstruction and novel view synthesis benchmarks, outperforming prior methods in recovering accurate 3D shape and deformation.