Vista4D：基于 4D 点云的视频重拍摄
Vista4D: Video Reshooting with 4D Point Clouds
把输入视频与目标相机统一锚定在 4D 点云表示上，用静态像素分割 + 4D 重建显式保留已见内容并提供稠密相机信号，再用重建的多视角动态数据训练以抵抗真实点云伪影——这套「4D 点云作为中间表示 + 合成数据增强鲁棒性」的思路可直接迁移到新视角合成、动态场景编辑与视频重渲染任务。
原文依据 We build a 4D-grounded point cloud representation with static pixel segmentation and 4D reconstruction to explicitly preserve seen content
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论本文提出 Vista4D，一个鲁棒且灵活的视频重拍摄框架，将输入视频与目标相机锚定在 4D 点云中。具体而言，给定一段输入视频，本方法以相同的动态内容、从不同的相机轨迹与视角重新合成该场景。现有的视频重拍摄方法往往受困于真实动态视频的深度估计伪影，同时难以在具有挑战性的新轨迹下保持内容外观并维持精确的相机控制。我们构建了以 4D 为基准的点云表示，结合静态像素分割与 4D 重建，以显式保留已见内容并提供丰富的相机信号；同时使用重建的多视角动态数据进行训练，以在真实世界推理时对点云伪影具备鲁棒性。实验结果表明，在多种视频与相机路径下，本方法相比最优基线在 4D 一致性、相机控制与视觉质量上均有提升。此外，本方法可泛化至动态场景扩展与 4D 场景重组等真实世界应用。建议以视频形式在补充材料中查看结果。
英文原文摘要 1158 字符
We present **Vista4D**, a robust and flexible video reshooting framework that grounds the input video and target cameras in a 4D point cloud. Specifically, given an input video, our method re-synthesizes the scene with the same dynamics from a different camera trajectory and viewpoint. Existing video reshooting methods often struggle with depth estimation artifacts of real-world dynamic videos, while also failing to preserve content appearance and maintain precise camera control for challenging new trajectories. We build a 4D-grounded point cloud representation with static pixel segmentation and 4D reconstruction to explicitly preserve seen content and provide rich camera signals, and we train with reconstructed multiview dynamic data for robustness against point cloud artifacts during real-world inference. Our results demonstrate improved 4D consistency, camera control, and visual quality compared to state-of-the-art baselines under a variety of videos and camera paths. Moreover, our method generalizes to real-world applications such as dynamic scene expansion and 4D scene recomposition. Results are best viewed as videos in the Supplement.