Point4Cast：流式动态场景重建与预测
Point4Cast: Streaming Dynamic Scene Reconstruction and Forecasting
把「持续演化的潜时空表征 + 更新/读出」这一流式范式用于任意时刻查询，可让在线三维重建从「逐帧点图」升级为可外推的未来几何预测，并顺带免费得到场景流；对做动态 SLAM、自动驾驶 BEV/点图预测、视频到三维的工作都可直接复用其更新-读出接口。
原文依据 a persistently evolving latent spacetime representation that models the environment's evolution across time
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论理解三维世界如何随时间演化是计算机视觉中的一项基础任务，对具身场景、自动驾驶等至关重要。它不仅要求重建已观测到的场景，还要求预判场景动态在未来将如何展开。尽管随着近期前馈神经网络的出现，三维重建领域进展迅速，但在给定视频二维帧的条件下预测三维未来动态仍属空白。我们提出 Point4Cast，一个统一框架，处理视频的流式二维帧序列，以三维方式估计底层动态场景的过去、现在与未来。我们方法的核心是一个持续演化的潜时空表征，用于建模环境随时间的演化。在接收到新的二维帧时，更新操作会整合新到的证据以细化该潜时空表征。当对任意时刻进行查询时——无论该时刻早于、等于还是晚于最近一次更新的时间戳——读出过程会预测以时间为条件的点图与相机参数，用以描述所查询时刻的场景几何。与以往在线的动态场景重建方法仅在最近观测帧的时间戳处估计每一帧的点图不同，Point4Cast 在任意查询时刻都能实现连贯的重建。实证评估表明，Point4Cast 在流式动态场景重建与预测基准上、在多个具有挑战性的数据集上均取得最优性能，同时免费提供场景流估计与预测。代码将公开发布。
英文原文摘要 1689 字符
Understanding how the 3D world evolves over time is a fundamental task in computer vision, essential for embodied settings, autonomous driving, etc. It requires not only the reconstruction of the observed scene but also the anticipation of how the scene dynamics will unfold in the future. While the area of 3D reconstruction has progressed rapidly with the advent of recent feed-forward neural networks, forecasting future dynamics in 3D, given the 2D frames of a video remains unexplored. We present Point4Cast, a unified framework that processes streaming 2D frame sequences of a video to estimate the past, present, and future of the underlying dynamic scene, in 3D. At the core of our approach lies a persistently evolving latent spacetime representation that models the environment's evolution across time. Upon receiving a new 2D frame, an update operation integrates the incoming evidence to refine the latent spacetime representation. When queried for any time instant, whether before, at, or beyond the timestamp of the last update. A readout procedure predicts temporally conditioned point maps and camera parameters describing the scene geometry at the queried time. Unlike prior approaches for online dynamic scene reconstruction that estimate each frame's point map solely at the timestamp of the last observed frame, Point4Cast achieves coherent reconstruction across any queried time. Empirical evaluations show that Point4Cast achieves state-of-the-art performance on streaming dynamic scene reconstruction and forecasting benchmarks, across multiple challenging datasets, while providing scene flow estimation and forecasting for free. The code will be released publicly.