D4RT:高效重建动态场景
Efficiently Reconstructing Dynamic Scenes One D4RT at a Time
用统一 Transformer 解码接口从视频中按任意时空点查询 3D 位置,绕开逐帧稠密解码,为动态 4D 重建提供了一种轻量、可扩展的前馈范式,可直接借鉴到新视角合成与点追踪任务。
原文依据 allows the model to independently and efficiently probe the 3D position of any point in space and time
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论从视频中理解并重建动态 4D 场景的复杂几何与运动,仍是计算机视觉中一项艰巨挑战。本文提出 D4RT,一个简单而强大的前馈网络,旨在高效求解该任务。D4RT 采用统一的 Transformer 架构,从单段视频中联合推断深度、时空对应关系以及完整的相机参数。其核心创新是一种新机制,规避了逐帧稠密解码的高昂计算开销,也避免了管理多个任务专用解码器的复杂性。我们的统一解码接口使模型能够独立且高效地探测空间中任意点在任意时刻的 3D 位置。由此得到一种轻量且高度可扩展的方法,可实现极为高效的训练与推理。我们证明该方法达到了新的最优性能,在广泛的 4D 重建任务上优于此前方法。
英文原文摘要 975 字符
Understanding and reconstructing the complex geometry and motion of dynamic 4D scenes from video remains a formidable challenge in computer vision. This paper introduces D4RT, a simple yet powerful feedforward network designed to efficiently solve this task. D4RT utilizes a unified transformer architecture to jointly infer depth, spatio-temporal correspondence, and full camera parameters from a single video. Its core innovation is a novel mechanism that sidesteps the heavy computation of dense, per-frame decoding and the complexity of managing multiple, task-specific decoders. Our unified decoding interface allows the model to independently and efficiently probe the 3D position of any point in space and time. The result is a lightweight and highly scalable method that enables remarkably efficient training and inference. We demonstrate that our approach sets a new state-of-the-art, outperforming previous methods across a wide spectrum of 4D reconstruction tasks.