WorldStereo：通过三维几何记忆连接相机引导视频生成与场景重建
WorldStereo: Bridging Camera-Guided Video Generation and Scene Reconstruction via 3D Geometric Memories
把增量更新的点云作为全局几何记忆、用3D对应关系约束注意力感受野，为相机可控视频扩散提供了一种可复用的几何条件注入方式；同时其蒸馏骨干网络免联合训练的轻量控制分支设计，对做可控生成与少步采样的人有直接参考价值。
原文依据 the spatial-stereo memory constrains the model's attention receptive fields with 3D correspondence to focus on fine-grained details from the memory bank
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近期基础视频扩散模型（VDM）的进展已取得显著成果。然而，尽管生成的视频具有出色的视觉质量，由于相机可控性有限、且从不同相机轨迹观看时生成内容不一致，从这些输出中重建出一致的三维场景仍然颇具挑战。本文提出 WorldStereo，一个通过两个专用几何记忆模块连接相机引导视频生成与三维重建的新框架。具体而言，全局几何记忆通过增量更新的点云注入粗粒度结构先验，同时实现精确的相机控制。此外，空间立体记忆以三维对应关系约束模型的注意力感受野，使其聚焦于记忆库中的细粒度细节。这些组件使 WorldStereo 能够在精确相机控制下生成多视角一致的视频，从而促进高质量的三维重建。此外，基于灵活控制分支的 WorldStereo 展现出令人印象深刻的效率，这得益于无需联合训练的分布匹配蒸馏 VDM 骨干网络。在相机引导视频生成与三维重建基准上的大量实验证明了本方法的有效性。值得注意的是，我们展示了 WorldStereo 可作为一个强大的世界模型，以高保真三维结果处理多样的场景生成任务（无论从透视图像还是全景图像出发）。模型将开源。
英文原文摘要 1528 字符
Recent advances in foundational Video Diffusion Models (VDMs) have yielded significant progress. Yet, despite the remarkable visual quality of generated videos, reconstructing consistent 3D scenes from these outputs remains challenging, due to limited camera controllability and inconsistent generated content when viewed from distinct camera trajectories.In this paper, we propose WorldStereo, a novel framework that bridges camera-guided video generation and 3D reconstruction via two dedicated geometric memory modules. Formally, the global-geometric memory enables precise camera control while injecting coarse structural priors through incrementally updated point clouds.Moreover, the spatial-stereo memory constrains the model's attention receptive fields with 3D correspondence to focus on fine-grained details from the memory bank.These components enable WorldStereo to generate multi-view-consistent videos under precise camera control, facilitating high-quality 3D reconstruction.Furthermore, the flexible control branch-based WorldStereo shows impressive efficiency, benefiting from the distribution matching distilled VDM backbone without joint training.Extensive experiments across both camera-guided video generation and 3D reconstruction benchmarks demonstrate the effectiveness of our approach. Notably, we show that WorldStereo acts as a powerful world model, tackling diverse scene generation tasks (whether starting from perspective or panoramic images) with high-fidelity 3D results. Models will be released.