Extend3D：城镇尺度三维生成
Extend3D: Town-Scale 3D Generation
把以物体为中心的 3D 生成模型的潜空间在 x/y 方向扩展并切成重叠图像块、逐时间步耦合，配合单目深度点云先验做空间对齐，是把固定尺寸 3D 生成器推广到城市场景的可复用套路；其中「under-noising」把结构缺失当作噪声、用 SDEdit 迭代补全遮挡区域，可直接迁移到场景补全与长序列外扩生成。
原文依据 we initialize the scene using a point cloud prior from a monocular depth estimator and iteratively refine occluded regions through SDEdit
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论本文提出 Extend3D，一个基于以物体为中心的 3D 生成模型、从单张图像生成三维场景的免训练流程。为克服以物体为中心的模型中固定尺寸潜空间难以表示宽广场景的局限，我们在 x 与 y 方向上扩展潜空间。随后，将扩展后的潜空间划分为重叠的图像块，对每个图像块施加该以物体为中心的 3D 生成模型，并在每个时间步上将它们耦合。由于带图像条件的逐块 3D 生成要求图像与潜变量图像块之间严格的空间对齐，我们利用来自单目深度估计器的点云先验初始化场景，并通过 SDEdit 迭代精修被遮挡区域。我们发现，在 3D 精修过程中将三维结构的不完整性视为噪声，可以借助概念实现三维补全，我们将这一做法称为 under-noising。此外，为缓解以物体为中心的模型在子场景生成上的次优性，我们在去噪过程中优化扩展后的潜变量，确保去噪轨迹与子场景动态保持一致。为此，我们引入 3D 感知的优化目标，以提升几何结构与纹理的保真度。人类偏好实验与定量实验表明，本方法取得了优于先前方法的结果。
英文原文摘要 1390 字符
In this paper, we propose Extend3D, a training-free pipeline for 3D scene generation from a single image, built upon an object-centric 3D generative model. To overcome the limitations of fixed-size latent spaces in object-centric models for representing wide scenes, we extend the latent space in the x and y directions. Then, by dividing the extended latent space into overlapping patches, we apply the object-centric 3D generative model to each patch and couple them at each time step. Since patch-wise 3D generation with image conditioning requires strict spatial alignment between image and latent patches, we initialize the scene using a point cloud prior from a monocular depth estimator and iteratively refine occluded regions through SDEdit. We discovered that treating the incompleteness of 3D structure as noise during 3D refinement enables 3D completion via a concept, which we term under-noising. Furthermore, to address the sub-optimality of object-centric models for sub-scene generation, we optimize the extended latent during denoising, ensuring that the denoising trajectories remain consistent with the sub-scene dynamics. To this end, we introduce 3D-aware optimization objectives for improved geometric structure and texture fidelity. We demonstrate that our method yields better results than prior methods, as evidenced by human preference and quantitative experiments.