Points-to-3D：以点云先验实现结构感知的三维生成
Points-to-3D: Structure-Aware 3D Generation with Point Cloud Priors
把可见区域点云（LiDAR 或 VGGT 前馈预测）编码进 TRELLIS 的稀疏结构潜变量，用结构图像修复网络加两阶段采样补全不可见几何，为视觉研究者提供了一条用现成点云先验做几何可控 3D 生成的通用路径。
原文依据 replaces pure-noise sparse structure latent initialization with a point cloud priors tailored input formulation
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近期三维生成领域的进展主要由以图像或文本为条件的模型推动，而易于获取的三维先验仍未得到充分利用。在许多真实场景中，可见区域点云很容易获得——既可来自激光雷达等主动传感器，也可来自 VGGT 这类前馈预测器——它们提供了显式的几何约束，而现有方法未能加以利用。本工作提出 Points-to-3D，一个基于扩散的框架，利用点云先验实现几何可控的三维资产与场景生成。Points-to-3D 构建于潜空间三维扩散模型 TRELLIS 之上，首先用面向点云先验定制的输入形式替代纯噪声的稀疏结构潜变量初始化。随后，在 TRELLIS 框架内于任务特定数据上训练的结构图像修复网络被用于推理，该网络旨在学习全局结构修复，并配合分阶段采样策略（先结构修复、后边界精化），在补全整体几何的同时保留输入先验的可见区域。在实践中，Points-to-3D 既可接受精确的点云先验，也可接受由单张图像经 VGGT 估计的点云作为输入。在单物体与多物体场景上的实验一致表明，该方法在渲染质量与几何保真度上优于最优基线，凸显了显式嵌入点云先验对于实现更准确、结构更可控的三维生成的有效性。
英文原文摘要 1538 字符
Recent progress in 3D generation has been driven largely by models conditioned on images or text, while readily available 3D priors are still underused. In many real-world scenarios, the visible-region point cloud are easy to obtain--from active sensors such as LiDAR or from feed-forward predictors like VGGT--offering explicit geometric constraints that current methods fail to exploit. In this work, we introduce Points-to-3D, a diffusion-based framework that leverages point cloud priors for geometry-controllable 3D asset and scene generation. Built on a latent 3D diffusion model TRELLIS, Points-to-3D first replaces pure-noise sparse structure latent initialization with a point cloud priors tailored input formulation. A structure inpainting network, trained within the TRELLIS framework on task-specific data designed to learn global structural inpainting, is then used for inference with a staged sampling strategy (structural inpainting followed by boundary refinement), completing the global geometry while preserving the visible regions of the input priors.In practice, Points-to-3D can take either accurate point-cloud priors or VGGT-estimated point clouds from single images as input.Experiments on both single-object and multi-object scenarios consistently demonstrate superior performance over state-of-the-art baselines in terms of rendering quality and geometric fidelity, highlighting the effectiveness of explicitly embedding point-cloud priors for achieving more accurate and structurally controllable 3D generation.