GIFSplat:生成先验引导的迭代式前馈稀疏视角三维高斯泼溅
GIFSplat: Generative Prior-Guided Iterative Feed-Forward 3D Gaussian Splatting from Sparse Views
把「一次性前馈预测」改成少量前馈式残差迭代,并用冻结扩散先验蒸馏出高斯级线索,无需测试时梯度优化即可在稀疏无位姿视角下做场景自适应,可直接迁移到其他前馈式 3DGS/NeRF 管线以提升域外泛化与质量。
原文依据 A small number of forward-only residual updates progressively refine current 3D scene using rendering evidence
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论前馈三维重建在运行时间上相较逐场景优化具有显著优势,后者在推理时仍然缓慢,且在稀疏视角下往往较为脆弱。然而,现有前馈方法仍有进一步提升性能的空间,尤其是在域外数据上,并且一旦引入生成先验,就难以维持秒级推理时间。这些局限源于现有前馈管线中的一次性预测范式:模型严格受限于容量,缺乏推理时细化能力,也不适合持续注入生成先验。我们提出 GIFSplat,一个纯前馈的迭代细化框架,用于从稀疏且无位姿的视角重建三维高斯泼溅。少量仅前向的残差更新利用渲染证据逐步细化当前三维场景,在效率与质量之间取得良好平衡。此外,我们将冻结的扩散先验蒸馏为来自增强新视角渲染的高斯级线索,无需梯度反向传播,也无需不断扩张的视角集合,从而在保持前馈效率的同时实现带生成先验的逐场景自适应。在 DL3DV、RealEstate10K 和 DTU 上,GIFSplat 一致优于最优的前馈基线,峰值信噪比最高提升 +2.1 dB,并且在无需相机位姿、也不进行任何测试时梯度优化的情况下保持秒级推理时间。项目页面见:https://terencepp.github.io/gifsplat-project-page/。
英文原文摘要 1543 字符
Feed-forward 3D reconstruction offers substantial runtime advantages over per-scene optimization, which remains slow at inference and often fragile under sparse views. However, existing feed-forward methods still have potential for further performance gains, especially for out-of-domain data, and struggle to retain second-level inference time once a generative prior is introduced. These limitations stem from the one-shot prediction paradigm in existing feed-forward pipeline: models are strictly bounded by capacity, lack inference-time refinement, and are ill-suited for continuously injecting generative priors. We introduce GIFSplat, a purely feed-forward iterative refinement framework for 3D Gaussian Splatting from sparse unposed views. A small number of forward-only residual updates progressively refine current 3D scene using rendering evidence, achieve favorable balance between efficiency and quality. Furthermore, we distill a frozen diffusion prior into Gaussian-level cues from enhanced novel renderings without gradient backpropagation or ever-increasing view-set expansion, thereby enabling per-scene adaptation with generative prior while preserving feed-forward efficiency. Across DL3DV, RealEstate10K, and DTU, GIFSplat consistently outperforms state-of-the-art feed-forward baselines, improving PSNR by up to +2.1 dB, and it maintains second-scale inference time without requiring camera poses or any test-time gradient optimization. The project page can be found at: https://terencepp.github.io/gifsplat-project-page/.