Splatent：泼溅扩散潜变量用于新视角合成
Splatent: Splatting Diffusion Latents for Novel View Synthesis
在 VAE 潜空间做 3DGS 新视角合成时，不必在 3D 空间里补细节，而是用多视角注意力从输入视角的 2D 特征中恢复高频细节，从而在不微调 VAE、不依赖扩散模型幻觉的前提下提升重建保真度；这一思路可直接迁移到任何潜空间辐射场或前馈式稀疏视角重建管线。
原文依据 rather than reconstructing fine-grained details in 3D space, we recover them in 2D from input views through multi-view attention mechanisms
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论辐射场表示近期被探索于扩散模型常用的变分自编码器潜空间中。这一方向带来了高效渲染以及与基于扩散的流程的无缝集成。然而，这类方法面临一个根本性局限：变分自编码器的潜空间缺乏多视角一致性，导致三维重建过程中纹理模糊、细节缺失。现有方法试图通过微调变分自编码器来解决该问题，但以重建质量为代价；或依赖预训练的扩散模型来恢复细粒度细节，但存在产生幻觉的风险。我们提出 Splatent，一个基于扩散的增强框架，作用于变分自编码器潜空间中三维高斯泼溅之上。我们的关键洞见偏离了以三维为中心的传统视角：我们不在三维空间中重建细粒度细节，而是通过多视角注意力机制从输入视角在二维中恢复这些细节。该方法在保持预训练变分自编码器重建质量的同时，实现了忠实的细节恢复。在多个基准上的评估表明，Splatent 为变分自编码器潜空间辐射场重建建立了新的最优性能。我们进一步证明，将本方法与现有的前馈框架集成，能够持续改善细节保持能力，为高质量稀疏视角三维重建开辟了新的可能。代码已在项目主页公开。
英文原文摘要 1468 字符
Radiance field representations have recently been explored in the latent space of VAEs that are commonly used by diffusion models. This direction offers efficient rendering and seamless integration with diffusion-based pipelines. However, these methods face a fundamental limitation: The VAE latent space lacks multi-view consistency, leading to blurred textures and missing details during 3D reconstruction. Existing approaches attempt to address this by fine-tuning the VAE, at the cost of reconstruction quality, or by relying on pre-trained diffusion models to recover fine-grained details, at the risk of some hallucinations. We present Splatent, a diffusion-based enhancement framework designed to operate on top of 3D Gaussian Splatting (3DGS) in the latent space of VAEs. Our key insight departs from the conventional 3D-centric view: rather than reconstructing fine-grained details in 3D space, we recover them in 2D from input views through multi-view attention mechanisms. This approach preserves the reconstruction quality of pretrained VAEs while achieving faithful detail recovery.Evaluated across multiple benchmarks, Splatent establishes a new state-of-the-art for VAE latent radiance field reconstruction. We further demonstrate that integrating our method with existing feed-forward frameworks, consistently improves detail preservation, opening new possibilities for high-quality sparse-view 3D reconstruction. Code is available on our project page.