HAD：面向三维重建的幻觉感知扩散先验
HAD: Hallucination-Aware Diffusion Priors for 3D Reconstruction
用前馈新视角合成网络做多视角一致性推理，为扩散生成的增广视图逐像素打幻觉分数并掩码，可直接迁移到任何用扩散先验补视角的重建/生成管线中抑制不一致伪影。
原文依据 estimates pixel-wise hallucination score maps for augmented images by leveraging multi-view reasoning capabilities from a feedforward novel view synthesis (NVS) network
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论扩散先验近期在通过在新视角处增广训练视图以提升稀疏视角三维重建质量方面展现出强大能力，但其不可避免地会将幻觉内容——与输入视图不一致的伪影——引入最终的三维模型。为应对这一挑战，我们提出幻觉感知扩散先验（HAD），它借助在大规模三维数据上预训练的前馈新视角合成（NVS）网络的多视角推理能力，为增广图像估计逐像素的幻觉分数图。这些幻觉分数使得在渐进式三维重建过程中能够对不可靠像素进行选择性掩码，从而防止不存在的伪影被引入三维模型。为进一步提升性能，我们在每个新视角处通过以不同输入视图作为扩散先验的条件，生成增广图像的多个版本，随后将其融合为最终图像，以利用所有输入视图提供的更广上下文。我们表明，本方法大幅减少了扩散辅助三维重建中的幻觉伪影，从而在新视角合成的多个基准上取得最优性能。
英文原文摘要 1254 字符
Diffusion priors have recently demonstrated strong capability in enhancing the quality of sparse-view 3D reconstruction by augmenting training views at novel viewpoints, but they inevitably introduce hallucinated content-- artifacts inconsistent with the input views -- into the final 3D model. To address this challenge, we propose Hallucination-Aware Diffusion prior (HAD), which estimates pixel-wise hallucination score maps for augmented images by leveraging multi-view reasoning capabilities from a feedforward novel view synthesis (NVS) network pre-trained on large-scale 3D data. These hallucination scores enable selective masking of unreliable pixels during the progressive 3D reconstruction procedure, preventing the introduction of non-existent artifacts into the 3D model. To further enhance performance, we create multiple versions of augmented images at each novel view by conditioning the diffusion prior on different input views, which are then fused into a final image that leverages the broader context across all input views. We show that our method substantially reduces hallucination artifacts in diffusion-assisted 3D reconstruction, thereby achieving state-of-the-art performance across multiple benchmarks on novel view synthesis.