FaithFusion:通过像素级信息增益协调重建与生成
FaithFusion: Harmonizing Reconstruction and Generation via Pixel-wise Information Gain
将像素级期望信息增益(EIG)作为统一策略,同时引导扩散模型精修高不确定性区域并将编辑结果蒸馏回3DGS,为3DGS与扩散模型的融合提供了一种无需额外先验条件的即插即用方案,可复用于其他需要几何保真与外观生成平衡的新视角合成任务。
原文依据 EIG acts as a unified policy for coherent spatio-temporal synthesis: it guides diffusion as a spatial prior to refine high-uncertainty regions, while its pixel-level weighting distills the edits back into 3DGS.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论在可控驾驶场景重建与三维场景生成中,在大视角偏移下保持几何保真度并合成视觉上合理的外观至关重要。然而,基于几何的三维高斯泼溅(3DGS)与外观驱动的扩散模型的有效融合面临固有挑战,因为缺乏像素级、三维一致的编辑准则往往导致过度修复和几何漂移。为解决这些问题,我们提出了FaithFusion,一个由像素级期望信息增益(EIG)驱动的3DGS-扩散融合框架。EIG作为连贯时空合成的统一策略:它作为空间先验引导扩散模型精修高不确定性区域,同时其像素级加权将编辑结果蒸馏回3DGS。由此得到的即插即用系统无需额外先验条件和结构修改。在Waymo数据集上的大量实验表明,我们的方法在NTA-IoU、NTL-IoU和弗雷歇初始距离(FID)上均达到最先进性能,即使在6米车道偏移下仍保持107.47的FID。
英文原文摘要 1064 字符
In controllable driving-scene reconstruction and 3D scene generation, maintaining geometric fidelity while synthesizing visually plausible appearance under large viewpoint shifts is crucial. However, effective fusion of geometry-based 3DGS and appearance-driven diffusion models faces inherent challenges, as the absence of pixel-wise, 3D-consistent editing criteria often leads to over-restoration and geometric drift. To address these issues, we introduce **FaithFusion**, a 3DGS-diffusion fusion framework driven by pixel-wise Expected Information Gain (EIG). EIG acts as a unified policy for coherent spatio-temporal synthesis: it guides diffusion as a spatial prior to refine high-uncertainty regions, while its pixel-level weighting distills the edits back into 3DGS. The resulting plug-and-play system is free from extra prior conditions and structural modifications. Extensive experiments on the Waymo dataset demonstrate that our approach attains SOTA performance across NTA-IoU, NTL-IoU, and FID, maintaining an FID of 107.47 even at 6 meters lane shift.