生成模型与扩散 · CVPR2026 ·2026.06.03

DiffusionHarmonizer:以在线扩散增强器连接神经重建与照片级真实感仿真

DiffusionHarmonizer: Bridging Neural Reconstruction and Photorealistic Simulation with Online Diffusion Enhancer

Yuxuan Zhang、Katarína Tóthová、Zian Wang、Kangxue Yin、Haithem Turki、Riccardo de Lutio 等 10 人

与你有关的部分

把预训练多步图像扩散模型蒸馏成单步、时序条件化的在线增强器,可直接套在 NeRF/3DGS 渲染结果之后做后处理,为神经重建的新视角伪影与插入物体光照不匹配提供可复用的实时修复方案。

原文依据 a single-step temporally-conditioned enhancer that is converted from a pretrained multi-step image diffusion model
可迁移到 新视角合成图像和谐化视频增强扩散模型蒸馏

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

仿真是自动驾驶车辆等自主机器人开发与评估的关键环节。神经重建正成为一种有前景的解决方案,因为它能够仅凭真实世界数据,以自动化、可扩展的方式仿真多种场景。然而,神经辐射场与三维高斯泼溅等方法虽然能产生视觉上颇具说服力的结果,却常常出现伪影,尤其是在渲染新视角时;并且无法真实地融合插入的动态物体,特别是当这些物体来自不同场景时。为克服这些局限,我们提出 DiffusionHarmonizer,一个在线生成式增强框架,可将此类不完美场景的渲染结果转化为照片级真实感且时序一致的输出。其核心是一个单步时序条件增强器,由预训练的多步图像扩散模型转换而来,能够在单块 GPU 上的在线仿真器中运行。有效训练它的关键在于一套定制的数据策展流程,该流程构建合成—真实配对数据,重点覆盖外观和谐化、伪影校正与光照真实感。实验表明,DiffusionHarmonizer 显著提升了感知真实感,在我们的对比研究中被 84.28% 的用户选为优于第二名的方案。此外,它在保持单步图像模型推理效率的同时,达到了当前最优视频模型的时序一致性,为研究与生产环境中的照片级真实感仿真提供了可扩展且实用的解决方案。

英文原文摘要 1602 字符

Simulation is essential to the development and evaluation of autonomous robots such as self-driving vehicles. Neural reconstruction is emerging as a promising solution as it enables simulating a wide variety of scenarios from real-world data alone in an automated and scalable way. However, while methods such as NeRF and 3D Gaussian Splatting can produce visually compelling results, they often exhibit artifacts particularly when rendering novel views, and fail to realistically integrate inserted dynamic objects, especially when they were captured from different scenes. To overcome these limitations we introduce DiffusionHarmonizer, an online generative enhancement framework that transforms renderings from such imperfect scenes into photorealistic, temporally consistent outputs. At its core is a single-step temporally-conditioned enhancer that is converted from a pretrained multi-step image diffusion model, capable of running in online simulators on a single GPU. The key to training it effectively, is a custom data curation pipeline that constructs synthetic-real pairs emphasizing appearance harmonization, artifact correction, and lighting realism. Experiments show that DiffusionHarmonizer substantially improves perceptual realism, being chosen by 84.28% of users in our comparative study over the second best method. Furthermore, it matches the temporal coherence of state-of-the art video models while maintaining the inference efficiency of single-step image models, offering a scalable and practical solution for photorealistic simulation in both research and production settings.

标签

会议 CVPR2026
页码 43494-43504
Bibkey
来源 CVF Open Access