3D 视觉与重建 · CVPR2026 ·2026.06.03

MVInverse:秒级前馈多视图逆渲染

MVInverse: Feed-forward Multiview Inverse Rendering in Seconds

Xiangzuo Wu、Chengwei Ren、Jun Zhou、Xiu Li、Yuan Liu

与你有关的部分

提出前馈多视图逆渲染框架,通过跨视图注意力实现单次前向预测空间变化的材质与法线,并利用无标签真实视频进行一致性微调,为多视图逆渲染提供了可扩展的实时方案。

原文依据 we introduce a feed-forward multi-view inverse rendering framework that directly predicts spatially varying albedo, metallicity, roughness, diffuse shading, and surface normals from sequences of RGB images.
可迁移到 逆渲染材质估计法线估计多视图一致性

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

多视图逆渲染旨在从多个视角一致地恢复几何、材质和光照。现有的单视图方法往往忽略跨视图关系,导致结果不一致;而多视图优化方法依赖缓慢的可微渲染和逐场景细化,计算成本高且难以扩展。为解决这些局限,我们提出了一种前馈多视图逆渲染框架,可直接从RGB图像序列中预测空间变化的反照率、金属度、粗糙度、漫反射着色和表面法线。通过跨视图交替注意力,我们的模型同时捕捉视图内的长程光照交互和视图间的材质一致性,从而在单次前向传播中实现连贯的场景级推理。由于真实世界训练数据稀缺,在现有合成数据集上训练的模型往往难以泛化到真实场景。为克服这一限制,我们提出了一种基于一致性的微调策略,利用无标签的真实世界视频来增强多视图连贯性和野外条件下的鲁棒性。在基准数据集上的大量实验表明,我们的方法在多视图一致性、材质和法线估计质量以及对真实世界图像的泛化方面均达到了最优性能。

英文原文摘要 1416 字符

Multi-view inverse rendering aims to recover geometry, materials, and illumination consistently across multiple viewpoints. Existing single-view approaches often ignore cross-view relationships, leading to inconsistent results, while multi-view optimization methods rely on slow differentiable rendering and per-scene refinement, making them computationally expensive and hard to scale. To address these limitations, we introduce a feed-forward multi-view inverse rendering framework that directly predicts spatially varying albedo, metallicity, roughness, diffuse shading, and surface normals from sequences of RGB images. By alternating attention across views, our model captures both intra-view long-range lighting interactions and inter-view material consistency, enabling coherent scene-level reasoning within a single forward pass. Due to the scarcity of real-world training data, models trained on existing synthetic datasets often struggle to generalize to real-world scenes. To overcome this limitation, we propose a consistency-based finetuning strategy that leverages unlabeled real-world videos to enhance both multi-view coherence and robustness under in-the-wild conditions. Extensive experiments on benchmark datasets demonstrate that our method achieves state-of-the-art performance in terms of multi-view consistency, material and normal estimation quality, and generalization to real-world imagery.

标签

会议 CVPR2026
页码 37343-37357
Bibkey
来源 CVF Open Access