WildCap：通过混合逆向渲染实现野外环境下的人脸反照率捕捉
WildCap: Facial Albedo Capture in the Wild via Hybrid Inverse Rendering
提出混合逆向渲染框架：先用数据驱动方法（SwitchLight）把野外图像转成受约束条件，再用基于模型的逆向渲染；并用 texel grid 光照模型把网络预测的非物理伪影（如阴影烘焙）解释为局部物理光照下的干净反照率，配合扩散先验联合优化以消解局部光照与反照率的尺度歧义。这套「数据驱动预处理 + 物理逆向渲染 + 生成先验正则」的思路可迁移到其他本征分解与材质估计任务。
原文依据 we propose a novel texel grid lighting model to explain non-physical effects as clean albedo illuminated by local physical lighting
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论现有方法在可控光照条件下能够实现高质量的人脸反照率捕捉，但这会提高采集成本并限制其可用性。我们提出 WildCap，一种从野外拍摄的智能手机视频中高质量捕捉人脸反照率的新方法。为从野外采集中的复杂光照效应中分离出高质量反照率，我们提出一种新颖的混合逆向渲染框架。我们首先采用数据驱动方法，即 SwitchLight，将采集图像转换为约束更强的条件，然后采用基于模型的逆向渲染。然而，网络预测中不可避免的局部伪影（如阴影烘焙）是非物理的，因而会阻碍对光照与材质的准确逆向渲染。为解决这一问题，我们提出一种新颖的 texel 网格光照模型，将非物理效应解释为受局部物理光照照亮的干净反照率。在优化过程中，我们联合采样反照率图的扩散先验并优化光照，从而有效消解局部光源与反照率之间的尺度歧义。随后由反照率预测其他反射率图。在相同采集设置下，我们的方法取得的结果显著优于先前工作，大幅缩小了野外采集与可控采集之间的质量差距。
英文原文摘要 1306 字符
Existing methods achieve high-quality facial albedo capture under controllable lighting, which increases capture cost and limits usability. We propose WildCap, a novel method for high-quality facial albedo capture from a smartphone video recorded in the wild. To disentangle high-quality albedo from complex lighting effects in in-the-wild captures, we propose a novel hybrid inverse rendering framework. We first apply a data-driven method, i.e., SwitchLight, to convert the captured images into more constrained conditions and then adopt model-based inverse rendering. However, unavoidable local artifacts in network predictions, such as shadow-baking, are non-physical and thus hinder accurate inverse rendering of lighting and material. To address this, we propose a novel texel grid lighting model to explain non-physical effects as clean albedo illuminated by local physical lighting. During optimization, we jointly sample a diffusion prior for the albedo map and optimize the lighting, effectively resolving scale ambiguity between local lights and albedo. Other reflectance maps are then predicted from the albedo. Our method achieves significantly better results than prior arts in the same capture setup, closing the quality gap between in-the-wild and controllable recordings by a large margin.