学习潜变量代理实现可控单图重光照
Learning Latent Proxies for Controllable Single-Image Relighting
用少量 PBR 监督训练一个少样本潜变量代理编码器，把稀疏材质-几何线索注入扩散去噪器，并用光照感知掩码只引导对光照敏感的像素；这套「稀疏物理先验 + 掩码引导」的思路可直接迁移到阴影编辑、材质编辑等需要物理可控性的图像生成任务。
原文依据 a few-shot latent proxy encoder that extracts compact material-geometry cues from limited PBR supervision
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论单图重光照是高度欠约束的问题：微小的光照变化会在着色、阴影和高光上产生大幅非线性变化，而几何与材质均不可观测。现有的基于扩散的方法要么依赖内在分解或 G-buffer 流程，需要稠密且脆弱的监督，要么完全在潜空间中操作而缺乏物理依据，导致对方向、强度和颜色的细粒度控制不可靠。我们观察到，精确重光照并不需要完整的内在分解。相反，稀疏但具有物理意义的线索——指示光照应在何处变化以及材质应如何响应——就足以引导扩散模型。基于这一洞见，我们提出 LightCtrl，在两个层面集成最小化的物理先验：一个少样本潜变量代理编码器，从有限的 PBR 监督中提取紧凑的材质-几何线索；以及一个光照感知掩码，用于识别对光照敏感的区域，并引导去噪器关注与着色相关的像素。为弥补 PBR 数据稀缺，我们使用基于 DPO 的目标对代理分支进行精炼，使预测线索与感知上更受偏好的重光照行为对齐。我们进一步提出 ScaLight，一个大规模物体级数据集，具有系统变化的光照和完整的相机-光源元数据，支持物理一致且可控的训练。在物体级和场景级基准上，我们的方法实现了光度上忠实的重光照与准确的连续控制，超越了先前的扩散基线和内在分解基线，在受控光照变化下峰值信噪比最高提升 2.4 dB，RMSE 降低 35%。
英文原文摘要 1716 字符
Single-image relighting is highly under-constrained: small illumination changes can produce large, nonlinear variations in shading, shadows, and specularities, while geometry and materials remain unobserved. Existing diffusion-based approaches either rely on intrinsic- or G-buffer-based pipelines that require dense and fragile supervision, or operate purely in latent space without physical grounding, making fine-grained control of direction, intensity, and color unreliable. We observe that full intrinsic decomposition is unnecessary for accurate relighting. Instead, sparse but physically meaningful cues--indicating where illumination should change and how materials should respond--are sufficient to guide a diffusion model. Based on this insight, we introduce LightCtrl that integrates minimal physical priors at two levels: a few-shot latent proxy encoder that extracts compact material-geometry cues from limited PBR supervision, and a lighting-aware mask that identifies illumination-sensitive regions and steers the denoiser toward shading-relevant pixels. To compensate for scarce PBR data, we refine the proxy branch using a DPO-based objective that aligns predicted cues with perceptually preferred relighting behavior. We further present ScaLight, a large-scale object-level dataset with systematically varied illumination and complete camera-light metadata, enabling physically consistent and controllable training. Across object- and scene-level benchmarks, our method achieves photometrically faithful relighting with accurate continuous control, surpassing prior diffusion- and intrinsic-based baselines, including gains of up to +2.4 dB PSNR and 35% lower RMSE under controlled lighting shifts.