重打光视频人像协调
Relightful Video Portrait Harmonization
提出基于视频扩散模型的重打光协调框架，利用去闪烁模型稳定逐帧协调结果以生成训练数据，并用非对称 alpha 掩码条件学习干净边界，可复用于视频编辑中的时序一致性与边界处理。
原文依据 We overcome this problem by introducing a novel lighting deflickering model that can stabilize the global and local lighting flickering artifacts.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论我们提出一种方法，用于协调前景视频的光照以匹配目标背景场景，调整阴影、色调和光照强度（重打光协调）。与图像不同，获取视频的标注数据——即在不同的光照条件下记录相同动作——实际上不可行且不可扩展。虽然创建此类配对数据的一种方式是逐帧将现有的基于图像的协调模型应用于视频，但得到的输出往往存在显著的时序抖动。我们通过引入一种新颖的光照去闪烁模型来克服这个问题，该模型能够稳定全局和局部的光照闪烁伪影。我们的视频扩散模型从这些升级后的去闪烁数据中学习，并结合大量真实和合成视频，以生成高质量的视频协调结果。我们进一步提出一种非对称 alpha 掩码条件技术，以从真实视频中学习干净的边界。实验表明，与先前的基于图像和基于视频的协调方法相比，我们的模型实现了强时序一致性、自然性、更干净的边界和物理上合理的光照行为，同时保持了强大的重打光表现力。
英文原文摘要 1254 字符
We present a method for harmonizing the lighting of a foreground video to match a target background scene, adjusting shadows, color tone, and illumination intensity (relightful harmonization). Unlike images, acquiring labeled data for videos, where identical motions are recorded under different lighting conditions, is practically infeasible and non-scalable. While one way to create such paired data is to apply existing image-based harmonization models frame by frame to a video, the resulting outputs often suffer from significant temporal jitters. We overcome this problem by introducing a novel lighting deflickering model that can stabilize the global and local lighting flickering artifacts. Our video diffusion model learns from these upgraded deflickered data with a volume of real and synthetic videos to generate high-quality video harmonization results. We further propose an asymmetric alpha mask conditioning technique to learn the clean boundaries from real videos. Experiments demonstrate that our model achieves strong temporal coherence, naturalness, cleaner boundaries, and physically meaningful lighting behavior, while maintaining strong relighting expressiveness compared to prior image-based and video-based harmonization methods.