基于奖励引导优化的身份保持图像到视频生成
Identity-Preserving Image-to-Video Generation via Reward-Guided Optimization
提出基于强化学习的奖励引导优化框架，通过人脸身份评分器反向传播奖励信号至采样链最后几步，并利用真实视频人脸特征池增强泛化性，为扩散模型的身份保持提供可复用技术。
原文依据 backpropagates the reward signal through the last steps of the sampling chain
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近期图像到视频（I2V）生成在从静态图像合成高质量、时序连贯视频方面取得了显著进展。在I2V的所有应用中，以人为中心的视频生成占据很大比例。然而，现有I2V模型难以保持输入人物图像与生成视频之间的身份一致性，尤其是当视频中的人物表现出显著的表情变化和运动时。当人脸仅占图像很小一部分时，这一问题变得尤为关键。由于人类对身份变化高度敏感，这在I2V生成中构成了一个关键但尚未充分探索的挑战。本文提出身份保持奖励引导优化（IPRO），一种基于强化学习的新型视频扩散框架，以增强身份保持。我们的方法不引入辅助模块或改变模型架构，而是引入一种直接有效的调优算法，利用人脸身份评分器优化扩散模型。为提高性能并加速收敛，我们的方法通过采样链的最后几步反向传播奖励信号，从而获得更丰富的梯度反馈。我们还提出一种新颖的面部评分机制，将真实视频中的人脸视为面部特征池，提供多角度面部信息以增强泛化性。进一步引入KL散度正则化以稳定训练并防止对奖励信号过拟合。在Wan 2.2 I2V模型和我们自研的I2V模型上的大量实验证明了我们方法的有效性。项目见 https://dof-gaussian.github.io/。
英文原文摘要 1753 字符
Recent advances in image-to-video (I2V) generation have achieved remarkable progress in synthesizing high-quality, temporally coherent videos from static images. Among all the applications of I2V, human-centric video generation includes a large portion. However, existing I2V models encounter difficulties in maintaining identity consistency between the input human image and the generated video, especially when the person in the video exhibits significant expression changes and movements. This issue becomes critical when the human face occupies merely a small fraction of the image. Since humans are highly sensitive to identity variations, this poses a critical yet under-explored challenge in I2V generation. In this paper, we propose Identity-Preserving Reward-guided Optimization (IPRO), a novel video diffusion framework based on reinforcement learning to enhance identity preservation. Instead of introducing auxiliary modules or altering model architectures, our approach introduces a direct and effective tuning algorithm that optimizes diffusion models using a face identity scorer. To improve performance and accelerate convergence, our method backpropagates the reward signal through the last steps of the sampling chain, enabling richer gradient feedback. We also propose a novel facial scoring mechanism that treats faces in ground-truth videos as facial feature pools, providing multi-angle facial information to enhance generalization. A KL-divergence regularization is further incorporated to stabilize training and prevent overfitting to the reward signal. Extensive experiments on Wan 2.2 I2V model and our in-house I2V model demonstrate the effectiveness of our method. Our project is available at https://dof-gaussian.github.io/.