Stand-In：一种轻量且即插即用的视频生成身份控制方法
Stand-In: A Lightweight and Plug-and-Play Identity Control for Video Generation
通过受限自注意力与条件位置映射把条件图像分支注入预训练视频生成模型，仅用约1%额外参数即可实现身份可控生成，为视觉研究者提供了一种低成本、可复用的个性化/换脸/姿态参考视频生成控制接口。
原文依据 Identity control is achieved through restricted self-attentions with conditional position mapping.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论生成与用户指定身份匹配的高保真人物视频在生成式人工智能领域重要但具有挑战性。现有方法往往依赖过多的训练参数，且与其他 AIGC 工具缺乏兼容性。本文提出 Stand-In，一种用于视频生成中身份保持的轻量且即插即用的框架。具体而言，我们在预训练视频生成模型中引入一个条件图像分支。身份控制通过带有条件位置映射的受限自注意力实现。得益于这些设计，它们极大地保留了视频生成模型的预训练先验，我们的方法在视频质量和身份保持方面能够超越其他全参数训练方法，即使仅使用 1% 的额外参数和仅 2000 对训练数据。此外，我们的框架可以无缝集成用于其他任务，例如主体驱动的视频生成、姿态参考的视频生成、风格化和换脸。代码和数据集将向社区公开。
英文原文摘要 1087 字符
Generating high-fidelity human videos that match user-specified identities is important yet challenging in the field of generative AI.Existing methods often rely on an excessive number of training parameters and lack compatibility with other AIGC tools.In this paper, we propose Stand-In, a lightweight and plug-and-play framework for identity preservation in video generation.Specifically, we introduce a conditional image branch into the pre-trained video generation model.Identity control is achieved through restricted self-attentions with conditional position mapping.Thanks to these designs, which greatly preserve the pretrained prior of the video generation model, our approach is able to outperform other full-parameter training methods in video quality and identity preservation, even with just 1% additional parameters and only 2000 training pairs.Moreover, our framework can be seamlessly integrated for other tasks, such as subject-driven video generation, pose-referenced video generation, stylization, and face swapping.Code and dataset will be available to the community.