Lynx:面向高保真个性化视频生成
Lynx: Towards High-Fidelity Personalized Video Generation
用 Perceiver Resampler 把 ArcFace 人脸嵌入压成身份 token,再用 Ref-adapter 通过 cross attention 把冻结参考分支的 VAE 密集特征注入 DiT 所有层,是单图身份保持视频生成中可直接复用的双适配器条件注入方案。
原文依据 The ID-adapter employs a Perceiver Resampler to convert ArcFace-derived facial embeddings into compact identity tokens for conditioning
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论本文提出 Lynx,一个从单张输入图像生成高保真个性化视频的模型。Lynx 构建于开源扩散 Transformer(DiT)基础模型之上,引入两个轻量适配器以保证身份保真度。ID-adapter 采用 Perceiver Resampler 将 ArcFace 导出的人脸嵌入转换为紧凑的身份 token 作为条件,而 Ref-adapter 则整合来自冻结参考通路的密集变分自编码器特征,通过交叉注意力在所有 Transformer 层注入细粒度细节。这些模块共同实现了稳健的身份保持,同时维持时序连贯性与视觉真实感。在由 40 个受试者和 20 条无偏提示词构建、共产生 800 个测试用例的精选基准上评估,Lynx 展现出更优的人脸相似度、有竞争力的提示词跟随能力以及较强的视频质量,从而推进了个性化视频生成的水平。代码与模型将在论文发表后公开。
英文原文摘要 1005 字符
We present Lynx, a high-fidelity model for personalized video synthesis from a single input image. Built on an open-source Diffusion Transformer (DiT) foundation model, Lynx introduces two lightweight adapters to ensure identity fidelity. The ID-adapter employs a Perceiver Resampler to convert ArcFace-derived facial embeddings into compact identity tokens for conditioning, while the Ref-adapter integrates dense VAE features from a frozen reference pathway, injecting fine-grained details across all transformer layers through cross attention. These modules collectively enable robust identity preservation while maintaining temporal coherence and visual realism. Through evaluation on a curated benchmark of 40 subjects and 20 unbiased prompts, which yielded 800 test cases, Lynx has demonstrated superior face resemblance, competitive prompt following, and strong video quality, thereby advancing the state of personalized video generation. Code and models will be released publicly upon publication.