扩展零样本参考图到视频生成
Scaling Zero-Shot Reference-to-Video Generation
用掩码训练加定制注意力在纯视频-文本对上习得参考感知表征，绕开昂贵的参考图-视频-文本三元组标注，为身份保持视频生成提供了一条可扩展的零样本训练路径。
原文依据 Trained exclusively on video-text pairs, Saber employs a masked training strategy and a tailored attention-based model design
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论参考图到视频（R2V）生成旨在合成与文本提示词一致、同时保持参考图像中主体身份的视频。然而，现有 R2V 方法受制于对显式参考图-视频-文本三元组的依赖，而这类三元组的构建成本极高且难以规模化。我们通过引入 Saber 绕开这一瓶颈，这是一个可扩展的零样本框架，无需任何显式 R2V 数据。Saber 仅在视频-文本对上训练，采用掩码训练策略和定制的基于注意力的模型设计，以学习身份一致且参考感知的表征。进一步引入掩码增强技术，以缓解参考图到视频生成中常见的复制粘贴伪影。此外，Saber 在参考图数量变化时展现出显著的泛化能力，并在 OpenS2V-Eval 基准上取得了优于使用 R2V 数据训练的方法的性能。
英文原文摘要 968 字符
Reference-to-video (R2V) generation aims to synthesize videos that align with a text prompt while preserving the subject identity from reference images. However, current R2V methods are hindered by the reliance on explicit reference image-video-text triplets, whose construction is highly expensive and difficult to scale. We bypass this bottleneck by introducing Saber, a scalable zero-shot framework that requires no explicit R2V data. Trained exclusively on video-text pairs, Saber employs a masked training strategy and a tailored attention-based model design to learn identity-consistent and reference-aware representations. Mask augmentation techniques are further integrated to mitigate copy-paste artifacts common in reference-to-video generation. Moreover, Saber demonstrates remarkable generalization capabilities across a varying number of references and achieves superior performance on the OpenS2V-Eval benchmark compared to methods trained with R2V data.