HoloCine:电影级多镜头长视频叙事的整体生成
HoloCine: Holistic Generation of Cinematic Multi-Shot Long Video Narratives
窗口交叉注意力把文本提示定位到具体镜头、镜头内稠密镜头间稀疏的自注意力模式,是让长视频生成在分钟级尺度上保持全局一致且算力可控的可复用设计,可直接迁移到任何需要跨片段一致性的长序列生成任务。
原文依据 a Sparse Inter-Shot Self-Attention pattern (dense within shots but sparse between them) ensures the efficiency required for minute-scale generation
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论最优的文本到视频模型擅长生成孤立的片段,却难以创作连贯的多镜头叙事,而后者正是讲故事的精髓。我们以 HoloCine 弥合这一「叙事鸿沟」,该模型整体地生成完整场景,从而确保从第一个镜头到最后一个镜头的全局一致性。我们的架构通过窗口交叉注意力机制实现精确的导演控制,该机制将文本提示定位到特定镜头;同时采用稀疏镜头间自注意力模式(镜头内稠密、镜头间稀疏),保证了分钟级生成所需的效率。除了在叙事连贯性上树立新的最优水平之外,HoloCine 还展现出显著的新兴能力:对角色与场景的持久记忆,以及对电影技法的直觉性把握。我们的工作标志着从片段合成迈向自动化电影制作的关键转变,使端到端的电影级创作成为触手可及的未来。
英文原文摘要 975 字符
State-of-the-art text-to-video models excel at generating isolated clips but fall short of creating the coherent, multi-shot narratives, which are the essence of storytelling. We bridge this "narrative gap" with HoloCine, a model that generates entire scenes holistically to ensure global consistency from the first shot to the last. Our architecture achieves precise directorial control through a Window Cross-Attention mechanism that localizes text prompts to specific shots, while a Sparse Inter-Shot Self-Attention pattern (dense within shots but sparse between them) ensures the efficiency required for minute-scale generation. Beyond setting a new state-of-the-art in narrative coherence, HoloCine develops remarkable emergent abilities: a persistent memory for characters and scenes, and an intuitive grasp of cinematic techniques. Our work marks a pivotal shift from clip synthesis towards automated filmmaking, making end-to-end cinematic creation a tangible future.