STAGE:故事板锚定的电影级多镜头叙事生成
STAGE: Storyboard-Anchored Generation for Cinematic Multi-shot Narrative
提出以结构化故事板(每镜头的起止帧对)替代稀疏关键帧来锚定多镜头视频生成,并引入多镜头记忆包与双编码策略保证跨镜头实体一致性和镜头内连贯性,为可控叙事视频生成提供了可复用的条件控制与记忆机制。
原文依据 we propose STEP^2 to predict a structural storyboard composed of start-end frame pairs for each shot
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论尽管生成模型的最新进展在视频合成中实现了卓越的视觉保真度,但创建连贯的多镜头叙事仍是一项重大挑战。为此,基于关键帧的方法已成为计算密集型端到端方法的一种有前景的替代方案,具有细粒度控制和更高效率的优势。然而,这些方法往往无法保持跨镜头一致性并捕捉电影语言。本文提出 STAGE,一种故事板锚定的生成工作流,以重构基于关键帧的多镜头视频生成任务。我们不再使用稀疏关键帧,而是提出 STEP^2 来预测由每个镜头的起止帧对组成的结构化故事板。我们引入多镜头记忆包以确保长程实体一致性,采用双编码策略实现镜头内连贯性,并设计两阶段训练方案以学习电影级的镜头间转场。我们还贡献了大规模 ConStoryBoard 数据集,包含高质量电影片段,并带有故事进展、电影属性和人类偏好的细粒度标注。大量实验表明,STAGE 在结构化叙事控制和跨镜头连贯性方面取得了优越性能。我们的代码将在该网址提供。
英文原文摘要 1341 字符
While recent advancements in generative models have achieved remarkable visual fidelity in video synthesis, creating coherent multi-shot narratives remains a significant challenge. To address this, keyframe-based approaches have emerged as a promising alternative to computationally intensive end-to-end methods, offering the advantages of fine-grained control and greater efficiency. However, these methods often fail to maintain cross-shot consistency and capture cinematic language. In this paper, we introduce STAGE, a SToryboard-Anchored GEneration workflow to reformulate the keyframe-based multi-shot video generation task. Instead of using sparse keyframes, we propose STEP^2 to predict a structural storyboard composed of start-end frame pairs for each shot. We introduce the multi-shot memory pack to ensure long-range entity consistency, the dual-encoding strategy for intra-shot coherence, and the two-stage training scheme to learn cinematic inter-shot transition. We also contribute the large-scale ConStoryBoard dataset, including high-quality movie clips with fine-grained annotations for story progression, cinematic attributes, and human preferences. Extensive experiments demonstrate that STAGE achieves superior performance in structured narrative control and cross-shot coherence. Our code will be available at this url.