DreamingComics：基于视频模型的主体与布局定制化故事可视化流程
DreamingComics: A Story Visualization Pipeline via Subject and Layout Customized Generation using Video Models
RegionalRoPE 按目标布局对 token 位置索引重编码，配合 masked condition loss 把主体特征约束到指定区域，这套「区域感知位置编码 + 掩码条件损失」可直接迁移到任何基于 DiT 的布局可控图像/视频生成任务。
原文依据 we propose RegionalRoPE, a region-aware positional encoding scheme that re-indexes embeddings based on the target layout
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论现有故事可视化方法倾向于仅通过文本定位主体，并在保持艺术一致性方面面临挑战。针对这些局限，我们提出 DreamingComics，一个布局感知的故事可视化框架。我们以预训练视频扩散 Transformer（DiT）模型为基础，利用其时空先验增强身份与风格一致性。为实现基于布局的位置控制，我们提出 RegionalRoPE，一种区域感知的位置编码方案，依据目标布局对嵌入重新索引。此外，我们引入掩码条件损失，进一步将每个主体的视觉特征约束到其指定区域。为从自然语言脚本推断布局，我们集成一个基于 LLM 的布局生成器，并将其训练为生成漫画风格布局，从而实现灵活且可控的布局条件控制。我们对该方法进行了全面评估，结果显示，与先前方法相比，角色一致性提升 29.2%，风格相似度提升 36.2%，同时展现出较高的空间准确度。
英文原文摘要 1075 字符
Current story visualization methods tend to position subjects solely by text and face challenges in maintaining artistic consistency. To address these limitations, we introduce DreamingComics, a layout-aware story visualization framework. We build upon a pretrained video diffusion-transformer (DiT) model, leveraging its spatiotemporal priors to enhance identity and style consistency. For layout-based position control, we propose RegionalRoPE, a region-aware positional encoding scheme that re-indexes embeddings based on the target layout. Additionally, we introduce a masked condition loss to further constrain each subject's visual features to their designated region. To infer layouts from natural language scripts, we integrate an LLM-based layout generator trained to produce comic-style layouts, enabling flexible and controllable layout conditioning. We present a comprehensive evaluation of our approach, showing a 29.2% increase in character consistency and 36.2% increase in style similarity compared to previous methods, while displaying high spatial accuracy.