MagicQuill V2:基于分层视觉线索的精确交互式图像编辑
MagicQuill V2: Precise and Interactive Image Editing with Layered Visual Cues
把单一文本提示拆成内容/空间/结构/颜色四层视觉线索并用统一控制模块注入扩散 Transformer,为需要精细局部控制(含物体移除)的编辑任务提供了可复用的条件注入与数据构造思路。
原文依据 a content layer for what to create, a spatial layer for where to place it, a structural layer for how it is shaped, and a color layer for its palette
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论我们提出 MagicQuill V2,一个将分层组合范式引入生成式图像编辑的新框架,弥合了现代扩散模型的语义能力与传统图形软件的细粒度控制之间的差距。尽管最优的扩散 Transformer 擅长整体生成,但其使用单一、整体的提示,无法解耦用户对内容、位置和风格的不同意图。为克服这一局限,我们的方法将创作意图解构为一组可独立控制的视觉线索:内容层决定创建什么,空间层决定放置在何处,结构层决定其形状,颜色层决定其配色。我们的技术贡献包括:一个用于上下文感知内容集成的专用数据生成管线、一个处理所有视觉线索的统一控制模块,以及一个用于精确局部编辑(包括物体移除)的微调空间分支。大量实验验证,这种分层方法有效弥合了用户意图差距,使创作者能够对生成过程获得直接、直观且强大的控制。
英文原文摘要 1165 字符
We propose MagicQuill V2, a novel framework that introduces a layered composition paradigm to generative image editing, bridging the gap between the semantic power of modern diffusion models and the granular control of traditional graphics software. While state-of-the-art diffusion transformers excel at holistic generation, their use of singular, monolithic prompts fails to disentangle distinct user intentions for content, position, and style. To overcome this limitation, our method deconstructs creative intent into a stack of independently controllable visual cues: a content layer for what to create, a spatial layer for where to place it, a structural layer for how it is shaped, and a color layer for its palette. Our technical contributions include a specialized data generation pipeline for context-aware content integration, a unified control module to process all visual cues, and a fine-tuned spatial branch for precise local editing, including object removal. Extensive experiments validate that this layered approach effectively resolves the user intention gap, granting creators direct, intuitive, and powerful control over the generative process.