DynFusion:重新思考自适应多条件文本到图像生成中的条件融合
DynFusion: Rethinking Condition Fusion for Adaptive Multi-Conditional Text-to-Image Generation
提出一种数据驱动的自适应条件融合机制,根据扩散时间步、任务特性和特征注入位置动态选择并融合多个异构条件(如深度、边缘、参考图),可解决多条件控制中的冲突与计算冗余问题,为可控生成提供可复用的融合模块。
原文依据 We propose a data-driven adaptive condition fusion mechanism for multi-conditional diffusion.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论文本到图像扩散模型已取得显著进展,能够根据文本提示生成视觉逼真且语义连贯的图像。然而,自然语言本身缺乏设计类应用所需的精确性,这些应用要求严格的空间和结构保真度——尤其是在表示融合多层级信息的复杂概念时,例如产品或场景设计。为解决这一局限,可控扩散框架引入辅助条件(如深度图、边缘图或参考图像)来引导生成过程。ControlNet 和 IP-Adapter 等模型有效注入了此类先验,改善了结构或外观对齐。然而,现实世界中的设计任务很少仅依赖单一类型的条件。它们通常需要同时整合多种异构线索——例如,从深度图保留空间布局,从边缘图保留结构轮廓,从参考图像保留风格属性。当前方法要么仅处理单一条件,要么简单地堆叠多个条件,导致计算效率低下和引导冲突,从而降低生成质量。这种多条件不一致性构成了将扩散模型应用于实际设计工作流的关键瓶颈,促使我们提出该框架。我们提出了一种数据驱动的自适应条件融合机制,用于多条件扩散。该方法引入了一种新颖的条件自适应模块,能够根据扩散时间步、任务特性和特征注入位置动态选择并融合条件子集。这种自适应策略协调了多样的结构和外观先验,在复杂设计场景中实现了可控而灵活的生成。实验表明,在多条件任务中,该方法在保真度、一致性和可控性方面均有显著提升,为实际、细节保留的基于扩散的设计生成开辟了新方向。
英文原文摘要 2002 字符
Text-to-image diffusion models have achieved remarkable progress, generating visually realistic and semantically coherent images from textual prompts. However, natural language alone lacks the precision required for design-centric applications that demand strict spatial and structural fidelity--particularly when representing complex concepts that integrate multi-level information, such as product or scene design. To address this limitation, controllable diffusion frameworks introduce auxiliary conditions (e.g., depth, edge, or reference images) to guide the generative process. Models like ControlNet and IP-Adapter effectively inject such priors, improving structural or appearance alignment. Yet, real-world design tasks rarely depend on a single type of condition. They often require simultaneous integration of multiple heterogeneous cues--for instance, preserving spatial layout from depth maps, structural outlines from edge maps, and stylistic attributes from reference images. Current approaches either handle only one condition or naively stack multiple ones, resulting in computational inefficiency and conflicting guidance that degrade generation quality. This multi-condition inconsistency forms a critical bottleneck for applying diffusion models to real-world design workflows, motivating our proposed framework. We propose a data-driven adaptive condition fusion mechanism for multi-conditional diffusion. Our method introduces a novel condition adaptation module that dynamically selects and fuses subsets of conditions based on the diffusion timestep, task characteristics, and feature injection position. This adaptive strategy harmonizes diverse structural and appearance priors, achieving controllable yet flexible generation in complex design scenarios. Experiments demonstrate significant improvements in fidelity, consistency, and controllability across multi-condition tasks, establishing a new direction for practical, detail-preserving diffusion-based design generation.