语义导数流:用于可控实例交互的图引导扩散
Semantic Derivative Flow: Graph-Guided Diffusion for Controllable Instance Interactions
将实例间的交互关系建模为有向无环图,并通过导数注意力机制强制谓词表征由主语派生、宾语由谓词派生,为可控生成中「多实例交互」这一难点提供了可复用的结构化条件注入思路。
原文依据 a theoretically-motivated derivative attention mechanism, which explicitly enforces the semantic representation of a predicate to be derived from its subject
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论尽管文本到图像扩散模型取得了显著进展,但控制相互交互实例之间的语义与空间关系仍是一项根本性挑战。当前注入空间约束的方法往往无法建模实体之间内在的功能依赖关系,导致生成不合理的交互。本文提出语义导数流(SDF),一种新颖的图引导框架,将有向无环交互图结构引入扩散过程。我们的核心创新是一种具有理论依据的导数注意力机制,它显式地强制谓词的语义表征由其主语派生、宾语由谓词派生,从而形式化了一个可微的语义图。这一具有原则性的方法促使生成过程遵循交互的逻辑链条。我们进一步引入全局上下文节点与实时区域精化模块,以整体地将图锚定在视觉域中。大量实验表明,我们的模型(SDF 的一个实例化)在 HICODet 基准上于保真度与可控性方面确立了新的最优。我们以理论分析补充实证结果,将本方法刻画为交互图上的结构化消息传递,为其有效性与泛化优势提供了严格的依据。
英文原文摘要 1408 字符
Despite remarkable progress in text-to-image diffusion models, controlling the semantic and spatial relationships between interacting instances remains a fundamental challenge. Current methods that inject spatial constraints often fail to model the intrinsic functional dependencies between entities, leading to implausible interactions. In this paper, we introduce Semantic Derivative Flow (SDF), a novel graph-guided framework that structures the diffusion process within a directed acyclic interaction graph. Our core innovation is a theoretically-motivated derivative attention mechanism, which explicitly enforces the semantic representation of a predicate to be derived from its subject, and the object from the predicate, formalizing a differentiable semantic graph. This principled approach compels the generative process to adhere to the logical chain of interaction. We further integrate a global context node and a real-time regional refinement module to ground the graph in the visual domain holistically. Extensive experiments demonstrate that our model, an instantiation of SDF, establishes a new state-of-the-art in fidelity and controllability on the HICODet benchmark. We complement our empirical results with a theoretical analysis, framing our method as structured message passing on interaction graphs, which provides a rigorous justification for its efficacy and generalization benefits.