通过概念-提示词绑定组合来自图像和视频的概念
Composing Concepts from Images and Videos via Concept-prompt Binding
提出通过绑定视觉概念与提示词词元，在扩散 Transformer 的交叉注意力中实现单样本概念组合，可复用于个性化生成与视频编辑中的概念解耦控制。
原文依据 a one-shot method that enables flexible visual concept composition by binding visual concepts with corresponding prompt tokens
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉概念组合旨在将来自图像和视频的不同元素整合为单一、连贯的视觉输出，但现有方法在从视觉输入中准确提取复杂概念以及灵活组合来自图像和视频的概念方面仍存在不足。我们提出 Bind & Compose，一种单样本方法，通过将视觉概念与对应的提示词词元绑定，并利用来自不同来源的绑定词元组合目标提示词，实现灵活的视觉概念组合。该方法采用分层绑定器结构，在扩散 Transformer 中进行交叉注意力条件化，将视觉概念编码为对应的提示词词元，从而准确分解复杂视觉概念。为提高概念-词元绑定准确性，我们设计了多样化与吸收机制，使用额外的吸收词元，在使用多样化提示词训练时消除概念无关细节的影响。为增强图像与视频概念之间的兼容性，我们提出时序解耦策略，将视频概念的训练过程解耦为两个阶段，并采用双分支绑定器结构进行时序建模。评估表明，我们的方法在概念一致性、提示词保真度和运动质量方面优于现有方法，为视觉创意开辟了新的可能性。
英文原文摘要 1362 字符
Visual concept composition, which aims to integrate different elements from images and videos into a single, coherent visual output, still falls short in accurately extracting complex concepts from visual inputs and flexibly combining concepts from both images and videos. We introduce Bind & Compose, a one-shot method that enables flexible visual concept composition by binding visual concepts with corresponding prompt tokens and composing the target prompt with bound tokens from various sources. It adopts a hierarchical binder structure for cross-attention conditioning in Diffusion Transformers to encode visual concepts into corresponding prompt tokens for accurate decomposition of complex visual concepts. To improve concept-token binding accuracy, we design a Diversify-and-Absorb Mechanism that uses an extra absorbent token to eliminate the impact of concept-irrelevant details when training with diversified prompts. To enhance the compatibility between image and video concepts, we present a Temporal Disentanglement Strategy that decouples the training process of video concepts into two stages with a dual-branch binder structure for temporal modeling. Evaluations demonstrate that our method achieves superior concept consistency, prompt fidelity, and motion quality over existing approaches, opening up new possibilities for visual creativity.