Scone:通过统一理解-生成建模弥合主体驱动图像生成中的组合与区分
Scone: Bridging Composition and Distinction in Subject-Driven Image Generation via Unified Understanding-Generation Modeling
把理解专家当作语义桥梁,向生成专家传递语义并配合注意力掩码抑制主体间干扰,为多主体个性化生成中「选对主体」的问题提供了可复用的两阶段训练与掩码引导思路。
原文依据 Scone enables the understanding expert to act as a semantic bridge, conveying semantic information and guiding the generation expert
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论主体驱动图像生成已从单主体发展到多主体组合,但忽视了区分能力,即在输入包含多个候选主体时辨别并生成正确主体的能力。这一局限制约了其在复杂、真实视觉场景中的有效性。我们提出 Scone,一种统一理解-生成的方法,将组合与区分融为一体。Scone 使理解专家充当语义桥梁,传递语义信息并引导生成专家在保持主体身份的同时将干扰降至最低。两阶段训练方案先学习组合,再通过语义对齐与基于注意力的掩码增强区分能力。我们还提出 SconeEval,一个用于在多样场景下评估组合与区分能力的基准。实验表明,Scone 在两个基准上的组合与区分任务中优于现有开源模型。我们的模型、基准与训练数据见:https://github.com/Ryann-Ran/Scone。
英文原文摘要 1071 字符
Subject-driven image generation has advanced from single- to multi-subject composition, while neglecting distinction, the ability to distinguish and generate the correct subject when inputs contain multiple candidates. This limitation restricts effectiveness in complex, realistic visual settings. We propose Scone, a unified understanding-generation method that integrates composition and distinction. Scone enables the understanding expert to act as a semantic bridge, conveying semantic information and guiding the generation expert to preserve subject identity while minimizing interference. A two-stage training scheme first learns composition, then enhances distinction through semantic alignment and attention-based masking. We also introduce SconeEval, a benchmark for evaluating both composition and distinction across diverse scenarios. Experiments demonstrate that Scone outperforms existing open-source models in composition and distinction tasks on two benchmarks. Our model, benchmark, and training data are available at: https://github.com/Ryann-Ran/Scone.