基于区域感知双模态直接偏好优化的组合式文本到图像生成
Compositional Text-to-Image Generation Via Region-aware Bimodal Direct Preference Optimization
提出区域感知的双模态直接偏好优化（BIDPO），可复用于提升扩散模型对复杂组合提示词的遵循能力，尤其是属性绑定、对象关系和计数等细粒度对齐任务。
原文依据 we extend Diffusion DPO to jointly optimize image and text preferences
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论尽管文本到图像（T2I）模型取得了快速进展，但生成能够准确反映复杂组合提示词（涵盖属性绑定、对象关系和计数）的图像仍然具有挑战性。为此，我们提出了 BIDPO，一个旨在增强 T2I 模型组合式文本到图像生成能力的框架。我们首先引入一个精心设计的流程来构建大规模偏好数据集 BICOMP，并进行了严格的质量控制。然后，我们扩展了 Diffusion DPO 以联合优化图像和文本偏好，实验表明这在提升模型生成时遵循复杂文本提示词方面非常有效。为了进一步增强模型的细粒度对齐能力，我们采用了一种区域级引导方法，以聚焦于与组合概念相关的区域。实验结果表明，我们的 BIDPO 显著提升了组合保真度，在多个基准测试上持续优于先前方法。我们的方法凸显了基于偏好的微调在复杂文本到图像任务中的潜力，为现有技术提供了一种灵活且可扩展的替代方案。代码可在 https://github.com/anzeameol/BiDPO 获取。
英文原文摘要 1214 字符
Despite the rapid progress of text-to-image (T2I) models, generating images that accurately reflect complex compositional prompts (covering attribute bindings, object relationships, counting) still remains challenging. To address this, we propose BIDPO, a framework to enhance T2I model's capability of compositional text-to-image generation. We begin by introducing an carefully designed pipeline to construct a large-scale preference dataset, BICOMP, with strictly quality control. Then, we extend Diffusion DPO to jointly optimize image and text preferences, which is shown to greatly effective in improving the models to follow complex text prompt in generation. To further enhance the models for fine-grained alignment, we employ a region-level guidance method to focus on regions relevant to compositional concepts. Experimental results demonstrate that our BIDPO substantially improves compositional fidelity, consistently outperforming prior methods across multiple benchmarks. Our approach highlights the potential of preference-based fine-tuning for complex text-to-image tasks, offering a flexible and scalable alternative to existing techniques. Code is available at https://github.com/anzeameol/BiDPO.