OSPO：面向文本到图像生成的以物体为中心的自改进偏好优化
OSPO: Object-Centric Self-Improving Preference Optimization for Text-to-Image Generation
把偏好优化从整图级细化到物体级：用注意力图导出物体掩码，构造物体级偏好对并施加物体加权 SimPO 损失，可直接迁移到任何统一多模态理解-生成模型的细粒度对齐后训练中。
原文依据 leverages attention-based object masks together with an object-weighted SimPO loss to enhance object-specific fidelity
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多模态大语言模型（MLLM）的最新进展使统一的多模态理解与生成成为可能。然而，它们在细粒度文本-图像对齐方面仍存在困难，往往无法忠实描绘具有正确属性（如颜色、形状和空间关系）的物体。为缓解这一问题，已有研究探索了 DPO 与 GRPO 等偏好优化方法，但这些方法在构建偏好数据和执行优化两方面都会带来可观的计算开销。这促使了自改进偏好优化方法的出现：MLLM 自主生成自身的训练数据、自估计偏好反馈，并利用由此自构建的偏好对进行自优化。然而，现有的自改进方法仍忽略了细粒度的物体级语义，使得物体幻觉持续存在。为解决该问题，我们提出以物体为中心的自改进偏好优化（OSPO），这是一个旨在增强物体级文本-图像对齐的自改进框架。OSPO 在不依赖任何外部数据或模型的情况下显式构建以物体为中心的偏好数据。我们还引入了一种新方法，利用基于注意力的物体掩码并结合物体加权 SimPO 损失，以增强物体特定的保真度。在三个组合式图像生成基准上的大量实验表明，OSPO 显著提升了细粒度对齐并减少了物体幻觉，优于先前的自改进方法，甚至优于专门的基于扩散的文本到图像模型。
英文原文摘要 1643 字符
Recent advances in Multimodal Large Language Models (MLLMs) have enabled unified multimodal understanding and generation. However, they still struggle with fine-grained text-image alignment, often failing to faithfully depict objects with correct attributes such as color, shape, and spatial relationships. To mitigate this issue, previous studies have explored preference optimization methods such as DPO and GRPO, but these approaches incur substantial computational cost, both in constructing preference data and in performing optimization. This has motivated self-improving preference optimization approaches, in which the MLLM autonomously generates its own training data, self-estimates preference feedback, and self-optimizes using the resulting self-constructed preference pairs. However, existing self-improving methods still overlook fine-grained, object-level semantics, allowing object hallucination to persist. To tackle this problem, we propose Object-centric Self-improving Preference Optimization (OSPO), a self-improving framework designed to enhance object-level text-image alignment. OSPO explicitly constructs object-centric preference data without relying on any external data or models. We also introduce a new approach that leverages attention-based object masks together with an object-weighted SimPO loss to enhance object-specific fidelity. Extensive experiments on three compositional image generation benchmarks demonstrate that OSPO significantly improves fine-grained alignment and reduces object hallucination, outperforming prior self-improving methods and even specialized diffusion-based text-to-image models.