基于解耦奖励建模的统一定制化生成
Unified Customized Generation by Disentangled Reward Modeling
把主体定制与风格定制统一到一个框架，并用辅助风格奖励对齐风格与内容特征，为个性化生成中「参考图多属性解耦」提供了可复用的奖励建模思路，可直接迁移到其他条件生成任务的多条件控制。
原文依据 introduces an auxiliary style reward to simultaneously align style and content features
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论现有文献通常将各类定制化生成任务（例如主体定制化生成、风格定制化生成）视为彼此独立、互不相干的问题，每个任务只专注于定制参考图像的某一个特定方面。然而，我们认为这些不同定制化任务的目标本质上是互补的，可以在一个统一框架内相互增强，因为它们从根本上都涉及对参考图像多个特征方面的解耦。为此，我们提出 USO，一个统一同时优化框架，以同时统一不同的定制化任务（即主体与风格）。具体而言，USO 引入一个循环式数据—模型框架，通过一条「以主体换风格」的数据构造流水线和一条「以风格换主体」的模型训练流水线将这两个任务连接起来。「以主体换风格」的数据构造流水线利用一个最优的主体定制化模型生成高质量的三元组数据，包括内容图像、风格图像及其对应的风格化内容图像。在此基础上，「以风格换主体」的模型训练流水线引入一个辅助风格奖励，以同时对齐风格特征与内容特征，从而强化模型从参考图像中提取所需风格特征或内容特征的能力。大量实验表明，USO 在开源模型中达到最优性能，在主体一致性与风格相似度两方面均表现优异。代码与模型：https://github.com/bytedance/USO。
英文原文摘要 1593 字符
Existing literature typically treats various customized generation tasks (e.g., subject-customized generation, style-customized generation) as distinct and disjoint problems, with each task focusing solely on customizing a specific aspect of the reference image. However, we argue that the objectives of these different customization tasks are inherently complementary and can be mutually enhanced within a unified framework, as they fundamentally involve the disentanglement of multiple feature aspects from the reference image. To this end, we introduce USO, a Unified Simultaneous Optimization framework to simultaneously unify different customized tasks (i.e., subject and style). Specifically, USO introduces a cyclical data-model framework that connects these two tasks by a subject-for-style data curation pipeline and a style-for-subject model training pipeline. The subject-for-style data curation pipeline leverages a state-of-the-art subject-customized model to generate high-quality triplet data comprising content images, style images, and their corresponding stylized content images. Building on this foundation, the style-for-subject model training pipeline introduces an auxiliary style reward to simultaneously align style and content features, thereby reinforcing the model's ability to extract the desired style or content features from the reference image. Extensive experiments demonstrate that USO achieves state-of-the-art performance among open-source models, excelling in both subject consistency and style similarity. Code and model: https://github.com/bytedance/USO.