StyleDoctor：面向以风格为中心的生成任务的专家奖励模型
StyleDoctor: Towards Specialist Reward Model for Style-centric Generation Tasks
为风格化扩散生成提供可复用的风格感知奖励模型，可直接接入 RLHF/奖励微调流程，替代通用人类偏好奖励模型来提升风格一致性与文图对齐。
原文依据 existing human-preference reward models struggle to effectively perceive image style
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论风格生成借助扩散模型已取得显著进展。近期工作探索了利用人类偏好奖励模型进行强化学习，以增强扩散模型在通用下游应用中的表现。然而，我们发现了一个关键局限：现有的人类偏好奖励模型难以有效感知图像风格，导致强化微调后的性能欠佳。为解决这一问题，我们首先构建了一个大规模风格奖励建模数据集，包含 40 万个配对样本，覆盖 1000 个多样化的风格类别，并配有文本指令与风格奖励标注。随后，我们提出了 StyleDoctor，一种新颖的风格感知奖励模型，能够联合评估配对图像之间的风格一致性以及风格与文本的对齐程度。StyleDoctor 在风格检索与生成任务上均优于现有的风格感知模型。大量定量与定性实验表明，StyleDoctor 优于对比方法，展现了其在风格条件生成中的高效性与通用性。数据集与代码见 StyleDoctor。
英文原文摘要 1151 字符
Style generation has made significant progress through diffusion models. Recent efforts have explored reinforcement learning with human-preference reward models to enhance diffusion models for general downstream applications. However, we identify a critical limitation: existing human-preference reward models struggle to effectively perceive image style, resulting in suboptimal performance after reinforcement fine-tuning. To address this, we first introduce a large-scale style reward modeling dataset comprising 400K paired samples spanning 1,000 diverse style categories, augmented with textual instructions and style reward annotations. We then propose StyleDoctor, a novel style perception reward model capable of jointly evaluating style consistency between paired images and style-text alignment. StyleDoctor outperforms existing style perception models in both style retrieval and generation tasks. Extensive quantitative and qualitative experiments demonstrate the superiority of StyleDoctor over competing approaches, showcasing its efficiency and versatility in style-conditioned generation. Dataset and code are available at StyleDoctor.