BeautyGRPO：通过动态路径引导与细粒度偏好建模实现人脸修图的美学对齐
BeautyGRPO: Aesthetic Alignment for Face Retouching via Dynamic Path Guidance and Fine-Grained Preference Modeling
把在线 RL 的偏好对齐用于人脸修图这类高保真编辑任务时，用基于锚点的 ODE 路径在每个采样步重规划来抑制随机漂移，这个「动态路径引导」思路可直接迁移到其他需要保真度的扩散编辑/复原任务中，避免 RL 探索引入的噪声伪影。
原文依据 DPG stabilizes the stochastic sampling trajectory by dynamically computing an anchor-based ODE path and replanning a guided trajectory at each sampling timestep
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论人脸修图需要在去除细微瑕疵的同时保留独特的面部身份特征，以提升整体美学吸引力。然而，现有方法面临一个根本性的权衡。基于标注数据的监督学习被限制在像素级标签模仿上，无法捕捉复杂的主观人类审美偏好。相反，在线强化学习（RL）擅长偏好对齐，但其随机探索范式与人脸修图的高保真需求相冲突，并且由于累积的随机漂移，常常引入明显的噪声伪影。为解决这些局限，我们提出 BeautyGRPO，一个将人脸修图与人类审美偏好对齐的强化学习框架。我们构建了 FRPref-10K，一个覆盖五个关键修图维度的细粒度偏好数据集，并训练了一个能够评估细微感知差异的专用奖励模型。为协调探索与保真，我们引入动态路径引导（DPG）。DPG 通过动态计算基于锚点的 ODE 路径，并在每个采样时间步重新规划引导轨迹，从而稳定随机采样轨迹，在保持受控探索的同时有效纠正随机漂移。大量实验表明，BeautyGRPO 优于专用人脸修图方法和通用图像编辑模型，实现了更优的纹理质量、更准确的瑕疵去除，以及整体上更符合人类审美偏好的结果。我们的代码将公开。
英文原文摘要 1603 字符
Face retouching requires removing subtle imperfections while preserving unique facial identity features, in order to enhance overall aesthetic appeal. However, existing methods suffer from a fundamental trade-off. Supervised learning on labeled data is constrained to pixel-level label mimicry, failing to capture complex subjective human aesthetic preferences. Conversely, while online reinforcement learning (RL) excels at preference alignment, its stochastic exploration paradigm conflicts with the high-fidelity demands of face retouching and often introduces noticeable noise artifacts due to accumulated stochastic drift. To address these limitations, we propose BeautyGRPO, a reinforcement learning framework that aligns face retouching with human aesthetic preferences. We construct FRPref-10K, a fine-grained preference dataset covering five key retouching dimensions, and train a specialized reward model capable of evaluating subtle perceptual differences. To reconcile exploration and fidelity, we introduce Dynamic Path Guidance (DPG). DPG stabilizes the stochastic sampling trajectory by dynamically computing an anchor-based ODE path and replanning a guided trajectory at each sampling timestep, effectively correcting stochastic drift while maintaining controlled exploration. Extensive experiments show that BeautyGRPO outperforms both specialized face retouching methods and general image editing models, achieving superior texture quality, more accurate blemish removal, and overall results that better align with human aesthetic preferences. Our code will be made publicly available.