通过强化学习精炼少步文本到多视图扩散模型
Refining Few-Step Text-to-Multiview Diffusion via Reinforcement Learning
把少步 T2MV 扩散的 RL 微调建模为联合多视图 MDP，并用自精炼采样构造优势信号、用拉格朗日对偶平衡单视图保真与跨视图一致性，可直接迁移到任何需要多输出协同一致性的少步扩散/流模型微调场景。
原文依据 a new MDP formulation that jointly models all generated views and assesses their collective quality via a joint-view reward
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论文本到多视图（T2MV）扩散模型在从单一文本提示生成场景的多个视图方面已展现出巨大潜力。尽管少步主干网络能够实现实时的 T2MV 生成，但它们往往在生成质量的关键方面有所妥协，例如单视图保真度和跨视图一致性。强化学习（RL）微调提供了一种潜在的解决方案，然而现有为单图像扩散设计的方法难以直接扩展到少步 T2MV 设定，因为它们忽略了跨视图协调，并且在少步情形下学习信号微弱。为解决这一问题，我们提出 MVC-ZigAL，一个为少步 T2MV 扩散模型量身定制的 RL 微调框架。具体而言，其核心思想是：（1）一种新的马尔可夫决策过程（MDP）建模方式，联合建模所有生成的视图，并通过联合视图奖励评估它们的整体质量；（2）一种新颖的优势学习策略，利用自精炼采样方案相对于标准采样所带来的性能增益，从而产生更强的学习信号以实现有效的 RL 微调；（3）一个统一的 RL 框架，将优势学习与拉格朗日对偶形式相结合以进行多视图约束优化，在自定步调的阈值课程下通过自适应原始-对偶更新来平衡单视图与联合视图目标，从而协调探索与约束执行。总体而言，这些设计使少步 T2MV 扩散模型能够进行稳健且均衡的 RL 微调，在单视图保真度和跨视图一致性两方面均取得显著提升。代码见 https://github.com/ZiyiZhang27/MVC-ZigAL。
英文原文摘要 1627 字符
Text-to-multiview (T2MV) diffusion models have shown great promise in generating multiple views of a scene from a single text prompt. While few-step backbones enable real-time T2MV generation, they often compromise key aspects of generation quality, such as per-view fidelity and cross-view consistency. Reinforcement learning (RL) finetuning offers a potential solution, yet existing approaches designed for single-image diffusion do not readily extend to the few-step T2MV setting, as they neglect cross-view coordination and suffer from weak learning signals in few-step regimes. To address this, we propose MVC-ZigAL, a tailored RL finetuning framework for few-step T2MV diffusion models. Specifically, its core insights are: (1) a new MDP formulation that jointly models all generated views and assesses their collective quality via a joint-view reward; (2) a novel advantage learning strategy that exploits the performance gains of a self-refinement sampling scheme over standard sampling, yielding stronger learning signals for effective RL finetuning; and (3) a unified RL framework that extends advantage learning with a Lagrangian dual formulation for multiview-constrained optimization, balancing single-view and joint-view objectives through adaptive primal-dual updates under a self-paced threshold curriculum that harmonizes exploration and constraint enforcement. Collectively, these designs enable robust and balanced RL finetuning for few-step T2MV diffusion models, yielding substantial gains in both per-view fidelity and cross-view consistency. Code is available at https://github.com/ZiyiZhang27/MVC-ZigAL.