PlannerRFT：通过闭环与样本高效微调强化扩散规划器
PlannerRFT: Reinforcing Diffusion Planners through Closed-Loop and Sample-Efficient Fine-Tuning
对做扩散策略/规划的研究者，这篇给出一种在推理管线不变的前提下，用双分支优化同时精修轨迹分布并引导去噪探索的强化微调范式，可直接迁移到机器人操作或视觉导航中的扩散策略后训练。
原文依据 PlannerRFT adopts a dual-branch optimization that simultaneously refines the trajectory distribution and adaptively guides the denoising process
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论基于扩散的规划器已成为自动驾驶中类人轨迹生成的一种有前景的方法。近期工作引入强化微调，通过在生成—评估循环中进行面向奖励的优化来增强扩散规划器的鲁棒性。然而，这些方法难以生成多模态、场景自适应的轨迹，从而阻碍了微调过程中信息性奖励的利用效率。为解决这一问题，我们提出 PlannerRFT，一个面向扩散规划器的样本高效强化微调框架。PlannerRFT 采用双分支优化，在精修轨迹分布的同时自适应地引导去噪过程走向更有希望的探索，且不改变原始推理管线。为支持大规模并行学习，我们开发了 nuMax，一个优化后的模拟器，其 rollout 速度相比原生 nuPlan 提升 10 倍。大量实验表明，PlannerRFT 取得了最优性能，并在学习过程中涌现出明显不同的行为。
英文原文摘要 1093 字符
Diffusion-based planners have emerged as a promising approach for human-like trajectory generation in autonomous driving. Recent works incorporate reinforcement fine-tuning to enhance the robustness of diffusion planners through reward-oriented optimization in a generation-evaluation loop. However, they struggle to generate multi-modal, scenario-adaptive trajectories, hindering the exploitation efficiency of informative rewards during fine-tuning. To resolve this, we propose PlannerRFT, a sample-efficient reinforcement fine-tuning framework for diffusion-based planners. PlannerRFT adopts a dual-branch optimization that simultaneously refines the trajectory distribution and adaptively guides the denoising process toward more promising exploration, without altering the original inference pipeline. To support parallel learning at scale, we develop nuMax, an optimized simulator that achieves 10 times faster rollout compared to native nuPlan. Extensive experiments shows that PlannerRFT yields state-of-the-art performance with distinct behaviors emerging during the learning process.