RLFTSim：通过强化学习微调实现真实且可控的多智能体交通仿真
RLFTSim: Realistic and Controllable Multi-Agent Traffic Simulation via Reinforcement Learning Fine-Tuning
对做自动驾驶仿真与轨迹预测的研究者，这篇给出了用强化学习微调替代开环监督训练、以低方差稠密奖励把仿真 rollout 分布对齐真实数据的具体做法，可直接迁移到自车规划器的闭环评测与可控场景生成。
原文依据 a reinforcement-learning-based fine-tuning framework that enhances scenario realism by aligning simulator rollouts with real-world data distributions
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论监督式开环训练已被广泛用于训练交通仿真模型；然而，它无法捕捉复杂驾驶场景中固有的动态多智能体交互。我们提出 RLFTSim，一个基于强化学习的微调框架，通过使仿真器 rollout 与真实世界数据分布对齐来提升场景真实性，并提供一种在场景生成中蒸馏目标条件可控性的方法。我们在一个预训练仿真模型之上实例化 RLFTSim，设计了平衡保真度与可控性的奖励，并在 Waymo Open Motion Dataset 上开展了全面实验。结果表明真实性得到提升，达到最优性能。与其他基于启发式搜索的微调方法相比，RLFTSim 由于所提出的低方差稠密奖励信号而所需样本显著更少，并且它在设计上直接解决了真实性对齐问题。我们还证明了该方法通过目标条件蒸馏交通仿真可控性的有效性。项目页面见 https://ehsan-ami.github.io/rlftsim。
英文原文摘要 1180 字符
Supervised open-loop training has been widely adopted for training traffic simulation models; however, it fails to capture the inherently dynamic, multi-agent interactions common in complex driving scenarios. We introduce RLFTSim, a reinforcement-learning-based fine-tuning framework that enhances scenario realism by aligning simulator rollouts with real-world data distributions and provides a method for distilling goal-conditioned controllability in scenario generation. We instantiate RLFTSim on top of a pre-trained simulation model, design a reward that balances fidelity and controllability, and perform comprehensive experiments on the Waymo Open Motion Dataset. Our results show improvements in realism, achieving state-of-the-art performance. Compared with other heuristic search-based fine-tuning methods, RLFTSim requires significantly fewer samples due to a proposed low-variance and dense reward signal, and it directly addresses the realism alignment issue by design. We also demonstrate the effectiveness of our approach for distilling traffic simulation controllability through goal conditioning. Project page is available at https://ehsan-ami.github.io/rlftsim.