超越基于规则的智能体：面向自动驾驶中真实多智能体交互的主动马尔可夫博弈
Beyond Rule-Based Agents: Active Markov Games for Realistic Multi-Agent Interaction in Autonomous Driving
把驾驶交互建模为状态转移与奖励随智能体策略演化的主动马尔可夫博弈，并用自我车辆策略与多样化对手策略池协同进化训练，可直接用于视觉感知驱动的端到端驾驶策略在非平稳对手下的鲁棒性提升与长尾场景评测。
原文依据 we model the driving environment as an Active Markov Game (AMG) and introduce a multi-agent co-evolutionary training framework
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论当前自动驾驶研究严重依赖大规模驾驶数据集进行模型拟合，或在仿真环境中采用试错式学习策略。然而，这些方法存在行为多样性有限的问题，无法覆盖复杂的边缘案例交互。为解决上述局限，我们将驾驶环境建模为主动马尔可夫博弈（AMG），并引入多智能体协同进化训练框架，以实现更真实的交互式学习。AMG 形式化通过显式地让状态转移与奖励依赖于智能体不断演化的策略，扩展了传统马尔可夫博弈，从而刻画自车与周围智能体之间的交互动态与策略耦合。在此基础上，我们的多智能体协同进化训练机制联合优化自车策略与一个多样化的对手策略池，使所有智能体在训练过程中相互适应彼此行为。这种博弈论方法产生了一个鲁棒的自车智能体，能够应对多样且非平稳的驾驶策略，克服了先前方法中存在的“对手无响应”局限。在 CARLA 仿真的无信号交叉口与长尾场景中，我们的方法表现优异，成功率接近完美（98%），并在泛化性、安全裕度与控制平滑性方面显著优于 PPO、DDPG 与 IPPO 等最优基线。这些结果表明，我们的方法大幅提升了复杂多智能体环境下自动驾驶的鲁棒性、安全性与策略适应性。
英文原文摘要 1668 字符
Current research in autonomous driving heavily relies on large-scale driving datasets for model fitting or trial-and-error learning strategies in simulation environments. However, these approaches suffer from limited behavioral diversity and fail to cover complex edge-case interactions. To address these limitations, we model the driving environment as an Active Markov Game (AMG) and introduce a multi-agent co-evolutionary training framework for more realistic interactive learning. The AMG formulation extends traditional Markov games by explicitly making state transitions and rewards dependent on the evolving strategies of the agents, thus capturing the interactive dynamics and strategic coupling between the ego vehicle and surrounding agents. Building on this, our multi-agent co-evolutionary training mechanism jointly optimizes the ego vehicle's policy and a diverse pool of opponent strategies, allowing all agents to adapt to each other's behaviors during training. This game-theoretic approach produces a robust ego agent capable of handling diverse, non-stationary driving strategies, overcoming the "non-responsive opponent" limitation found in prior methods. In CARLA simulations of unsignalized intersections and long-tail scenarios, our method performs exceptionally well, achieving near-perfect success rates (98%), and significantly outperforming state-of-the-art baselines such as PPO, DDPG, and IPPO in terms of generalization, safety margins, and control smoothness. These results demonstrate that our approach substantially enhances the robustness, safety, and strategic adaptability of autonomous driving in complex multi-agent environments.