MindDriver：面向自动驾驶的渐进式多模态推理
MindDriver: Introducing Progressive Multimodal Reasoning for Autonomous Driving
将VLM的思维链从纯文本扩展到语义理解→语义到物理空间想象→物理空间轨迹规划的多模态渐进推理，并用反馈引导的自动标注和渐进强化微调对齐推理过程，为端到端自动驾驶提供可复用的推理框架。
原文依据 MindDriver presents semantic understanding, semantic-to-physical space imagination, and physical-space trajectory planning.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉语言模型（VLM）展现出强大的推理能力，为端到端自动驾驶系统带来了希望。思维链（CoT）作为VLM广泛使用的推理策略，正面临严峻挑战。现有文本思维链在文本语义空间与轨迹物理空间之间存在巨大鸿沟。尽管近期方法利用未来图像替代文本作为思维链过程，但其缺乏明确的面向规划的目标引导来生成具有准确场景演化的图像。为解决这些问题，我们创新性地提出MindDriver，一个渐进式多模态推理框架，使VLM能够模仿人类渐进式思维进行自动驾驶。MindDriver呈现语义理解、语义到物理空间想象以及物理空间轨迹规划。为实现MindDriver中对齐的推理过程，我们开发了一个反馈引导的自动数据标注流程，以生成对齐的多模态推理训练数据。此外，我们开发了一种渐进式强化微调方法，通过渐进式高层奖励学习来优化对齐。MindDriver在nuScenes开环和Bench2Drive闭环评估中均展现出优越性能。代码：https://github.com/hotdogcheesewhite/MindDriver。
英文原文摘要 1300 字符
Vision-Language Models (VLM) exhibit strong reasoning capabilities, showing promise for end-to-end autonomous driving systems. Chain-of-Thought (CoT), as VLM's widely used reasoning strategy, is facing critical challenges. Existing textual CoT has a large gap between text semantic space and trajectory physical space. Although the recent approach utilizes future image to replace text as CoT process, it lacks clear planning-oriented objective guidance to generate images with accurate scene evolution. To address these, we innovatively propose MindDriver, a progressive multimodal reasoning framework that enables VLM to imitate human-like progressive thinking for autonomous driving.MindDriver presents semantic understanding, semantic-to-physical space imagination, and physical-space trajectory planning.To achieve aligned reasoning processes in MindDriver, we develop a feedback-guided automatic data annotation pipeline to generate aligned multimodal reasoning training data. Furthermore, we develop a progressive reinforcement fine-tuningmethod to optimize the alignment through progressive high-level reward-based learning.MindDriver demonstrates superior performance in both nuScences open-loop and Bench2Drive closed-loop evaluation. Codes: https://github.com/hotdogcheesewhite/MindDriver.