视觉语言模型 · CVPR2026 ·2026.06.03

SMAP:基于地图锚定的多模态对齐的语义路线规划

SMAP: Semantic Route Planning with Map-Grounded Multimodal Alignment

Wenjie Zhang、Chen Yang、Xin Lu、Zhen Wang、Yue Liu、Bobo Xi 等 7 人

与你有关的部分

把地图瓦片当作视觉输入喂给多模态大模型,并用「草稿—校验」自编辑加幻觉惩罚 DPO 抑制空间幻觉,这套「视觉证据校验 + 偏好对齐」流程可迁移到任何需要地理/空间落地的视觉-语言任务(如视觉导航、地图问答、具身路径规划)。

原文依据 a map-grounded self-editing pipeline where a multimodal LLM (MLLM) drafts routes and a second MLLM verifies and refines them using geographic evidence
可迁移到 多模态路线规划地图视觉理解视觉语言导航空间推理

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

语义路线规划旨在生成既符合用户意图、又满足真实世界空间约束的行程。然而,纯文本大语言模型(LLM)由于空间锚定能力不足,常常幻觉出在地理上不合理的路线。受人类使用地图进行路线规划的启发,我们提出 SMAP,这是首个结合用户查询、POI 元数据与地图瓦片以生成空间连贯、偏好感知路线的多模态框架。为增强空间一致性,SMAP 采用两阶段抗幻觉机制:(1)地图锚定的自编辑流程,由多模态大模型(MLLM)起草路线,再由第二个 MLLM 借助地理证据进行校验与精炼;(2)幻觉惩罚直接偏好优化(HDPO),以经校验的路线作为被接受回答、以幻觉草稿作为被拒绝回答,引导路线生成器趋向空间上合理的路线。此外,我们提出 MM-Route,这是首个面向语义路线规划的多模态数据集,包含 3,000 条多样化查询,并标注了 POI 元数据与地图瓦片,覆盖广泛的地理粒度与用户意图。实验结果表明,SMAP 显著减少了地理幻觉,并在空间合理性与用户对齐方面优于强基线。项目主页:https://amap-mobility-intelligence.github.io/SMAP/。

英文原文摘要 1438 字符

Semantic route planning involves generating itineraries that align with user intent while respecting real-world spatial constraints. However, text-only large language models (LLMs) often hallucinate geographically implausible routes due to poor spatial grounding. Inspired by how humans use maps for route planning, we propose the SMAP, which is the first multimodal framework combining user queries, POI metadata, and map tiles to produce spatially coherent, preference-aware routes. To enhance the spatial consistency, the SMAP features a two-stage anti-hallucination mechanism: (1) a map-grounded self-editing pipeline where a multimodal LLM (MLLM) drafts routes and a second MLLM verifies and refines them using geographic evidence; and (2) hallucination-penalized Direct Preference Optimization (HDPO) that steers the route generator toward spatially plausible routes by using verified routes as accepted responses and hallucinated drafts as rejected ones. Additionally, we introduce MM-Route, the first multimodal dataset for semantic route planning, with 3,000 diverse queries annotated with POI metadata and map tiles, covering a broad spectrum of geographic granularities and user intents. Experimental results demonstrate that SMAP significantly reduces geographical hallucinations and outperforms strong baselines in spatial plausibility and user alignment. Our project is at https://amap-mobility-intelligence.github.io/SMAP/.

标签

会议 CVPR2026
页码 40108-40118
Bibkey
来源 CVF Open Access