自动驾驶与遥感 · CVPR2026 ·2026.06.03

GEM:基于可变形 Mamba 的激光雷达世界模型生成

GEM: Generating LiDAR World Model via Deformable Mamba

Yang Wu、Zhaojiang Liu、Qiang Meng、Youquan Liu、Renliang Weng、Jianjun Qian 等 8 人

与你有关的部分

把序列激光扫描与 Mamba 的选择性扫描机制对齐,并用三路可变形 Mamba 在解耦后的动态/静态 token 上做时空建模,为激光雷达点云生成式世界模型提供了一个可复用的骨干设计;其 BEV 布局控制器与规划器接口也便于把生成模型接入自动驾驶闭环仿真。

原文依据 a tri-path deformable Mamba is introduced to perform selective scanning and adaptive gating fusion over the disentangled features
可迁移到 激光雷达点云生成世界模型自动驾驶仿真鸟瞰图布局生成

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

世界模型通过模拟环境动态并生成传感器观测,在自动驾驶领域正受到越来越多的关注。然而,基于激光雷达的世界模型进展落后于基于相机视频或占用数据的同类工作,主要原因是两个核心挑战:激光雷达点云固有的无序性,以及区分动态物体与静态结构的困难。为解决这些问题,我们提出 GEM:一种利用可变形 Mamba 架构的生成式激光雷达世界模型,显著提升了保真度与想象能力。具体而言,利用序列激光扫描与 Mamba 处理机制之间的结构相似性,我们首先通过定制的激光雷达场景分词器将激光雷达扫描帧分词为紧凑表示。在通过动态-静态分离器对分词特征进行无监督解耦之后,引入三路可变形 Mamba 对解耦后的特征执行选择性扫描与自适应门控融合,从而增强对世界演化的时空理解。可选地,可集成规划器与鸟瞰图布局控制器,以探索模型自主推演的能力及其生成“假设”场景的潜力。大量实验表明,GEM 在多种基准与评测设置下均取得最优性能,证明了其优越性与有效性。项目主页:https://github.com/wuyang98/GEM 。

英文原文摘要 1500 字符

World models, which simulate environmental dynamics and generate sensor observations, are gaining increasing attention in autonomous driving. However, progress in LiDAR-based world models has lagged behind those built on camera videos or occupancy data, primarily due to two core challenges: the inherent disorder of LiDAR point clouds and the difficulty of distinguishing dynamic objects from static structures. To address these issues, we propose GEM: a Generative LiDAR world model that leverages dEformable Mamba architecture, significantly improving fidelity and imaginative capability. Specifically, leveraging the structural similarity between sequential laser scanning and Mamba's processing mechanism, we first tokenize LiDAR sweeps into compact representations via a custom LiDAR scene tokenizer. After unsupervised disentanglement of tokenized features via a dynamic-static separator, a tri-path deformable Mamba is introduced to perform selective scanning and adaptive gating fusion over the disentangled features, leading to enhanced spatial-temporal understanding of the world evolution. Optionally, a planner and a BEV layout controller can be integrated to explore the model's capability for autonomous rollout and its potential to generate "what-if" scenarios. Extensive experiments show that GEM achieves state-of-the-art performances across diverse benchmarks and evaluation settings, demonstrating its superiority and effectiveness. Project page: https://github.com/wuyang98/GEM .

标签

会议 CVPR2026
页码 24227-24236
Bibkey
来源 CVF Open Access