CycleBEV:通过视图循环一致性正则化视图变换网络用于鸟瞰图语义分割
CycleBEV: Regularizing View Transformation Networks via View Cycle Consistency for Bird's-Eye-View Semantic Segmentation
提出一种仅训练期使用的逆视图变换(IVT)网络与视图循环一致性损失,可作为即插即用正则化手段提升任意 PV→BEV 视图变换模型的语义分割精度,且不增加推理开销;其将循环一致性扩展到几何空间与表征空间的思路,可迁移到其他跨视角表征学习任务。
原文依据 we devise an inverse view transformation (IVT) network that maps BEV segmentation maps back to PV segmentation maps
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论由于深度歧义与遮挡,将图像特征从透视视图(PV)空间变换到鸟瞰图(BEV)空间在自动驾驶中仍具挑战性。尽管已提出多种视图变换(VT)范式,该挑战依然存在。本文提出一种新的正则化框架 CycleBEV,用于增强现有 VT 模型以完成 BEV 语义分割。受广泛用于图像分布建模的循环一致性启发,我们设计了一个逆视图变换(IVT)网络,将 BEV 分割图映射回 PV 分割图,并在训练期间通过循环一致性损失用它正则化 VT 网络,使其能够从输入 PV 图像中捕获更丰富的语义与几何信息。为进一步发挥 IVT 网络的能力,我们引入两个新思路,将循环一致性扩展到几何空间与表征空间。我们在大规模 nuScenes 数据集上对覆盖三大范式的四个代表性 VT 模型评估 CycleBEV。实验结果表明性能获得一致提升——可行驶区域、车辆与行人类别分别最高提升 0.74、4.86 与 3.74 平均交并比——且不增加推理复杂度,因为 IVT 网络仅在训练时使用。
英文原文摘要 1314 字符
Transforming image features from perspective view (PV) space to bird's-eye-view (BEV) space remains challenging in autonomous driving due to depth ambiguity and occlusion. Although several view transformation (VT) paradigms have been proposed, the challenge still remains. In this paper, we propose a new regularization framework, dubbed CycleBEV, that enhances existing VT models for BEV semantic segmentation. Inspired by cycle consistency, widely used in image distribution modeling, we devise an inverse view transformation (IVT) network that maps BEV segmentation maps back to PV segmentation maps and use it to regularize VT networks during training through cycle consistency losses, enabling them to capture richer semantic and geometric information from input PV images. To further exploit the capacity of the IVT network, we introduce two novel ideas that extend cycle consistency into geometric and representation spaces. We evaluate CycleBEV on four representative VT models covering three major paradigms using the large-scale nuScenes dataset. Experimental results show consistent improvements---with gains of up to 0.74, 4.86, and 3.74 mIoU for drivable area, vehicle, and pedestrian classes, respectively---without increasing inference complexity, since the IVT network is used only during training.