面向3D语义分割多模态训练的图文到点云特征反投影
Image-to-Point Cloud Feature Back-Projection for Multimodal Training of 3D Semantic Segmentation
把图像特征中心按估计深度反投影到点云特征空间，使图像信息在训练期注入点云分支、推理期可关闭，为纯激光雷达部署提供了一种可插拔的多模态训练增强手段。
原文依据 back-projects aggregated image-feature centers (from non-projection-aligned image pixels) into the point-cloud feature set via the estimated depth map
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论有效整合与利用来自图像相机和激光雷达的多模态数据，对感知系统至关重要。本文提出图像到点云特征反投影（IPFP），一种用于训练多模态融合网络的新方法：它借助估计的深度图，将聚合后的图像特征中心（来自未与投影对齐的图像像素）反投影到点云特征集中。由此，图像特征与点云特征位于同一三维空间，使得在网络前向传播过程中能够自然地将图像信息丰富到点云中。该过程可按需选择性启用——例如在训练时启用，而在缺少多模态数据时关闭——例如在测试时仅有激光雷达传感器可用的情况下。实验结果表明，IPFP 能够持续提升最优的3D语义分割模型，同时在推理阶段保留处理纯激光雷达数据的能力。
英文原文摘要 1062 字符
The effective integration and utilization of multimodal data acquired from image cameras and LiDAR is of paramount importance for perception systems. This paper proposes **I**mage-to-**P**oint Cloud **F**eature Back-**P**rojection (**IPFP**), a novel method for training multimodal fusion networks that back-projects aggregated image-feature centers (from non-projection-aligned image pixels) into the point-cloud feature set via the estimated depth map. Consequently, image features and point cloud features reside within the same three-dimensional space, enabling the natural enrichment of image information into the point cloud during the network forward pass. This process can be selectively enabled when desired -- for instance, at training time -- and turned off in the absence of multimodal data -- for example, at testing time if only LiDAR sensors are available. Experimental results demonstrate that **IPFP** can consistently improve state-of-the-art 3D semantic segmentation models, while retaining the ability to process LiDAR-only data at inference.