Sensor2Sensor：面向自动驾驶的跨具身传感器转换
Sensor2Sensor: Cross-Embodiment Sensor Conversion for Autonomous Driving
用4D高斯泼溅把真实自动驾驶日志反向渲染成行车记录仪风格视频，从而构造配对数据，再用扩散模型完成单目视频到多视角相机加激光雷达的跨模态生成；这套「先合成配对数据、再做跨传感器转换」的流程，可直接复用于缺失模态补全与多传感器数据增广。
原文依据 converting real AV logs into dashcam-style videos via 4D Gaussian Splatting (4DGS) reconstruction and novel-view rendering
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论自动驾驶系统（ADS）的鲁棒训练与验证需要大规模、多样化的数据集。自动驾驶车辆（AV）车队采集的专有数据虽然保真度高，但在规模、传感器配置的多样性以及地理与长尾行为覆盖方面均存在局限。相比之下，来自行车记录仪等来源的自然场景数据具有极大的规模与多样性，能够捕捉关键的长尾场景与新颖环境。然而，这类非结构化的自然场景视频数据与期望结构化、多模态传感器输入以进行验证和训练的 ADS 并不兼容。为弥合这一数据鸿沟，我们提出 Sensor2Sensor，一种新的生成建模范式，可将自然场景下的单目行车记录仪视频转换为高保真、多模态的传感器套件（AV 日志），其中包含多视角相机图像与激光雷达点云。核心挑战在于缺乏配对训练数据。我们通过 4D 高斯泼溅（4DGS）重建与新视角渲染，将真实 AV 日志转换为行车记录仪风格的视频，以此解决该问题。随后 Sensor2Sensor 利用扩散架构执行生成式转换。我们对所生成传感器数据的保真度与真实感进行了全面的定量评估。通过将具有挑战性的互联网自然场景视频与行车记录仪影像转换为真实感的多模态数据格式，我们展示了 Sensor2Sensor 的实际效用，进一步为 AV 研发解锁了海量的外部数据源。
英文原文摘要 1485 字符
Robust training and validation of Autonomous Driving Systems (ADS) require massive, diverse datasets. Proprietary data collected by Autonomous Vehicle (AV) fleets, while high-fidelity, are limited in scale, diversity of sensor configurations, as well as geographic and long-tail-behavioral coverage. In contrast, in-the-wild data from sources like dashcams offers immense scale and diversity, capturing critical long-tail scenarios and novel environments. However, this unstructured, in-the-wild video data is incompatible with ADS expecting structured, multi-modal sensor inputs for validation and training. To bridge this data gap, we propose Sensor2Sensor, a novel generative modeling paradigm that translates in-the-wild monocular dashcam videos into a high-fidelity, multi-modal sensor suite (AV logs) comprising multi-view camera images and LiDAR point clouds. A core challenge is the lack of paired training data. We address this by converting real AV logs into dashcam-style videos via 4D Gaussian Splatting (4DGS) reconstruction and novel-view rendering. Sensor2Sensor then utilizes a diffusion architecture to perform the generative conversion. We perform comprehensive quantitative evaluations on the fidelity and realism of the generated sensor data. We demonstrate Sensor2Sensor's practical utility by converting challenging in-the-wild internet and dashcam footage into realistic, multi-modal data formats, further unlocking vast external data sources for AV development.