面向稀疏直接飞行时间传感器的稠密度量深度补全
Dense Metric Depth Completion from Sparse Direct Time-of-Flight Sensors
提出深度引导的双分支ViT编码器与掩码联合注意力，防止稀疏深度token被图像特征覆盖，为稀疏深度补全提供可复用的融合机制；同时开源dToF仿真管线，支持在合成数据上训练后零样本迁移到真实设备。
原文依据 a depth-guided dual-branch Vision Transformer encoder that processes RGB images and sparse dToF measurements separately
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论直接飞行时间（dToF）传感器可提供高精度的度量深度，并且在具有挑战性的真实世界条件下比间接ToF系统更鲁棒。然而，其高昂的制造成本和有限的光电二极管阵列尺寸导致深度图极其稀疏、低分辨率且含噪，使其不适用于需要稠密度量深度的VR/XR、机器人及3D感知任务。现有的单目与深度补全方法难以处理dToF设备独特的采样模式和硬件伪影，在严重稀疏或噪声条件下性能往往显著下降。我们提出一个可泛化的框架，用于从稀疏dToF测量中完成稠密度量深度补全，能够跨多种传感器类型、稀疏程度和噪声条件运行。我们的模型采用深度引导的双分支视觉Transformer编码器，分别处理RGB图像和稀疏dToF测量，同时一个掩码联合注意力模块使深度token能够可靠地引导图像特征而不被其覆盖。轻量级解码器高效重建稠密度量深度，无需基于扩散或繁重的细化后处理。为解决配对训练数据稀缺的问题，我们引入一个全面的dToF仿真管线，复现闪光、亚VGA闪光和旋转传感器的特性，包括硬件引起的退化、不规则稀疏性和真实噪声分布。完全在合成数据上训练，我们的模型在6个数据集和3个真实dToF设备上实现了强大的零样本泛化，在准确性和计算效率上均优于最优方法。这为从稀疏直接ToF传感器进行稠密度量深度补全建立了一个鲁棒且实用的解决方案。
英文原文摘要 1826 字符
Direct Time-of-Flight (dToF) sensors provide highly accurate metric depth and are more robust than indirect ToF systems in challenging real-world conditions. However, their high manufacturing cost and limited photodiode array size produce depth maps that are extremely sparse, low-resolution, and noisy, making them unsuitable for VR/XR, robotics, and 3D perception tasks that require dense metric depth. Existing monocular and depth completion methods struggle to handle the unique sampling patterns and hardware artifacts of dToF devices, and their performance often deteriorates significantly under severe sparsity or noise. We present a generalizable framework for dense metric depth completion from sparse dToF measurements, capable of operating across diverse sensor types, sparsity levels, and noise conditions. Our model employs a depth-guided dual-branch Vision Transformer encoder that processes RGB images and sparse dToF measurements separately, while a masked joint attention module allows depth tokens to reliably guide image features without being overwritten by them. A lightweight decoder reconstructs dense metric depth efficiently, without diffusion-based or refinement-heavy post-processing. To address the scarcity of paired training data, we introduce a comprehensive dToF simulation pipeline that reproduces the characteristics of flash, sub-VGA flash, and rotating sensors, including hardware-induced degradation, irregular sparsity, and realistic noise distributions. Trained entirely on synthetic data, our model achieves strong zero-shot generalization across 6 datasets and 3 real dToF devices, outperforming state-of-the-art approaches in both accuracy and computational efficiency. This establishes a robust and practical solution for dense metric depth completion from sparse direct ToF sensors.