TR2M：借助语言描述与双层尺度导向对比将单目相对深度迁移为度量深度
TR2M: Transferring Monocular Relative Depth to Metric Depth with Language Descriptions and Dual-Level Scale-Oriented Contrast
把相对深度经文本+图像双模态条件、以像素级重缩放图映射到度量深度，并用深度分布引导的双层对比学习约束尺度，可作为跨域深度尺度对齐与伪度量标签构建的可复用范式。
原文依据 estimates two rescale maps to transfer relative depth to metric depth at the pixel level
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论本文提出一个可将相对深度迁移为度量深度的可泛化框架。当前单目深度估计方法主要分为度量深度估计（MMDE）与相对深度估计（MRDE）。MMDE 以度量尺度估计深度，但往往局限于特定域；MRDE 在不同域间泛化良好，但尺度不确定，阻碍了下游应用。为此，我们旨在构建一个框架来解决尺度不确定性问题，并将相对深度迁移为度量深度。先前方法以语言为输入并估计两个因子以进行重缩放。我们的方法 TR2M 同时以文本描述和图像为输入，估计两张重缩放图，在像素级将相对深度迁移为度量深度。两种模态的特征通过跨模态注意力模块融合，以更好地捕捉尺度信息。我们设计了一种策略来构建并筛选置信的伪度量深度，以获得更全面的监督。我们还提出双层尺度导向对比学习，利用深度分布作为引导，促使模型学习与尺度分布一致的内在线索。TR2M 仅使用少量可训练参数即可在多个域的数据集上训练，实验不仅展示了 TR2M 在已见数据集上的优异性能，也揭示了其在五个未见数据集上更优的零样本能力。我们展示了借助语言辅助、以像素级方式将相对深度迁移为度量深度的巨大潜力，而无需依赖大规模训练数据训练的大体量度量深度模型。代码见：https://github.com/BeileiCui/TR2M。
英文原文摘要 1711 字符
This work presents a generalizable framework to transfer relative depth to metric depth. Current monocular depth estimation methods are mainly divided into metric depth estimation (MMDE) and relative depth estimation (MRDE). MMDEs estimate depth in metric scale but are often limited to a specific domain. MRDEs generalize well across different domains, but with uncertain scales that hinder downstream applications. To this end, we aim to build up a framework to solve scale uncertainty and transfer relative depth to metric depth. Previous methods used language as input and estimated two factors for conducting rescaling. Our approach, TR2M, utilizes both text descriptions and images as inputs and estimates two rescale maps to transfer relative depth to metric depth at the pixel level. Features from two modalities are fused with a cross-modality attention module to better capture scale information. A strategy is designed to construct and filter confident pseudo metric depth for more comprehensive supervision. We also develop dual-level scale-oriented contrastive learning to utilize depth distribution as guidance to enforce the model learning about intrinsic cues consistent with the scale distribution. TR2M only exploits a small number of trainable parameters to train on datasets in various domains and experiments not only demonstrate TR2M's great performance in seen datasets but also reveal superior zero-shot capabilities on five unseen datasets. We show the huge potential in pixel-wise transferring relative depth to metric depth with language assistance instead of large-size metric depth models with large amounts of training data. Code is available at: https://github.com/BeileiCui/TR2M.