面向稠密场景理解的跨视角关联三维感知多任务学习
3D-Aware Multi-Task Learning with Cross-View Correlations for Dense Scene Understanding
把跨视角代价体作为几何一致性先验注入多任务编码器，为分割/深度等稠密任务提供 3D 感知的跨任务关联；该模块与架构无关，可直接插到现有 MTL 网络中复用。
原文依据 we introduce a lightweight Cross-view Module (CvM), shared across tasks, to exchange information across views and capture cross-view correlations
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论本文研究训练单一网络联合执行多种稠密预测任务（如分割与深度估计）的难题，即多任务学习（MTL）。现有方法主要在二维图像空间中捕捉跨任务关系，往往导致缺乏三维感知的非结构化特征。我们认为，三维感知对于建模全面场景理解所必需的跨任务关联至关重要。为此，我们提出将跨视角关联（即代价体）作为几何一致性整合进 MTL 网络来解决该问题。具体而言，我们引入一个轻量级的跨视角模块（CvM），该模块在各任务间共享，用于跨视角交换信息并捕捉跨视角关联，这些关联与 MTL 编码器特征融合以进行多任务预测。该模块与架构无关，可同时应用于单视角与多视角数据。在 NYUv2 与 PASCAL-Context 上的大量结果表明，我们的方法能有效地将几何一致性注入现有 MTL 方法，从而提升性能。
英文原文摘要 1087 字符
This paper addresses the challenge of training a single network to jointly perform multiple dense prediction tasks, such as segmentation and depth estimation, i.e., multi-task learning (MTL). Current approaches mainly capture cross-task relations in the 2D image space, often leading to unstructured features lacking 3D-awareness. We argue that 3D-awareness is vital for modeling cross-task correlations essential for comprehensive scene understanding. We propose to address this problem by integrating correlations across views, i.e., cost volume, as geometric consistency in the MTL network. Specifically, we introduce a lightweight Cross-view Module (CvM), shared across tasks, to exchange information across views and capture cross-view correlations, which are integrated with MTL encoder features for multi-task prediction. This module is architecture-agnostic and can be applied to both single- and multi-view data. Extensive results on NYUv2 and PASCAL-Context demonstrate that our method effectively injects geometric consistency into existing MTL methods to improve performance.