迈向3D世界中的视觉查询定位
Towards Visual Query Localization in the 3D World
论文提出的3DVQL基准和LaF融合算法为视觉研究者提供了在3D点云与RGB-D数据上进行时空查询定位的新任务设定和可复用的多模态融合模块，可直接用于点云-图像融合的定位与跟踪研究。
原文依据 we make the first attempt to visual query localization in the 3D world by introducing a novel benchmark, dubbed 3DVQL
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉查询定位（VQL）旨在给定一个查询，从序列中预测最近一次出现位置的时空响应。目前，大多数研究集中于2D视频中的视觉查询定位，而其在3D空间中的对应研究却鲜有关注。本文首次尝试在3D世界中进行视觉查询定位，并引入了一个名为3DVQL的新基准。具体而言，3DVQL包含来自38个物体类别的2,002个序列，约170,000帧和6.4K个响应轨迹片段。3DVQL中的每个序列均提供多种模态，包括点云（PC）、RGB和深度图像，以支持灵活的研究。为确保高质量标注，每个序列均经过多轮人工验证与精炼。据我们所知，3DVQL是首个面向3D多模态视觉查询定位的基准。为便于后续研究的比较，我们利用点云和RGB实现了一系列具有代表性的3D多模态VQL基线。实验结果表明，现有方法在不同融合模块下表现出显著的性能差异。为促进未来研究，我们提出了一种名为LaF的提升与注意力融合算法，其性能显著优于现有基线模型。我们的基准和模型将在网页https://github.com/wuhengliangliang/3DVQL上公开。
英文原文摘要 1430 字符
Visual query localization (VQL) aims to predict a spatial-temporal response of the most recent occurrence from a sequence given a query. Currently, most research focuses on visual query localization from 2D videos, while its counterpart in 3D space has received little attention. In this paper, we make the first attempt to visual query localization in the 3D world by introducing a novel benchmark, dubbed 3DVQL. Specifically, 3DVQL contains 2,002 sequences with around 170,000 frames and 6.4K response track segments from 38 object categories. Each sequence in 3DVQL is provided with multiple modalities including point clouds (PC), RGB and depth images to support flexible research. To ensure high-quality annotation, each sequence is manually annotated with multiple rounds of verification and refinement. To our best knowledge, 3DVQL is the first benchmark towards 3D multimodal visual query localization. To facilitate comparison for subsequent research, we implement a series of representative 3D multimodal VQL baselines using PC and RGB. The experimental results show that existing methods exhibit significant performance variations across different fusion modules. To encourage future research, we propose a lift and attention fusion algorithm named LaF, which significantly outperforms existing baseline models. Our benchmark and model will be publicly released at our webpage https://github.com/wuhengliangliang/3DVQL.