超越基于描述文本的查询：视频时刻检索中的搜索式查询
Beyond Caption-Based Queries in Video Moment Retrieval
对做视频-语言定位的人：论文指出 DETR 式 VMR 模型在从描述性 caption 查询迁移到简短搜索查询时，会出现解码器查询坍缩（active decoder-query collapse），并给出通过增加活跃解码器查询数量来缓解的架构改法；这套诊断与改法可直接迁移到其他基于 DETR 的 grounding/定位模型上。
原文依据 we identify a critical issue in these architectures---an active decoder-query collapse---as a primary cause of the poor generalization to multi-moment instances
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论当前的视频时刻检索（VMR）模型在配有描述文本的视频上训练，这些描述文本由标注者在观看视频后撰写。这些描述文本被用作文本查询——我们称之为基于描述文本的查询。这种标注过程引入了视觉偏差，导致查询过于描述性和细粒度，与用户在实践中可能使用的更通用的搜索查询存在显著差异。在本工作中，我们研究了现有 VMR 方法（尤其是 DETR 架构）在基于描述文本的查询上训练、但在搜索查询上评估时的性能退化。为此，我们通过修改三个公开 VMR 数据集（即 HD-EPIC、YouCook2 和 ActivityNet-Captions）中的文本查询，引入了三个基准。我们的分析揭示了两个关键的泛化挑战：(i) 语言鸿沟，源于搜索查询在语言上的欠指定；(ii) 多时刻鸿沟，由从单时刻查询向多时刻查询的转变引起。我们还识别出这些架构中的一个关键问题——活跃解码器查询坍缩——是导致对多时刻实例泛化不佳的主要原因。我们通过架构修改有效增加了活跃解码器查询的数量，从而缓解了该问题。大量实验表明，我们的方法在搜索查询上将性能提升了最多 14.82% mAP_m，在多时刻搜索查询上提升了最多 21.83% mAP_m。
英文原文摘要 1463 字符
Current Video Moment Retrieval (VMR) models are trained on videos paired with captions, which are written by annotators after watching the videos. These captions are used as textual queries---which we term caption-based queries. This annotation process induces a visual bias, leading to overly descriptive and fine-grained queries, which significantly differ from the more general search queries that users are likely to employ in practice. In this work, we investigate the degradation of existing VMR methods, particularly of DETR architectures, when trained on caption-based queries but evaluated on search queries. For this, we introduce three benchmarks by modifying the textual queries in three public VMR datasets---i.e., HD-EPIC, YouCook2 and ActivityNet-Captions. Our analysis reveals two key generalization challenges: (i) A language gap, arising from the linguistic under-specification of search-queries, and (ii) a multi-moment gap, caused by the shift from single moment to multi-moment queries. We also identify a critical issue in these architectures---an active decoder-query collapse---as a primary cause of the poor generalization to multi-moment instances. We mitigate this issue with architectural modifications that effectively increase the number of active decoder queries. Extensive experiments demonstrate that our approach improves performance on search queries by up to 14.82% mAP_m, and up to 21.83% mAP_m on multi-moment search queries.