重思指代多目标跟踪中的两阶段“跟踪即指代”:让它重新强大
Rethinking Two-Stage Referring-by-Tracking in Referring Multi-Object Tracking: Make it Strong Again
对做视觉语言跟踪的人,这篇给出了一个可复用的两阶段范式:用采样式 Conditioning Hook 把语言线索注入检测特征,再用 Pairwise Correspondence Decoder 主动建模轨迹与表达的对应关系,替代 CLIP 相似度匹配,从而在低训练成本下达到最优性能。
原文依据 we introduce a Pairwise Correspondence Decoder (PCD) that replaces CLIP-based similarity matching with active correspondence modeling
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论指代多目标跟踪(RMOT)旨在视频中跟踪由自然语言表达式指定的多个目标。随着近期单阶段方法的显著进展,两阶段的“跟踪即指代”(RBT)范式逐渐失去热度。然而,其更低的训练成本和灵活的增量部署能力仍不可替代。重新审视现有的两阶段 RBT 框架,我们发现两个根本性局限:过于启发式的特征构建与脆弱的对应关系建模。为解决这些问题,我们提出 FlexHook,一种新的两阶段 RBT 框架。在 FlexHook 中,所提出的条件钩子(C-Hook)通过基于采样的策略与语言条件线索注入重新定义了特征构建。随后,我们引入成对对应解码器(PCD),以主动对应关系建模取代基于 CLIP 的相似度匹配,从而得到更灵活、更鲁棒的策略。在多个基准(Refer-KITTI/v2、Refer-Dance 和 LaMOT)上的大量实验表明,FlexHook 成为首个在性能与效率上均超越当前最优方法的两阶段 RBT 方法。代码见 https://github.com/buptLwz/FlexHook。
英文原文摘要 1238 字符
Referring Multi-Object Tracking (RMOT) aims to track multiple objects specified by natural language expressions in videos. With the recent significant progress of one-stage methods, the two-stage Referring-by-Tracking (RBT) paradigm has gradually lost its popularity. However, its lower training cost and flexible incremental deployment remain irreplaceable. Rethinking existing two-stage RBT frameworks, we identify two fundamental limitations: the overly heuristic feature construction and fragile correspondence modeling. To address these issues, we propose FlexHook, a novel two-stage RBT framework. In FlexHook, the proposed Conditioning Hook (C-Hook) redefines the feature construction by a sampling-based strategy and language-conditioned cue injection. Then, we introduce a Pairwise Correspondence Decoder (PCD) that replaces CLIP-based similarity matching with active correspondence modeling, yielding a more flexible and robust strategy. Extensive experiments on multiple benchmarks (Refer-KITTI/v2, Refer-Dance, and LaMOT) demonstrate that FlexHook becomes the first two-stage RBT approach to outperform current state-of-the-art methods both performance and efficiency. Code is available at https://github.com/buptLwz/FlexHook.