HERO：面向开放词表视频时序句子定位的分层嵌入精炼
HERO: Hierarchical Embedding-Refinement for Open-Vocabulary Temporal Sentence Grounding in Videos
论文提出的开放词表时序句子定位任务和分层嵌入精炼框架，为视觉语言模型在词汇偏移下的泛化提供了可复用的评测基准和跨模态对齐方法，特别是语义引导的视觉过滤和对比掩码文本精炼技术，可直接用于视频-语言对齐任务。
原文依据 we introduce the Open-Vocabulary TSGV (OV-TSGV) task and construct the first dedicated benchmarks--Charades-OV and ActivityNet-OV
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视频时序句子定位（TSGV）旨在根据给定的自然语言查询在视频中时序定位对应的片段。尽管近期取得了进展，但大多数现有TSGV方法在闭词表设置下运行，限制了它们对涉及新颖或多样语言表达的真实世界查询的泛化能力。为弥合这一关键差距，我们引入了开放词表TSGV（OV-TSGV）任务，并构建了首个专用基准——Charades-OV和ActivityNet-OV——模拟真实的词汇偏移和释义变化。这些基准有助于系统评估模型在未见训练概念之外的泛化能力。为解决OV-TSGV问题，我们提出了HERO（面向开放词表定位的分层嵌入精炼），一个利用分层语言嵌入并执行并行跨模态精炼的统一框架。HERO联合建模多级语义，并通过语义引导的视觉过滤和对比掩码文本精炼增强视频-语言对齐。在标准和开放词表基准上的大量实验表明，HERO持续超越最优方法，尤其在开放词表场景下，验证了其强大的泛化能力，并凸显了OV-TSGV作为新研究方向的重要性。
英文原文摘要 1370 字符
Temporal Sentence Grounding in Videos (TSGV) aims to temporally localize segments of a video that correspond to a given natural language query. Despite recent progress, most existing TSGV approaches operate under closed-vocabulary settings, limiting their ability to generalize to real-world queries involving novel or diverse linguistic expressions. To bridge this critical gap, we introduce the Open-Vocabulary TSGV (OV-TSGV) task and construct the first dedicated benchmarks--Charades-OV and ActivityNet-OV--that simulate realistic vocabulary shifts and paraphrastic variations. These benchmarks facilitate systematic evaluation of model generalization beyond seen training concepts. To tackle OV-TSGV, we propose HERO (Hierarchical Embedding-Refinement for Open-Vocabulary grounding), a unified framework that leverages hierarchical linguistic embeddings and performs parallel cross-modal refinement. HERO jointly models multi-level semantics and enhances video-language alignment via semantic-guided visual filtering and contrastive masked text refinement. Extensive experiments on both standard and open vocabulary benchmarks demonstrate that HERO consistently surpasses state-of-the-art methods, particularly under open-vocabulary scenarios, validating its strong generalization capability and underscoring the significance of OV-TSGV as a new research direction.