TimeLens:重新审视多模态大语言模型的视频时序定位
TimeLens: Rethinking Video Temporal Grounding with Multimodal LLMs
对做视频-语言理解的人,这篇给出了把 MLLM 训练成强时序定位器的完整配方:交错文本时间编码、免思维链的 RLVR 训练范式,以及一套可复用的高质量重标注流程与修正后的评测基准,可直接用于重训或重评自己的时序定位模型。
原文依据 a thinking-free reinforcement learning with verifiable rewards (RLVR) approach as the training paradigm
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论本文并不提出新方法,而是为视频理解中的核心能力——视频时序定位(VTG)——建立一个直接、渐进却至关重要的基线。尽管多模态大语言模型(MLLM)在各类视频理解任务上表现优异,但针对 VTG 的优化方案仍缺乏探索。本文提出 TimeLens,从数据质量与算法设计两个主要维度,系统性地研究如何构建具备强 VTG 能力的 MLLM。我们首先揭示现有 VTG 基准中存在的严重质量问题,并引入 TimeLens-Bench,它由三个常用基准在严格质量标准下精心重标注的版本构成。我们的分析显示,与旧有基准相比,模型排名发生了剧烈变化,证实了以往评测标准的不可靠性。我们还通过自动化重标注流程解决训练数据噪声问题,得到 TimeLens-100K,一个大规模高质量训练数据集。在数据基础之上,我们深入探索算法设计原则,得出一系列有意义的洞见以及有效而高效的做法,包括用于时间表示的交错文本编码、以免思维链的可验证奖励强化学习(RLVR)作为训练范式,以及为 RLVR 训练精心设计的配方。这些工作最终形成 TimeLens 系列模型,其在开源模型中取得最优的 VTG 性能,甚至超越 GPT-5 与 Gemini-2.5-Flash 等专有模型。所有代码、数据与模型都将公开以促进未来研究。
英文原文摘要 1697 字符
This paper does not introduce a novel method but instead establishes a straightforward, incremental, yet essential baseline for video temporal grounding (VTG), a core capability in video understanding. While multimodal large language models (MLLMs) excel at various video understanding tasks, the recipes for optimizing them for VTG remain under-explored. In this paper, we present TimeLens, a systematic investigation into building MLLMs with strong VTG ability, along two primary dimensions: data quality and algorithmic design. We first expose critical quality issues in existing VTG benchmarks and introduce TimeLens-Bench, comprising meticulously re-annotated versions of three popular benchmarks with strict quality criteria. Our analysis reveals dramatic model re-rankings compared to legacy benchmarks, confirming the unreliability of prior evaluation standards. We also address noisy training data through an automated re-annotation pipeline, yielding TimeLens-100K, a large-scale, high-quality training dataset. Building on our data foundation, we conduct in-depth explorations of algorithmic design principles, yielding a series of meaningful insights and effective yet efficient practices. These include interleaved textual encoding for time representation, a thinking-free reinforcement learning with verifiable rewards (RLVR) approach as the training paradigm, and carefully designed recipes for RLVR training. These efforts culminate in TimeLens models, a family of MLLMs with state-of-the-art VTG performance among open-source models and even surpass proprietary models such as GPT-5 and Gemini-2.5-Flash. All codes, data, and models will be released to facilitate future research.