LongVT：通过原生工具调用激励“用长视频思考”
LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Calling
将LMM固有的时序定位能力作为原生视频裁剪工具，实现全局到局部的迭代推理，为长视频理解提供可复用的智能体框架与数据构造流程。
原文依据 we exploit LMMs' inherent temporal grounding ability as a native video cropping tool to zoom in on a specific video clip
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论大型多模态模型（LMM）在基于文本思维链的视频推理中展现出巨大潜力。然而，它们仍然容易产生幻觉，尤其是在处理证据稀疏且时序分散的长视频时。受人类理解长视频的方式——先全局浏览，再检查相关片段以获取细节——的启发，我们提出了LongVT，一个端到端的智能体框架，通过交错的多模态工具思维链实现“用长视频思考”。具体而言，我们利用LMM固有的时序定位能力作为原生视频裁剪工具，以放大特定视频片段并重新采样更细粒度的视频帧。这种从全局到局部的推理循环持续进行，直到答案建立在检索到的视觉证据之上。鉴于长视频推理任务中细粒度问答（QA）数据的稀缺，我们整理并发布了一个名为VideoSIAH的数据套件，以促进训练和评估。具体而言，我们的训练数据集包含247.9K个用于工具集成冷启动监督微调的样本、1.6K个用于智能体强化学习的样本以及15.4K个用于智能体强化微调的样本。我们的评估基准包含652个问答对，这些问答对通过半自动数据流程并经过人工在环验证精心整理而成。借助精心设计的三阶段训练策略和大量实证验证，LongVT在四个具有挑战性的长视频理解与推理基准上持续优于现有的强基线。我们的代码、数据和模型权重已公开于https://github.com/EvolvingLMMs-Lab/LongVT。
英文原文摘要 1698 字符
Large multimodal models (LMMs) have shown great potential for video reasoning with textual Chain-of-Thought. However, they remain vulnerable to hallucinations, especially when processing long-form videos where evidence is sparse and temporally dispersed. Inspired by how humans comprehend long videos--by first skimming globally and then examining relevant clips for details--we introduce LongVT, an end-to-end agentic framework that enables "Thinking with Long Videos" via interleaved Multimodal Chain-of-Tool-Thought. Specifically, we exploit LMMs' inherent temporal grounding ability as a native video cropping tool to zoom in on a specific video clip and resample finer-grained video frames. This global-to-local reasoning loop continues until answers are grounded in retrieved visual evidence. Given the scarcity of fine-grained question-answering (QA) data for the long video reasoning task, we curate and release a data suite named VideoSIAH to facilitate both training and evaluation. Specifically, our training dataset consists of 247.9K samples for tool-integrated cold-start supervised fine-tuning, 1.6K samples for agentic reinforcement learning, and 15.4K samples for agentic reinforcement fine-tuning. Our evaluation benchmark consists of 652 QA pairs that are carefully curated through a semi-automatic data pipeline with human-in-the-loop validation. With a meticulously designed three-stage training strategy and extensive empirical validation, LongVT consistently outperforms existing strong baselines across four challenging long-video understanding and reasoning benchmarks. Our code, data, and model weights are publicly available at https://github.com/EvolvingLMMs-Lab/LongVT.