流式视频指令微调
Streaming Video Instruction Tuning
把离线视频 LLM 改造成逐帧到达即可响应的流式助手，其 Streamo-Instruct-465K 的多任务指令构造方式与统一训练管线，可直接复用于需要实时解说、时序定位或时间敏感问答的视频-语言系统。
原文依据 a real-time streaming video LLM that serves as a general-purpose interactive assistant
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论我们提出 Streamo，一个实时流式视频大语言模型，可作为通用交互式助手。现有的在线视频模型大多局限于问答或描述生成，与之不同，Streamo 能够执行广泛的流式视频任务，包括实时解说、动作理解、事件描述、时序事件定位以及时间敏感问答。为获得这种通用性，我们构建了 Streamo-Instruct-465K，这是一个面向流式视频理解的大规模指令遵循数据集。该数据集覆盖多样的时序上下文与多任务监督，使得跨异构流式任务的统一训练成为可能。通过一条精简的流水线在该指令遵循数据集上进行端到端训练后，Streamo 展现出较强的时序推理能力、响应式交互能力，并在多种流式基准上表现出广泛的泛化能力。大量实验表明，Streamo 弥合了离线视频感知模型与实时多模态助手之间的差距，向着在连续视频流中实现统一、智能的视频理解迈出了一步。
英文原文摘要 1140 字符
We present Streamo, a real-time streaming video LLM that serves as a general-purpose interactive assistant. Unlike existing online video models that focus narrowly on question answering or captioning, Streamo performs a broad spectrum of streaming video tasks, including real-time narration, action understanding, event captioning, temporal event grounding, and time-sensitive question answering. To develop such versatility, we construct Streamo-Instruct-465K, a large-scale instruction-following dataset tailored for streaming video understanding. The dataset covers diverse temporal contexts and multi-task supervision, enabling unified training across heterogeneous streaming tasks. After training end-to-end on the instruction-following dataset through a streamlined pipeline, Streamo exhibits strong temporal reasoning, responsive interaction, and broad generalization across a variety of streaming benchmarks. Extensive experiments show that Streamo bridges the gap between offline video perception models and real-time multimodal assistants, making a step toward unified, intelligent video understanding in continuous video streams.