UFVideo：面向统一细粒度视频协同理解的大语言模型
UFVideo: Towards Unified Fine-Grained Video Cooperative Understanding with Large Language Models
把全局、像素、时序三种粒度的视频理解统一进单个 Video LLM，并用统一的视觉-语言引导对齐动态切换输出（文本、时序定位、grounded mask），为做视频问答、时序定位与指代分割的研究者提供了一个多任务协同的统一建模范式与配套基准。
原文依据 we design unified visual-language guided alignment to flexibly handle video understanding across global, pixel and temporal scales within a single model
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论随着多模态大语言模型（LLM）的发展，视频 LLM 得到进一步演进，能够执行整体性与专门化的视频理解。然而，现有工作局限于专门的视频理解任务，无法实现全面且多粒度的视频感知。为弥合这一差距，我们提出 UFVideo，这是首个具备统一多粒度协同理解能力的视频 LLM。具体而言，我们设计了统一的视觉-语言引导对齐，以在单一模型内灵活处理全局、像素与时序尺度上的视频理解。UFVideo 动态编码不同任务的视觉与文本输入，并生成文本响应、时序定位或 grounded mask。此外，为评估具有挑战性的多粒度视频理解任务，我们构建了 UFVideo-Bench，其中包含跨这些尺度的三个不同的协同任务，展示了 UFVideo 相较于 GPT-4o 的灵活性与优势。进一步地，我们在涵盖多种常见视频理解任务的 9 个公开基准上验证了模型的有效性，为未来的视频 LLM 提供了有价值的洞见。
英文原文摘要 1193 字符
With the advancement of multi-modal Large Language Models (LLMs), Video LLMs have been further developed to perform on holistic and specialized video understanding. However, existing works are limited to specialized video understanding tasks, failing to achieve a comprehensive and multi-grained video perception. To bridge this gap, we introduce UFVideo, the first Video LLM with unified multi-grained cooperative understanding capabilities. Specifically, we design unified visual-language guided alignment to flexibly handle video understanding across global, pixel and temporal scales within a single model. UFVideo dynamically encodes the visual and text inputs of different tasks and generates the textual response, temporal localization, or grounded mask. Additionally, to evaluate challenging multi-grained video understanding tasks, we construct the UFVideo-Bench consisting of three distinct collaborative tasks within the scales, which demonstrates UFVideo's flexibility and advantages over GPT-4o. Furthermore, we validate the effectiveness of our model across 9 public benchmarks covering various common video understanding tasks, providing valuable insights for future Video LLMs.