MotionEnhancer：利用视频扩散模型增强视觉语言模型的运动理解能力
MotionEnhancer: Leveraging Video Diffusion for Motion-Enhanced Vision-Language Models
把视频扩散模型内部对动态运动敏感的注意力当作先验，通过注意力对齐蒸馏进 VLM，且完全免训练、免改结构——做视频 VLM 的人可以直接复用这套「从生成模型借运动先验」的思路来补细粒度运动理解。
原文依据 leverages motion priors distilled from a powerful video diffusion model as auxiliary supervision to enhance the motion understanding capability of a VLM via attention alignment
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论新时代见证了将视觉语言模型（VLM）扩展至视频理解任务的显著能力。尽管当前 VLM 擅长事件级或故事级的理解，但其捕捉细粒度运动细节的能力仍然有限，主要原因是它们聚焦于高层的静态语义结构与宏观事件逻辑。相比之下，视频扩散模型（VDM）受益于大规模视频数据与时序生成的内在需求，擅长对动态运动模式建模。本文提出 MotionEnhancer，一种新方法，它利用从强大的视频扩散模型中蒸馏出的运动先验作为辅助监督，通过注意力对齐增强 VLM 的运动理解能力。MotionEnhancer 由两个简单的无参数模块组成，即运动敏感检测头选择（MHS）与运动显著文本词元识别（MTTI），以仅计算的方式直接从 VDM 中提取并优化与运动相关的注意力。MotionEnhancer 为运动理解提供了一种可扩展的解决方案，无需额外的训练参数、无需修改现有架构、也无需调用外部工具。大量实验表明，MotionEnhancer 在两个运动级视频理解基准上相较最优 VLM 取得了一致提升，尤其是在与运动相关的指标上。
英文原文摘要 1414 字符
The new era has witnessed a remarkable capability to extend Vision-Language Models (VLMs) for tackling tasks of video understanding. While current VLMs excel at event- or story-level understanding, their ability to capture fine-grained motion details remains limited, primarily due to their focus on high-level static semantic structures and macro-event logic. In contrast, Video Diffusion Models (VDMs) are adept at modeling dynamic motion patterns, benefiting from large-scale video data and the intrinsic requirement of temporal generation. In this paper, we introduce MotionEnhancer, a novel approach that leverages motion priors distilled from a powerful video diffusion model as auxiliary supervision to enhance the motion understanding capability of a VLM via attention alignment. MotionEnhancer comprises two simple parameter-free modules, Motion-sensitive Head Selection (MHS) and Motion-salient Text Token Identification (MTTI), to directly extract and optimize motion-related attentions from the VDM in a computation-only manner. MotionEnhancer provides a scalable solution for motion understanding without additional training parameters, modifications to existing architectures, or tool calling. Extensive experiments demonstrate that MotionEnhancer can achieve consistent improvements over state-of-the-art VLMs on two motion-level video understanding benchmarks, especially on motion-related metrics.