VidPrism：用于图像到视频迁移的异构混合专家
VidPrism: Heterogeneous Mixture of Experts for Image-to-Video Transfer
提出异构时序混合专家框架，通过内容感知多速率采样生成不同语义/运动强度的视频流，并分配给功能特化的专家，为视觉中的时序建模提供可复用的分工与融合机制。
原文依据 we propose VidPrism, a novel heterogeneous temporal Mixture-of-Experts framework
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论随着预训练技术的快速发展，将大规模视觉语言模型（VLM）适配于视频理解（即图像到视频迁移学习）已成为主流范式。为获得更优性能，近期进展中采用混合专家（MoE）来增强VLM的时序建模能力成为一种有效策略。然而，传统MoE设计存在专家同质化问题，所有专家都充当相同的通才，无法从无差别的视频流中高效学习时空特征。为解决该问题，我们提出VidPrism，一种新颖的异构时序混合专家框架。VidPrism开创性地进行分工，部署功能特化的专家，每个专家承担从空间理解到时序建模的不同角色。为恰当地为这些专家提供输入，我们引入内容感知的多速率采样模块，动态生成从语义丰富到运动聚焦的多种表示流，为专家提供特化输入。此外，动态双向融合机制使这些路径之间能够协同交换信息，从而获得全面的视频表示。在各种视频识别基准上的大量实验表明，VidPrism达到了最优性能，并有效促进了专家特化。
英文原文摘要 1428 字符
With the rapid development of pre-training technologies, adapting large-scale Vision-Language Models (VLMs) for video understanding i.e. image-to-video transfer learning has become a dominant paradigm. To achieve superior performance, it raises as an effective strategy among recent advances to employ Mixture-of-Experts (MoE) to enhance VLMs' temporal modeling capabilities. However, conventional MoE designs suffer from expert homogenization, where all experts act as identical generalists, inefficiently learning spatio-temporal features from undifferentiated video streams.To overcome this problem, we propose VidPrism, a novel heterogeneous temporal Mixture-of-Experts framework. VidPrism pioneers a division of labor by deploying functionally specialized experts, each assuming a role ranging from spatial understanding to temporal modeling.To feed these specialists appropriately, we introduce a content-aware, multi-rate sampling module that dynamically generates streams ranging from semantically rich to motion-focused representations, providing specialized inputs for experts.Furthermore, a dynamic, bidirectional fusion mechanism enables synergistic information exchange between these pathways, leading to a comprehensive video representation. Extensive experiments on various video recognition benchmarks demonstrate that VidPrism achieves state-of-the-art performance and effectively fosters expert specialization.