AdaSpark：面向高效长视频理解的自适应稀疏性
AdaSpark: Adaptive Sparsity for Efficient Long-Video Understanding
AdaSpark 的「3D 时空立方体划分 + 熵驱动 Top-p 自适应选择」可直接迁移到任何长视频或多帧输入的视觉 Transformer 中，用输入复杂度动态决定注意力与 FFN 的计算预算，在保留细粒度与长程时序依赖的同时降低浮点运算量。
原文依据 AdaSpark significantly reduces computational load by up to 57% FLOPs while maintaining comparable performance to dense models
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论使用视频大语言模型（Video-LLM）处理长视频在计算上代价高昂。现有的高效方法往往通过不可逆的信息丢弃损害细粒度感知，或通过僵化的预定义稀疏模式抑制长程时序建模。本文提出 AdaSpark，一个旨在解决上述局限的自适应稀疏性框架。AdaSpark 首先将视频输入划分为三维时空立方体，随后采用两个协同设计、上下文感知的组件：（1）自适应立方体选择注意力（AdaS-Attn），为每个查询 token 动态选择一部分相关视频立方体进行注意力计算；（2）自适应 token 选择前馈网络（AdaS-FFN），仅选择性地处理每个立方体内最显著的 token。一种基于熵的（Top-p）选择机制根据输入复杂度自适应地分配计算资源。实验表明，AdaSpark 在将浮点运算量最多降低 57% 的同时，保持了与稠密模型相当的性能，并保留了细粒度、长程依赖关系，这一点在具有挑战性的小时级视频基准上得到了验证。
英文原文摘要 1174 字符
Processing long-form videos with Video Large Language Models (Video-LLMs) is computationally prohibitive. Current efficiency methods often compromise fine-grained perception through irreversible information disposal or inhibit long-range temporal modeling via rigid, predefined sparse patterns. This paper introduces AdaSpark, an adaptive sparsity framework designed to address these limitations. AdaSpark first partitions video inputs into 3D spatio-temporal cubes. It then employs two co-designed, context-aware components: (1) Adaptive Cube-Selective Attention (AdaS-Attn), which dynamically selects a subset of relevant video cubes to attend for each query token, and (2) Adaptive Token-Selective FFN (AdaS-FFN), which selectively processes only the most salient tokens within each cube. An entropy-based (Top-p) selection mechanism adaptively allocates computational resources based on input complexity. Experiments demonstrate that AdaSpark significantly reduces computational load by up to 57% FLOPs while maintaining comparable performance to dense models and preserving fine-grained, long-range dependencies, as validated on challenging hour-scale video benchmarks.