GIFT：面向高效视频理解的全局不可替代性帧定位
GIFT: Global Irreplaceability Frame Targeting for Efficient Video Understanding
提出一种免训练的关键帧选择框架，用「定向多样性」把相关性与多样性统一成不可替代性分数，并配合预算感知的自适应迭代选帧；做视频 VLM 推理或长视频理解时，可直接把这一选帧策略插到现有 LLaVA-Video 类模型的输入前端，替代均匀采样。
原文依据 a novel training-free framework that selects frames by assessing their intrinsic irreplaceability
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视频大语言模型（VLM）在视频理解上取得了显著成功，但处理稠密帧所带来的巨大计算开销严重限制了其实际应用。现有方法通过选取关键帧来缓解这一问题，但其贪心式决策，加上对相关性与多样性的解耦评估，往往陷入局部最优，并错误地选中无关的噪声帧。为应对这些挑战，我们提出 GIFT：全局不可替代性帧定位，这是一个全新的免训练框架，通过评估帧的内在不可替代性来进行选帧。具体而言，我们首先引入定向多样性，以在相关性条件下量化一帧的独特性，从而使我们能够构建统一的不可替代性分数。随后，我们的预算感知精炼策略采用自适应迭代过程：先锁定不可替代性最高的一小组核心帧，随着预算扩大，再把优先级转向围绕这些已选帧构建关键的时序上下文。大量实验表明，在 LLaVA-Video-7B 上，与均匀采样相比，GIFT 在长视频基准上取得了最高 12.5% 的平均提升。代码即将发布。
英文原文摘要 1289 字符
Video Large Language Models (VLMs) have achieved remarkable success in video understanding, but the significant computational cost from processing dense frames severely limits their practical application. Existing methods alleviate this by selecting keyframes, but their greedy decision-making, combined with a decoupled evaluation of relevance and diversity, often falls into local optima and results in erroneously selecting irrelevant noise frames. To address these challenges, we propose GIFT: Global Irreplaceability Frame Targeting, a novel training-free framework that selects frames by assessing their intrinsic irreplaceability. Specifically, we first introduce Directed Diversity to quantify a frame's uniqueness conditioned on relevance, which allows us to formulate a unified irreplaceability score. Subsequently, our Budget-Aware Refinement strategy employs a adaptive iterative process that first secures a core set of frames with the highest irreplaceability, and then shifts its priority to building crucial temporal context around these selections as the budget expands. Extensive experiments demonstrate that GIFT achieves a maximum average improvement of 12.5% across long-form video benchmarks on LLaVA-Video-7B compared to uniform sampling. Code will be released soon.