视觉语言模型 · CVPR2026 ·2026.06.03

FluxMem:面向流式视频理解的自适应分层记忆

FluxMem: Adaptive Hierarchical Memory for Streaming Video Understanding

Yiweng Xie、Bo He、Junke Wang、Xiangyu Zheng、Ziyi Ye、Zuxuan Wu

与你有关的部分

提出免训练的分层视觉词元压缩方法(TAS+SDC),通过自适应压缩率减少冗余视觉词元,可直接用于视频语言模型的高效推理,降低延迟和显存。

原文依据 FluxMem adaptively compresses redundant visual memory through a hierarchical, two-stage design
可迁移到 视频问答视频理解视觉语言模型推理加速

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

本文提出 FluxMem,一种用于高效流式视频理解的免训练框架。FluxMem 通过分层两阶段设计自适应地压缩冗余视觉记忆:(1)时序邻接选择(TAS)模块移除相邻帧间的冗余视觉词元;(2)空间域整合(SDC)模块进一步将每帧内空间重复区域合并为紧凑表示。为有效适应动态场景,我们在 TAS 和 SDC 中引入自适应词元压缩机制,该机制基于内在场景统计自动确定压缩率,而非手动调参。大量实验表明,FluxMem 在现有在线视频基准上取得新的最优结果,在实时设置下 StreamingBench 达到 76.4,OVO-Bench 达到 67.2,同时在 OVO-Bench 上降低延迟 69.9%、峰值 GPU 显存 34.5%。此外,它保持强大的离线性能,在 MLVU 上达到 73.1,同时减少 65% 的视觉词元。

英文原文摘要 1067 字符

This paper presents FluxMem, a training-free framework for efficient streaming video understanding. FluxMem adaptively compresses redundant visual memory through a hierarchical, two-stage design: (1) a Temporal Adjacency Selection (TAS) module removes redundant visual tokens across adjacent frames, and (2) a Spatial Domain Consolidation (SDC) module further merges spatially repetitive regions within each frame into compact representations. To adapt effectively to dynamic scenes, we introduce a self-adaptive token compression mechanism in both TAS and SDC, which automatically determines the compression rate based on intrinsic scene statistics rather than manual tuning. Extensive experiments demonstrate that FluxMem achieves new state-of-the-art results on existing online video benchmarks, reaching 76.4 on StreamingBench and 67.2 on OVO-Bench under real-time settings, while reducing latency by 69.9% and peak GPU memory by 34.5% on OVO-Bench. Furthermore, it maintains strong offline performance, achieving 73.1 on MLVU while using 65% fewer visual tokens.

标签

会议 CVPR2026
页码 31272-31282
Bibkey
来源 CVF Open Access