StreamingTOM：面向高效视频理解的流式词元压缩
StreamingTOM: Streaming Token Compression for Efficient Video Understanding
提出免训练的两阶段词元压缩框架，在LLM前通过因果时序缩减减少每帧预填充成本，在LLM后通过4位量化内存管理KV缓存，为流式视频理解提供可预测延迟和有限内存的解决方案。
原文依据 StreamingTOM, a training-free, plug-and-play two-stage framework that addresses both pre-LLM and post-LLM bottlenecks.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论与离线处理不同，流式视频视觉语言模型面临两个基本约束：因果性和累积性。因果性阻止访问离线方法所利用的未来帧，而累积性导致词元无限增长，造成效率瓶颈。然而，现有方法仅调控LLM后的kv缓存，未改变昂贵的LLM前预填充。我们提出StreamingTOM，一个免训练、即插即用的两阶段框架，同时解决LLM前和LLM后的瓶颈。因果时序缩减施加固定的每帧预算，并基于相邻帧变化和词元显著性选择词元，通过仅处理紧凑的视觉词元子集大幅降低每帧预填充成本，确保可预测的延迟。在线量化内存以4位格式存储词元，按需检索相关组并反量化，使活跃kv缓存无论流长度如何都保持有界。实验表明，我们的方法实现了15.7倍的kv缓存压缩比；与先前最优方法（LiveVLM）相比，峰值内存降低1.2倍，首词元时间快2倍。StreamingTOM在免训练方法中达到最优准确率，离线基准平均63.8%，RVS上准确率55.8%、得分3.7。这些结果表明，无需模型重训练即可实现具有有界活跃内存的实时流式视频理解。
英文原文摘要 1427 字符
Unlike offline processing, streaming video vision-language models face two fundamental constraints: causality and accumulation. Causality prevents access to future frames that offline methods exploit, while accumulation causes tokens to grow unbounded, creating efficiency bottlenecks. However, existing approaches only regulate post-LLM kv-cache, leaving costly pre-LLM prefill unchanged. We introduce StreamingTOM, a training-free, plug-and-play two-stage framework that addresses both pre-LLM and post-LLM bottlenecks. Causal Temporal Reduction imposes a fixed per-frame budget and selects tokens based on adjacent-frame changes and token saliency, drastically reducing per-frame prefill cost by processing only a compact subset of visual tokens, ensuring predictable latency. Online Quantized Memory stores tokens in 4-bit format, retrieves relevant groups on demand, and dequantizes them, keeping the active kv-cache bounded regardless of stream length. Experiments demonstrate our method achieves 15.7x kv-cache compression ratio; compared to prior SOTA (LiveVLM), it delivers 1.2x lower peak memory and 2x faster TTFT. StreamingTOM achieves state-of-the-art accuracy among training-free methods with an average of 63.8% on offline benchmarks and 55.8% accuracy and 3.7 score on RVS. These results demonstrate that real-time streaming video understanding with bounded active memory is achievable without model retraining.