STAC：面向流式三维重建的即插即用时空感知缓存压缩
STAC: Plug-and-Play Spatio-Temporal Aware Cache Compression for Streaming 3D Reconstruction
针对因果 VGGT 类流式三维重建中 KV 缓存随流长线性增长的问题，利用注意力时空稀疏性做词元级缓存淘汰与体素对齐压缩，可直接迁移到任何使用长上下文 KV 缓存的视觉 Transformer 流式推理场景。
原文依据 attention in causal transformers for 3D reconstruction exhibits intrinsic spatio-temporal sparsity
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论从流式输入进行在线三维重建，既需要长期的时序一致性，也需要高效的内存占用。尽管 VGGT 的因果变体通过键值（KV）缓存机制应对这一挑战，但缓存会随流长度线性增长，造成严重的内存瓶颈。在有限的内存预算下，过早的缓存淘汰会显著降低重建质量与时序一致性。在本工作中，我们观察到用于三维重建的因果 Transformer 中的注意力呈现出内在的时空稀疏性。基于这一洞察，我们提出 STAC，一个面向大因果 Transformer 流式三维重建的时空感知缓存压缩框架。STAC 包含三个关键组件：（1）工作时序词元缓存机制，利用衰减累积注意力分数保留长期有信息的词元；（2）长期空间词元缓存方案，将空间冗余词元压缩为体素对齐表示，以实现内存高效的存储；（3）基于分块的多帧优化策略，联合处理连续帧以提升时序连贯性与 GPU 效率。大量实验表明，STAC 在达到最优重建质量的同时，将内存消耗降低近 10 倍、推理加速 4 倍，大幅提升了流式场景下实时三维重建的可扩展性。
英文原文摘要 1449 字符
Online 3D reconstruction from streaming inputs requires both long-term temporal consistency and efficient memory usage. Although causal variants of VGGT address this challenge through a key-value (KV) cache mechanism, the cache grows linearly with the stream length, creating a major memory bottleneck. Under limited memory budgets, early cache eviction significantly degrades reconstruction quality and temporal consistency. In this work, we observe that attention in causal transformers for 3D reconstruction exhibits intrinsic spatio-temporal sparsity. Based on this insight, we propose STAC, a Spatio-Temporally Aware Cache Compression framework for streaming 3D reconstruction with large causal transformers. STAC consists of three key components: (1) a Working Temporal Token Caching mechanism that preserves long-term informative tokens using decayed cumulative attention scores; (2) a Long-term Spatial Token Caching scheme that compresses spatially redundant tokens into voxel-aligned representations for memory-efficient storage; and (3) a Chunk-based Multi-frame Optimization strategy that jointly processes consecutive frames to improve temporal coherence and GPU efficiency. Extensive experiments show that STAC achieves state-of-the-art reconstruction quality while reducing memory consumption by nearly 10x and accelerating inference by 4x, substantially improving the scalability of real-time 3D reconstruction in streaming settings.