HTTM：用于加速 VGGT 的逐头时序词元合并
HTTM: Head-wise Temporal Token Merging for Faster VGGT
HTTM 提出一种免训练的词元合并方法，利用多头注意力中每个头的空间局部性和时序对应关系，在保持特征独特性的同时大幅降低 VGGT 的推理延迟，可复用于其他基于全局注意力的 3D 重建或视频理解模型。
原文依据 we propose head-wise temporal merging (HTTM), a training-free 3D token merging method for accelerating VGGT
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉几何基础 Transformer（VGGT）在三维场景重建方面取得了重大飞跃，因为它是首个能够一次性联合推断所有关键三维属性（相机位姿、深度和稠密几何）的模型。然而，这种联合推断机制需要全局注意力层，对所有视图的词元执行全对全的注意力计算。对于长序列输入的大场景重建，这会导致显著的延迟瓶颈。本文提出逐头时序合并（HTTM），一种免训练的 3D 词元合并方法，用于加速 VGGT。现有的合并技术在不同注意力头之间均匀地合并词元，导致层输出中出现相同的词元，从而阻碍了模型的表征能力。HTTM 通过在多头粒度上合并词元来解决这个问题，在头拼接后保持了特征词元的独特性。此外，这使得 HTTM 能够利用在头级别观察到的空间局部性和时序对应关系，与现有方法相比，以更低的合并成本实现更高的合并比率。因此，HTTM 在基于 GPU 的推理中实现了高达 7 倍的加速，而性能下降可忽略不计。
英文原文摘要 1263 字符
The Visual Geometry Grounded Transformer (VGGT) marks a significant leap forward in 3D scene reconstruction, as it is the first model that directly infers all key 3D attributes (camera poses, depths, and dense geometry) jointly in one pass. However, this joint inference mechanism requires global attention layers that perform all-to-all attention computation on tokens from all views. For reconstruction of large scenes with long-sequence inputs, this causes a significant latency bottleneck. In this paper, we propose head-wise temporal merging (HTTM), a training-free 3D token merging method for accelerating VGGT. Existing merging techniques merge tokens uniformly across different attention heads, resulting in identical tokens in the layers' output, which hinders the model's representational ability. HTTM tackles this problem by merging tokens in multi-head granularity, which preserves the uniqueness of feature tokens after head concatenation. Additionally, this enables HTTM to leverage the spatial locality and temporal correspondence observed at the head level to achieve higher merging ratios with lower merging costs compared to existing methods. Thus, HTTM achieves up to 7x acceleration with negligible performance drops in a GPU-based inference.