面向高效多视图几何 Transformer 的块稀疏全局注意力
Block-Sparse Global Attention for Efficient Multi-View Geometry Transformers
把稠密全局注意力换成免训练的块稀疏注意力，可直接插到 VGGT、π³、MapAnything 这类前馈多视图重建 Transformer 中，在几乎不掉性能的前提下把大图集推理加速 3 倍以上，是处理大规模多视图输入时一个即插即用的效率手段。
原文依据 we propose a training-free, block-sparse replacement for dense global attention, implemented with highly optimized kernels
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论高效且准确的前馈多视图重建长期以来是计算机视觉中的一项重要任务。近期 VGGT、π³ 和 MapAnything 等基于 Transformer 的模型以相对简单的架构取得了显著性能。然而，其可扩展性从根本上受限于全局注意力的二次复杂度，在处理大规模图像集时造成严重的运行时瓶颈。在本工作中，我们实证分析了这些模型的全局注意力矩阵，观察到概率质量集中于与跨视图几何对应关系相关的一小部分 patch 间交互上。基于这一观察，并受大语言模型近期进展的启发，我们提出了一种免训练的块稀疏全局注意力替代方案，用以取代稠密全局注意力，并采用高度优化的内核实现。我们的方法在保持相当任务性能的同时，将推理加速 3 倍以上。在一整套全面的多视图基准上的评测表明，我们的方法可以无缝集成到 VGGT、π³ 和 MapAnything 等现有基于全局注意力的架构中，同时显著提升对大规模图像集合的可扩展性。
英文原文摘要 1251 字符
Efficient and accurate feed-forward multi-view reconstruction has long been an important task in computer vision. Recent transformer-based models like VGGT, \pi^3 and MapAnything have demonstrated remarkable performance with relatively simple architectures. However, their scalability is fundamentally constrained by the quadratic complexity of global attention, which imposes a significant runtime bottleneck when processing large image sets. In this work, we empirically analyze the global attention matrix of these models and observe that the probability mass concentrates on a small subset of patch-patch interactions corresponding to cross-view geometric correspondences. Building on this insight and inspired by recent advances in large language models, we propose a training-free, block-sparse replacement for dense global attention, implemented with highly optimized kernels. Our method accelerates inference by more than 3xwhile maintaining comparable task performance. Evaluations on a comprehensive suite of multi-view benchmarks demonstrate that our approach seamlessly integrates into existing global attention-based architectures such as VGGT, \pi^3, and MapAnything, while substantially improving scalability to large image collections.