TTAPFormer：通过瞬态异步融合帧与事件实现鲁棒的任意点跟踪
TTAPFormer: Robust Arbitrary Point Tracking via Transient Asynchronous Fusion of Frames and Events
提出瞬态异步融合机制，用连续事件流更新离散帧间的时序演化，解决帧-事件异步融合的时序错位问题；跨模态局部加权融合模块根据模态可靠性自适应调整空间注意力，为多模态融合在模糊/低光下的鲁棒性提供可复用思路。
原文依据 Transient Asynchronous Fusion (TAF) mechanism, which explicitly models the temporal evolution between discrete frames through continuous event updates
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论任意点跟踪（TAP）是计算机视觉中一项基础但具有挑战性的任务，要求高精度和长期运动推理。近期结合RGB帧与事件流的尝试已显示出潜力，但它们通常依赖同步或非自适应融合，导致时序错位，并在某一模态失效时出现严重性能下降。我们提出TAPFormer，一个基于Transformer的框架，对帧和事件执行异步时序一致融合，以实现鲁棒且高频率的任意点跟踪。我们的核心创新是瞬态异步融合（TAF）机制，它通过连续事件更新显式建模离散帧之间的时序演化——弥合低速率帧与高速率事件之间的差距。此外，跨模态局部加权融合（CLWF）模块根据模态可靠性自适应调整空间注意力，即使在模糊或低光条件下也能产生稳定且具有判别力的特征。为了在真实条件下评估我们的方法，我们构建了一个新颖的真实世界帧-事件TAP数据集，涵盖多样化的光照和运动条件。我们的方法优于现有的点跟踪器，在阈值内的平均像素误差上实现了28.2%的改进。此外，在标准点跟踪基准上，我们的跟踪器始终取得最佳性能。我们将在接收后发布代码和数据集，以支持未来研究。
英文原文摘要 1455 字符
Tracking any point (TAP) is a fundamental yet challenging task in computer vision, requiring high precision and long-term motion reasoning. Recent attempts to combine RGB frames and event streams have shown promise, yet they typically rely on synchronous or non-adaptive fusion, leading to temporal misalignment and severe degradation when one modality fails. We introduce TAPFormer, a transformer-based framework that performs asynchronous temporal-consistent fusion of frames and events for robust and high-frequency arbitrary point tracking.Our key innovation is a Transient Asynchronous Fusion (TAF) mechanism, which explicitly models the temporal evolution between discrete frames through continuous event updates--bridging the gap between low-rate frames and high-rate events. In addition, a Cross-modal Locally Weighted Fusion (CLWF) module adaptively adjusts spatial attention according to modality reliability, yielding stable and discriminative features even under blur or low light.To evaluate our approach under realistic conditions, we construct a novel real-world frame-event TAP dataset under diverse illumination and motion conditions.Our method outperforms existing point trackers, achieving a 28.2% improvement in average pixel error within threshold. Moreover, on standard point tracking benchmarks, our tracker consistently achieves the best performance. We will release the code and dataset upon acceptance to support future research.