超越静态帧：用于人体姿态估计的时序聚合与恢复视觉 Transformer
Beyond Static Frames: Temporal Aggregate-and-Restore Vision Transformer for Human Pose Estimation
提出一种即插即用的时序聚合与恢复模块，可增强现有单帧 ViT 姿态估计器，利用相邻帧信息提升运动模糊、遮挡等挑战场景下的关键点定位稳定性。
原文依据 TAR-ViTPose enhances static ViT representations by aggregating temporal cues across frames in a plug-and-play manner
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉 Transformer（ViT）凭借强大的全局建模能力，近期在二维人体姿态估计中取得了最优性能。然而，现有的基于 ViT 的姿态估计器面向静态图像设计，独立处理每一帧，因而忽略了视频序列中存在的时序一致性。这一局限常导致预测不稳定，在涉及运动模糊、遮挡或失焦等挑战性场景中尤为明显。本文提出 TAR-ViTPose，一种专为基于视频的二维人体姿态估计设计的新型时序聚合与恢复视觉 Transformer。TAR-ViTPose 以即插即用的方式聚合跨帧时序线索，增强静态 ViT 表征，从而获得更鲁棒、更准确的姿态估计。为有效聚合跨帧时序对齐的关节特定特征，我们引入以关节为中心的时序聚合（JTA），为每个关节分配一个可学习查询词元，使其选择性地关注相邻帧中的对应区域。此外，我们开发了全局恢复注意力（GRA），将聚合后的时序特征恢复至当前帧的词元序列中，在充分保留全局上下文以精确定位关键点的同时，丰富其姿态表征。大量实验表明，TAR-ViTPose 显著优于单帧基线 ViTPose，在 PoseTrack2017 基准上取得 +2.3 平均精度均值的提升。此外，我们的方法优于现有最优的基于视频的方法，同时在实际应用中实现了明显更高的运行时帧率。项目页面：https://github.com/zgspose/TARViTPose。
英文原文摘要 1720 字符
Vision Transformers (ViTs) have recently achieved state-of-the-art performance in 2D human pose estimation due to their strong global modeling capability. However, existing ViT-based pose estimators are designed for static images and process each frame independently, thereby ignoring the temporal coherence that exists in video sequences. This limitation often results in unstable predictions, especially in challenging scenes involving motion blur, occlusion, or defocus. In this paper, we propose TAR-ViTPose, a novel Temporal Aggregate-and-Restore Vision Transformer tailored for video-based 2D human pose estimation. TAR-ViTPose enhances static ViT representations by aggregating temporal cues across frames in a plug-and-play manner, leading to more robust and accurate pose estimation. To effectively aggregate joint-specific features that are temporally aligned across frames, we introduce a joint-centric temporal aggregation (JTA) that assigns each joint a learnable query token to selectively attend to its corresponding regions from neighboring frames. Furthermore, we develop a global restoring attention (GRA) to restore the aggregated temporal features back into the token sequence of the current frame, enriching its pose representation while fully preserving global context for precise keypoint localization. Extensive experiments demonstrate that TAR-ViTPose substantially improves upon the single-frame baseline ViTPose, achieving a +2.3 mAP gain on the PoseTrack2017 benchmark. Moreover, our approach outperforms existing state-of-the-art video-based methods, while also achieving a noticeably higher runtime frame rate in real-world applications. Project page: https://github.com/zgspose/TARViTPose.