EgoPoseFormer v2：面向AR/VR的精确第一人称人体运动估计
EgoPoseFormer v2: Accurate Egocentric Human Motion Estimation for AR/VR
提出身份条件查询、多视角空间精炼和因果时序注意力，在恒定计算预算下实现时序一致且空间对齐的人体姿态估计；其不确定性感知的半监督自标注系统可复用于其他标注稀缺的视觉任务。
原文依据 identity-conditioned queries, multi-view spatial refinement, causal temporal attention
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论第一人称3D人体运动估计对AR/VR体验至关重要，但由于第一人称视角下身体覆盖范围有限、频繁遮挡以及标注数据稀缺，该任务仍具挑战性。我们提出EgoPoseFormer v2，通过两项关键贡献应对这些挑战：(1) 一个基于Transformer的模型，用于时序一致且空间对齐的人体姿态估计；(2) 一个自标注系统，能够利用大规模无标注数据集进行训练。所提模型完全可微，引入身份条件查询、多视角空间精炼和因果时序注意力，并在恒定计算预算下支持关键点和参数化人体表示。所提自标注系统通过不确定性感知的半监督训练将学习扩展到数千万无标注帧。该系统遵循教师-学生范式生成伪标签，并通过不确定性蒸馏指导训练，使模型能够泛化到不同环境。在EgoBody3M基准上的实验中，EgoPoseFormer v2在准确率上分别优于两种最优方法12.2%和19.4%，并将时序抖动分别降低22.2%和51.7%。此外，我们的自标注系统还将手腕MPJPE进一步提升了13.1%。
英文原文摘要 1337 字符
Egocentric 3D human motion estimation is essential for AR/VR experiences, yet remains challenging due to limited body coverage from the egocentric viewpoint, frequent occlusions, and scarce labeled data. We present EgoPoseFormer v2, a method that addresses these challenges through two key contributions: (1) a transformer-based model for temporally consistent and spatially grounded body pose estimation, and (2) an auto-labeling system that enables the use of large unlabeled datasets for training.The proposed model is fully differentiable, introduces identity-conditioned queries, multi-view spatial refinement, causal temporal attention, and supports both keypoints and parametric body representations under a constant compute budget.The proposed auto-labeling system scales learning to tens of millions of unlabeled frames via uncertainty-aware semi-supervised training. The system follows a teacher-student schema to generate pseudo-labels and guide training with uncertainty distillation, enabling the model to generalize to different environments. In experiments on the EgoBody3M benchmark, EgoPoseFormer v2 outperforms two state-of-the-art methods by 12.2% and 19.4% in accuracy, and reduces temporal jitter by 22.2% and 51.7%, respectively. Furthermore, our auto-labeling system additionally improves the wrist MPJPE by 13.1%.