第一人称视频中的 3D 扫描路径预测
Forecasting 3D Scanpaths in Egocentric Video
将注视点预测从 2D 图像序列扩展到第一人称视频中的 3D 笛卡尔坐标序列，利用 Transformer 融合视频帧、头部姿态和过去 3D 注视点，为 AR/VR 中的意图预测和交互提供可复用的时序建模框架。
原文依据 we formulate the novel task of egocentric scanpath prediction as forecasting a sequence of future fixations in 3D Cartesian coordinates
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论预测注视行为是理解用户意图以及构建能够预判用户下一步看向何处并进行交互的 AR/VR 系统的重要任务。尽管已有工作研究了静态图像中的扫描路径预测，但在第一人称视频中预测注视面临新的挑战，因为场景是动态的，且相机佩戴者持续在三维环境中移动。为应对这些挑战，我们将第一人称扫描路径预测定义为一个新任务：预测相对于最后一次观测到的相机位姿的一系列未来注视点的三维笛卡尔坐标序列，从而生成一个以环境为基准的 3D 扫描路径。我们提出了一种 Transformer 架构，利用第一人称视频帧、头部姿态和过去的 3D 注视观测来预测未来的 3D 注视序列。我们在 Aria Digital Twin 数据集上评估了我们的方法。我们的发现为 3D 扫描路径预测这一新任务建立了基线，并突出了该任务中重要的架构要素。
英文原文摘要 1035 字符
Forecasting gaze behavior is an important task for understanding user intent and creating AR/VR systems that can anticipate where users will look and interact next. While prior works have addressed predicting scanpaths in static images, forecasting gaze in egocentric videos presents new challenges due to the dynamic nature of the scene and the camera wearer's continuous movement through the 3D environment. To address these challenges, we formulate the novel task of egocentric scanpath prediction as forecasting a sequence of future fixations in 3D Cartesian coordinates relative to the last observed camera pose, producing a 3D scanpath that is grounded in the environment. We propose a transformer architecture that leverages egocentric video frames, head pose, and past 3D gaze observations to predict future 3D fixation sequences. We evaluate our method on the Aria Digital Twin dataset. Our findings establish a baseline for the novel task of 3D scanpath prediction and highlight important architectural elements for our task.