SkillSight：基于注视的高效第一人称技能评估
SkillSight: Efficient First-Person Skill Assessment with Gaze
把注视信号作为第一人称视频理解的额外模态，并用教师-学生蒸馏把多模态模型压缩成仅注视输入的轻量模型，为可穿戴设备上的低功耗视觉感知提供了可复用的范式。
原文依据 distills a gaze-only student model
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论智能眼镜上的第一人称感知有望改变我们在物理世界中学习新技能的方式，但自动技能评估仍是一项基础性技术挑战。我们提出 SkillSight，用于从第一人称数据中进行低功耗技能评估。我们方法的核心假设是：技能水平不仅体现在一个人如何完成某项活动（视频）中，也体现在其完成活动时如何分配注意力（注视）中。我们的两阶段框架首先学习在预测技能水平时联合建模注视与第一人称视频，然后蒸馏出一个仅使用注视的学生模型。在推理阶段，学生模型仅需注视输入，通过免除连续视频处理而大幅降低功耗。在涵盖烹饪、音乐和体育的三个数据集上的实验首次确立了注视在多样真实场景中理解技能方面的重要作用。我们的 SkillSight 教师模型达到最优性能，而仅使用注视的学生变体在保持高精度的同时，功耗比竞争方法低 73 倍。这些结果为在真实环境中由 AI 支持的技能学习铺平了道路。
英文原文摘要 1168 字符
Egocentric perception on smart glasses could transform how we learn new skills in the physical world, but automatic skill assessment remains a fundamental technical challenge. We introduce SkillSight for power-efficient skill assessment from first-person data. Central to our approach is the hypothesis that skill level is evident not only in how a person performs an activity (video), but also in how they directtheir attention when doing so (gaze). Our two-stage framework first learns to jointly model gaze and egocentric video when predicting skill level, then distills a gaze-only student model. At inference, the student model requires only gaze input, drastically reducing power consumption by eliminating continuous video processing. Experiments on three datasets spanning cooking, music, and sports establish, for the first time, the valuable role of gaze in skill understanding across diverse real-world settings. Our SkillSight teacher model achieves state-of-the-art performance, while our gaze-only student variant maintains high accuracy using 73x less power than competing methods. These results pave the way for in-the-wild AI-supported skill learning.