独特人生，共享世界：从单人生视频中学习
Unique Lives, Shared World: Learning from Single-Life Videos
提出仅用单个人第一人称视频进行自监督训练视觉编码器的范式，并设计跨注意力指标量化不同模型内部表征的功能对齐，为利用无标注个人视频学习通用几何表征提供了新思路。
原文依据 we train a distinct vision model exclusively on egocentric videos captured by one individual
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论我们引入了“单人生”学习范式，即仅使用某一个人拍摄的第一人称视频来训练一个独特的视觉模型。我们利用单个人生中自然捕获的多个视角，以自监督方式学习视觉编码器。实验展示了三个关键发现。第一，在不同人生数据上独立训练的模型发展出高度对齐的几何理解。我们通过在分别捕获不同人生（包括室内和室外）的不同数据集上训练视觉编码器，并引入一种新的基于跨注意力的指标来量化不同模型内部表征的功能对齐，以此证明这一点。第二，我们表明单人生模型学习到可泛化的几何表征，能有效迁移到下游任务（如深度估计）并在未见环境中表现良好。第三，我们证明，使用同一个人一周内长达30小时的数据进行训练，其性能与使用30小时多样化网络数据训练相当，突显了单人生表征学习的优势。总体而言，我们的结果确立了世界的共享结构既使得在个体人生上训练的模型具有一致性，也为视觉表征学习提供了强大的信号。
英文原文摘要 1345 字符
We introduce the "single-life" learning paradigm, where we train a distinct vision model exclusively on egocentric videos captured by one individual. We leverage the multiple viewpoints naturally captured within a single life to learn a visual encoder in a self-supervised manner. Our experiments demonstrate three key findings. First, models trained independently on different lives develop a highly aligned geometric understanding. We demonstrate this by training visual encoders on distinct datasets each capturing a different life, both indoors and outdoors, as well as introducing a novel cross-attention-based metric to quantify the functional alignment of the internal representations developed by different models. Second, we show that single-life models learn generalizable geometric representations that effectively transfer to downstream tasks, such as depth estimation, in unseen environments. Third, we demonstrate that training on up to 30 hours from one week of the same person's life leads to comparable performance to training on 30 hours of diverse web data, highlighting the strength of single-life representation learning. Overall, our results establish that the shared structure of the world, both leads to consistency in models trained on individual lives, and provides a powerful signal for visual representation learning.