通过预测随时间变化的三维高斯进行跟踪
Tracking by Predicting 3-D Gaussians Over Time
把视频编码为一组随时间运动的高斯泼溅，用重建式自监督预训练让跟踪能力自发涌现；做视频自监督表征或零样本点跟踪的研究者可直接复用这一归纳偏置与高斯轨迹投影到图像平面的读出方式。
原文依据 We find that tracking emerges when pre-training a network with this architecture.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论我们提出视频高斯掩码自编码器（Video Gaussian Masked Autoencoders, Video-GMAE），这是一种自监督表征学习方法，将一段图像序列编码为一组随时间运动的高斯泼溅。将视频表示为高斯集合引入了一种合理的归纳偏置：二维视频往往是动态三维场景的一致投影。我们发现，以该架构进行预训练时，跟踪能力会自发涌现。将所学高斯的轨迹映射到图像平面，可获得与当前最优方法相当的零样本跟踪性能。经过小规模微调，我们的模型在 Kinetics 上取得 34.6% 的提升，在 Kubric 数据集上取得 13.1% 的提升，超越了现有的自监督视频方法。
英文原文摘要 732 字符
We propose Video Gaussian Masked Autoencoders (Video-GMAE), a self-supervised approach for representation learning that encodes a sequence of images into a set of Gaussian splats moving over time. Representing a video as a set of Gaussians enforces a reasonable inductive bias: that 2-D videos are often consistent projections of a dynamic 3-D scene. We find that tracking emerges when pre-training a network with this architecture. Mapping the trajectory of the learnt Gaussians onto the image plane gives zero-shot tracking performance comparable to state-of-the-art. With small-scale finetuning, our models achieve 34.6% improvement on Kinetics, and 13.1% on Kubric datasets, surpassing existing self-supervised video approaches.