TimeBridge：基于首尾联合嵌入与中间帧预测的自监督视频表征学习
TimeBridge: Self-Supervised Video Representation Learning via Start-End Joint Embedding and In-Between Frame Prediction
用首尾帧的联合嵌入加轻量解码器重建中间帧，把「时序变换建模」变成可复用的自监督预训练目标，可直接迁移到视频分割、部件传播等密集预测任务的特征初始化。
原文依据 combines the joint embedding for video representation with learning temporal transformations by reconstructing in-between frames from only the start and end frames
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论学习时序变换，即视觉对象如何跨帧演化，是视频表征学习中的一项基础性挑战。帧间动态涉及复杂、非线性且非局部的变化，远超常规的空间数据增强所能覆盖。本文提出 TimeBridge，一种自监督方法，将视频表征的联合嵌入与仅由首帧和末帧重建中间帧的时序变换学习相结合。该形式促使模型推断连接两个端点的时序演化，而非仅仅编码静态帧表征。与缺乏显式变换建模的联合嵌入方法、或依赖无约束外推的未来预测目标不同，TimeBridge 通过促进时序一致性来学习具体的帧间动态。我们借助交叉拼接的类别 token 与轻量解码器实现这一点，将首帧与末帧的特征重新组合以重建中间帧。TimeBridge 在多个密集视频预测基准上取得新的最优性能，包括在 DAVIS 2017 视频目标分割上取得 73.5 J&F，在 VIP 语义部件传播上取得 47.5 平均交并比。
英文原文摘要 1255 字符
Learning temporal transformations, that is, how visual objects evolve across frames, is a fundamental challenge in video representation learning. Frame-to-frame dynamics involve complex, non-linear, and non-local changes that go far beyond conventional spatial augmentations. We propose TimeBridge, a self-supervised method that combines the joint embedding for video representation with learning temporal transformations by reconstructing in-between frames from only the start and end frames. This formulation encourages the model to infer the temporal evolution bridging the two endpoints, rather than merely encoding static frame representations. Unlike joint-embedding methods that lack explicit transformation modelling or future-prediction objectives that rely on unconstrained extrapolation, TimeBridge learns concrete frame-to-frame dynamics by promoting temporal consistency. We realise this through cross-concatenated class tokens and lightweight decoders, which recombine features from the start and end frames to reconstruct intermediates. TimeBridge achieves new state-of-the-art performance on multiple dense video prediction benchmarks, including 73.5 J&F on DAVIS 2017 video object segmentation, 47.5 mIoU on VIP semantic part propagation.