及时一针：通过自监督 Plackett-Luce 排序学习程序性工作流
A Stitch in Time: Learning Procedural Workflow via Self-Supervised Plackett-Luce Ranking
PL-Stitch 利用视频帧的固有时间顺序作为自监督信号，通过 Plackett-Luce 排序损失和时空拼图损失学习程序性活动的表征，可迁移到需要时序推理的视频理解任务中。
原文依据 we propose PL-Stitch, a self-supervised framework that harnesses the inherent temporal order of video frames as a powerful supervisory signal.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论程序性活动，从日常烹饪到复杂的外科手术，都是按特定时序执行的高度结构化的动作序列。尽管当前的自监督学习（SSL）方法在静态图像和短视频片段上取得了成功，但这些模型往往忽略了此类活动底层的序列结构。我们通过一个激励性实验揭示了这种程序性意识的缺失：在正向和时间反转序列上预训练的模型产生高度相似的特征，证实了它们的表征对底层程序顺序是盲目的。为了解决这一缺陷，我们提出了 PL-Stitch，一个自监督框架，利用视频帧固有的时序顺序作为强大的监督信号。我们的方法集成了两个基于 Plackett-Luce（PL）模型的新颖概率目标。主要的 PL 目标训练模型按时间顺序对采样帧进行排序，迫使其学习全局工作流进展。次要目标，即时空拼图损失，通过捕捉细粒度的跨帧物体对应关系来补充学习。我们的方法在五个外科手术和烹饪基准上持续取得优越性能。具体而言，PL-Stitch 在外科手术阶段识别（例如，在 Cholec80 上 k-NN 准确率提升 11.4 个百分点）和烹饪动作分割（例如，在 Breakfast 上线性探测准确率提升 5.7 个百分点）方面取得了显著增益，证明了其对于程序性视频表征学习的有效性。代码和模型可在 https://github.com/visurg-ai/PL-Stitch 获取。
英文原文摘要 1620 字符
Procedural activities, ranging from routine cooking to complex surgical operations, are highly structured sequences of actions performed in a specific temporal order. Despite the success of current self-supervised learning (SSL) methods on static images and short clips, these models often overlook the underlying sequential structure of such activities. We expose this lack of procedural awareness with a motivating experiment: models pretrained on forward and time-reversed sequences produce highly similar features, confirming that their representations are blind to the underlying procedural order. To address this shortcoming, we propose PL-Stitch, a self-supervised framework that harnesses the inherent temporal order of video frames as a powerful supervisory signal. Our approach integrates two novel probabilistic objectives based on the Plackett-Luce (PL) model. The primary PL objective trains the model to sort sampled frames chronologically, compelling it to learn the global workflow progression. The secondary objective, a spatio-temporal jigsaw loss, complements the learning by capturing fine-grained, cross-frame object correspondences. Our approach consistently achieves superior performance across five surgical and cooking benchmarks. Specifically, PL-Stitch yields significant gains in surgical phase recognition (e.g., +11.4 pp in k-NN accuracy on Cholec80) and cooking action segmentation (e.g., +5.7 pp in linear probing accuracy on Breakfast), demonstrating its effectiveness for procedural video representation learning. Code and models are available at https://github.com/visurg-ai/PL-Stitch.