面向弱监督动作分割的层次化动作学习
Hierarchical Action Learning for Weakly-Supervised Action Segmentation
论文提出层次化因果生成过程与稀疏转移约束，利用高层动作潜变量变化慢的特性来抑制过分割，这种时序潜变量对齐思路可迁移到其他视频时序任务（如动作识别、时序动作定位）中。
原文依据 we propose the Hierarchical Action Learning (HAL) model for weakly-supervised action segmentation
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论人类通过关键过渡来感知动作，这些过渡在多个抽象层次上构建动作，而依赖视觉特征的机器则倾向于过度分割。这凸显了在视频理解中实现层次化推理的困难。有趣的是，我们观察到低层视觉变量和高层动作潜变量以不同的速率演化，低层视觉变量变化迅速，而高层动作变量演化较慢，使其更容易被识别。基于这一洞察，我们提出了用于弱监督动作分割的层次化动作学习（HAL）模型。我们的方法引入了一个层次化因果数据生成过程，其中高层潜动作支配低层视觉特征的动态。为了有效建模这些不同的时间尺度，我们引入了确定性过程来随时间对齐这些潜变量。HAL模型采用层次化金字塔Transformer来同时捕捉视觉特征和潜变量，并应用稀疏转移约束来强制高层动作变量的较慢动态。这一机制增强了这些潜变量随时间的可识别性。在温和假设下，我们证明了这些潜动作变量是严格可识别的。在多个基准上的实验结果表明，HAL模型显著优于现有的弱监督动作分割方法，证实了其在实际应用中的有效性。
英文原文摘要 1535 字符
Humans perceive actions through key transitions that structure actions across multiple abstraction levels, whereas machines, relying on visual features, tend to over-segment. This highlights the difficulty of enabling hierarchical reasoning in video understanding. Interestingly, we observe that lower-level visual and high-level action latent variables evolve at different rates, with low-level visual variables changing rapidly, while high-level action variables evolve more slowly, making them easier to identify. Building on this insight, we propose the Hierarchical Action Learning (HAL) model for weakly-supervised action segmentation. Our approach introduces a hierarchical causal data generation process, where high-level latent action govern the dynamics of low-level visual features. To model these varying timescales effectively, we introduce deterministic processes to align these latent variables over time. The HAL model employs a hierarchical pyramid transformer to capture both visual features and latent variables, and a sparse transition constraint is applied to enforce the slower dynamics of high-level action variables. This mechanism enhances the identification of these latent variables over time. Under mild assumptions, we prove that these latent action variables are strictly identifiable. Experimental results on several benchmarks show that the HAL model significantly outperforms existing methods for weakly-supervised action segmentation, confirming its practical effectiveness in real-world applications.