ViterbiPlanNet：通过可微分Viterbi为教学视频规划注入程序性知识
ViterbiPlanNet: Injecting Procedural Knowledge via Differentiable Viterbi for Planning in Instructional Videos
将程序性知识图谱通过可微分Viterbi层嵌入端到端训练，为视频中的动作序列规划提供了一种结构感知且样本高效的方案，可复用于需要时序推理的视觉任务。
原文依据 explicitly integrates procedural knowledge into the learning process through a Differentiable Viterbi Layer (DVL)
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论程序性规划旨在预测一系列动作，将初始视觉状态转换为期望目标，这是在复杂环境中运行的智能体的基本能力。现有方法通常依赖大规模模型隐式学习程序结构，导致样本效率有限且计算成本高昂。本文提出ViterbiPlanNet，一个通过可微分Viterbi层（DVL）将程序性知识显式集成到学习过程中的原则性框架。DVL利用Viterbi解码算法直接嵌入程序性知识图谱（PKG），用平滑松弛替代不可微操作，从而实现端到端优化。该设计允许模型通过基于图的解码进行学习。在CrossTask、COIN和NIV上的实验表明，ViterbiPlanNet以比基于扩散和LLM的规划器少一个数量级的参数实现了最优性能。大量消融实验表明，性能提升源于我们可微的结构感知训练而非事后精炼，从而提高了样本效率和对更短未见过视野的鲁棒性。我们还解决了测试不一致性问题，建立了具有一致划分和评估指标的统一测试协议。借助这一新协议，我们多次运行实验并使用自助法报告结果以评估统计显著性。
英文原文摘要 1491 字符
Procedural planning aims to predict a sequence of actions that transforms an initial visual state into a desired goal, a fundamental ability for intelligent agents operating in complex environments. Existing approaches typically rely on large-scale models that learn procedural structures implicitly, resulting in limited sample-efficiency and high computational cost. In this work we introduce ViterbiPlanNet, a principled framework that explicitly integrates procedural knowledge into the learning process through a Differentiable Viterbi Layer (DVL). The DVL embeds a Procedural Knowledge Graph (PKG) directly with the Viterbi decoding algorithm, replacing non-differentiable operations with smooth relaxations that enable end-to-end optimization. This design allows the model to learn through graph-based decoding. Experiments on CrossTask, COIN, and NIV demonstrate that ViterbiPlanNet achieves state-of-the-art performance with an order of magnitude fewer parameters than diffusion- and LLM-based planners. Extensive ablations show that performance gains arise from our differentiable structure-aware training rather than post-hoc refinement, resulting in improved sample efficiency and robustness to shorter unseen horizons. We also address testing inconsistencies establishing a unified testing protocol with consistent splits and evaluation metrics. With this new protocol, we run experiments multiple times and report results using bootstrapping to assess statistical significance.