面向电影预告片生成的自步自纠正掩码预测
Self-Paced and Self-Corrective Masked Prediction for Movie Trailer Generation
把「掩码预测 + 自步掩码比例 + 逐步高置信填充再重掩码」的自纠正解码思路引入视频镜头序列生成，可迁移到视频摘要、镜头排序、故事线重组等时序编辑任务，替代易误差传播的「先选后排序」两阶段范式。
原文依据 the model fills the shot positions with high confidence at each step and re-masks the remaining positions for the next prediction
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论作为一项具有挑战性的视频编辑任务，电影预告片生成需要挑选并重新组织电影镜头，以制作出引人入胜的预告片。目前，大多数已有的自动预告片生成方法采用「先选择后排序」的范式（即先挑选关键镜头再对其排序），该范式存在不可避免的误差传播问题，并限制了所生成预告片的质量。在这一范式之外，我们提出了一种新的自步且自纠正的掩码预测方法 SSMP，它通过双向上下文建模与渐进式自纠正，在自动预告片生成上取得了最优结果。具体而言，SSMP 训练一个 Transformer 编码器，该编码器以电影镜头序列为提示，并据此生成相应的预告片镜头序列。模型通过掩码预测进行训练，即从随机掩码后的版本中重建每个预告片镜头序列。掩码比例是自步的，使任务难度能够自适应于模型，从而提升模型性能。在生成电影预告片时，模型在每一步以高置信度填充镜头位置，并对剩余位置重新掩码以进行下一次预测，形成一种渐进式自纠正机制，这与人类剪辑师的工作方式类似。定量结果与用户研究均表明，SSMP 相较于已有的自动电影预告片生成方法具有优越性。演示与代码见：https://github.com/Dixin-Lab/SSMP。
英文原文摘要 1528 字符
As a challenging video editing task, movie trailer generation involves selecting and reorganizing movie shots to create engaging trailers. Currently, most existing automatic trailer generation methods employ a "selection-then-ranking" paradigm (i.e., first selecting key shots and then ranking them), which suffers from inevitable error propagation and limits the quality of the generated trailers. Beyond this paradigm, we propose a new self-paced and self-corrective masked prediction method called SSMP, which achieves state-of-the-art results in automatic trailer generation via bi-directional contextual modeling and progressive self-correction. In particular, SSMP trains a Transformer encoder that takes the movie shot sequences as prompts and generates corresponding trailer shot sequences accordingly. The model is trained via masked prediction, reconstructing each trailer shot sequence from its randomly masked counterpart. The mask ratio is self-paced, allowing the task difficulty to adapt to the model and thereby improving model performance.When generating a movie trailer, the model fills the shot positions with high confidence at each step and re-masks the remaining positions for the next prediction, forming a progressive self-correction mechanism that is analogous to how human editors work. Both quantitative results and user studies demonstrate the superiority of SSMP in comparison to existing automatic movie trailer generation methods. Demo and code are available at: https://github.com/Dixin-Lab/SSMP.