SAIL：面向弱监督稠密视频描述学习的相似度感知引导与描述间增强
SAIL: Similarity-Aware Guidance and Inter-Caption Augmentation-based Learning for Weakly-Supervised Dense Video Captioning
用跨模态相似度作为弱监督信号来引导高斯掩码聚焦语义相关区域，并用 LLM 合成描述通过掩码间机制提供辅助对齐信号，这套思路可直接迁移到其他只有粗粒度标注的视频时序定位任务（如弱监督动作定位、视频片段检索）。
原文依据 constructs semantically-aware masks through cross-modal alignment
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论弱监督稠密视频描述旨在仅使用描述标注、在无时序边界的情况下训练模型来定位并描述视频中的事件。已有工作引入了基于高斯掩码与互补描述的隐式监督范式。然而，现有方法仅关注生成互不重叠的掩码，而未考虑其与对应事件的语义关系，导致生成的掩码简单且均匀分布，无法捕捉语义上有意义的区域。此外，由于现有数据集固有的稀疏性，仅依赖真实描述会导致性能次优。在本工作中，我们提出 SAIL，通过跨模态对齐构建语义感知掩码。我们的相似度感知训练目标引导掩码强调与对应事件描述高度相似的视频区域。此外，为在稀疏标注设置下引导更准确的掩码生成，我们引入一种基于大语言模型的增强策略，生成合成描述以提供额外的对齐信号。这些合成描述通过掩码间机制被纳入，为精确的时序定位提供辅助引导，同时不损害主目标。在 ActivityNet Captions 与 YouCook2 上的实验表明，该方法在描述与定位指标上均达到最优性能。
英文原文摘要 1416 字符
Weakly-Supervised Dense Video Captioning aims to localize and describe events in videos trained only on caption annotations, without temporal boundaries. Prior work introduced an implicit supervision paradigm based on Gaussian masking and complementary captioning. However, existing method focus merely on generating non-overlapping masks without considering their semantic relationship to corresponding events, resulting in simplistic, uniformly distributed masks that fail to capture semantically meaningful regions. Moreover, relying solely on ground-truth captions leads to sub-optimal performance due to the inherent sparsity of existing datasets. In this work, we propose SAIL, which constructs semantically-aware masks through cross-modal alignment. Our similarity-aware training objective guides masks to emphasize video regions with high similarity to their corresponding event captions. Furthermore, to guide more accurate mask generation under sparse annotation settings, we introduce an LLM-based augmentation strategy that generates synthetic captions to provide additional alignment signals. These synthetic captions are incorporated through an inter-mask mechanism, providing auxiliary guidance for precise temporal localization without degrading the main objective. Experiments on ActivityNet Captions and YouCook2 demonstrate state-of-the-art performance on both captioning and localization metrics.