EagleNet：面向文本-视频检索的能量感知细粒度关系学习网络
EagleNet: Energy-Aware Fine-Grained Relationship Learning Network for Text-Video Retrieval
用文本候选与帧构建文本-帧图来聚合出带帧上下文信息的增强文本嵌入，并用能量匹配建模真实文本-视频对分布，这一「图聚合+能量打分」思路可迁移到其他跨模态检索与图文对齐任务中。
原文依据 constructs a text-frame graph by the generated text candidates and frames, then learns relationships among texts and frames
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论得益于近期大规模视觉-语言预训练模型的发展，文本-视频检索任务取得了显著提升。传统方法主要关注视频表示或跨模态对齐，而近期工作转向丰富文本表达能力，以更好地匹配视频中丰富的语义。然而，这些方法仅利用文本与帧/视频之间的交互，忽略了视频内部各帧之间丰富的交互，因此最终扩展得到的文本无法捕捉帧上下文信息，导致文本与视频之间存在差异。为此，我们提出能量感知细粒度关系学习网络（EagleNet），以生成准确且具有上下文感知能力的增强文本嵌入。具体而言，所提出的细粒度关系学习机制（FRL）首先利用生成的文本候选与帧构建文本-帧图，然后学习文本与帧之间的关系，最终利用这些关系将文本候选聚合为融合帧上下文信息的增强文本嵌入。为进一步改进 FRL 中的细粒度关系学习，我们设计了能量感知匹配（EAM）来建模文本-帧交互的能量，从而准确捕捉真实文本-视频对的分布。此外，为实现更有效的跨模态对齐和稳定训练，我们用 sigmoid 损失替代了传统的基于 softmax 的对比损失。大量实验证明了 EagleNet 在 MSRVTT、DiDeMo、MSVD 和 VATEX 上的优越性。
英文原文摘要 1582 字符
Text-video retrieval tasks have seen significant improvements due to the recent development of large-scale vision-language pre-trained models. Traditional methods primarily focus on video representations or cross-modal alignment, while recent works shift toward enriching text expressiveness to better match the rich semantics in videos. However, these methods use only interactions between text and frames/video, and ignore rich interactions among the internal frames within a video, so the final expanded text cannot capture frame contextual information, leading to disparities between text and video. In response, we introduce Energy-Aware Fine-Grained Relationship Learning Network (EagleNet) to generate accurate and context-aware enriched text embeddings. Specifically, the proposed Fine-Grained Relationship Learning mechanism (FRL) first constructs a text-frame graph by the generated text candidates and frames, then learns relationships among texts and frames, which are finally used to aggregate text candidates into an enriched text embedding that incorporates frame contextual information. To further improve fine-grained relationship learning in FRL, we design Energy-Aware Matching (EAM) to model the energy of text-frame interactions and thus accurately capture the distribution of real text-video pairs. Moreover, for more effective cross-modal alignment and stable training, we replace the conventional softmax-based contrastive loss with the sigmoid loss. Extensive experiments have demonstrated the superiority of EagleNet across MSRVTT, DiDeMo, MSVD, and VATEX.