视觉语言模型 · CVPR2026 ·2026.06.03

先想象后聚焦:扩散引导的寄存器增强部分相关视频检索

Imagine Before Concentration: Diffusion-Guided Registers Enhance Partially Relevant Video Retrieval

Jun Li、Xuhang Lou、Jinpeng Wang、Yuting Wang、Yaowei Wang、Shu-Tao Xia 等 7 人

与你有关的部分

用文本监督的截断扩散模型从视频分布中生成全局语义寄存器,再经寄存器增强的高斯注意力与视频 token 融合,可作为跨模态检索中缓解局部噪声与查询歧义的通用特征增强模块。

原文依据 iteratively refined via a text-supervised truncated diffusion model
可迁移到 partially relevant video retrievaltext-video retrievalcross-modal matching

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

部分相关视频检索(PRVR)旨在根据仅描述部分事件的文本查询检索未裁剪视频。现有方法存在全局上下文感知不完整的问题,难以应对查询歧义以及虚假响应带来的局部噪声。为解决这些问题,我们提出 DreamPRVR,采用由粗到细的表征学习范式。该模型首先生成贯穿整个视频的粗粒度亮点作为全局上下文语义寄存器,然后专注于细粒度相似度优化以实现精确的跨模态匹配。具体而言,这些寄存器由概率变分采样器产生的以视频为中心的分布初始化,再通过文本监督的截断扩散模型迭代精炼。在此过程中,文本语义结构学习构建了结构良好的文本潜变量空间,增强了全局感知的可靠性。随后,寄存器通过寄存器增强的高斯注意力块与视频 token 自适应融合,实现上下文感知的特征学习。大量实验表明,DreamPRVR 优于最优方法。代码发布于 https://github.com/lijun2005/CVPR26-DreamPRVR。

英文原文摘要 1278 字符

Partially Relevant Video Retrieval (PRVR) aims to retrieve untrimmed videos based on text queries that describe only partial events. Existing methods suffer from incomplete global contextual perception, struggling with query ambiguity and local noise induced by spurious responses. To address these issues, we propose DreamPRVR, which adopts a coarse-to-fine representation learning paradigm. The model first generates global contextual semantic registers as coarse-grained highlights spanning the entire video and then concentrates on fine-grained similarity optimization for precise cross-modal matching. Concretely, these registers are generated by initializing from the video-centric distribution produced by a probabilistic variational sampler and then iteratively refined via a text-supervised truncated diffusion model. During this process, textual semantic structure learning constructs a well-formed textual latent space, enhancing the reliability of global perception. The registers are then adaptively fused with video tokens through register-augmented Gaussian attention blocks, enabling context-aware feature learning. Extensive experiments show that DreamPRVR outperforms state-of-the-art methods. Code is released at https://github.com/lijun2005/CVPR26-DreamPRVR.

标签

会议 CVPR2026
页码 9710-9721
Bibkey
来源 CVF Open Access