OpenDPR：基于视觉中心扩散引导原型检索的遥感影像开放词表变化检测
OpenDPR: Open-Vocabulary Change Detection via Vision-Centric Diffusion-Guided Prototype Retrieval for Remote Sensing Imagery
将扩散模型作为离线原型生成器，在视觉空间进行开放词表检索，为视觉基础模型（SAM、DINOv2）与视觉语言模型（CLIP）的类别识别瓶颈提供免训练解决方案；S2C 弱监督模块可复用为适配 VFM 空间先验的通用变化定位方法。
原文依据 We reformulate OVCD as a two-stage pipeline: first generate class-agnostic change proposals using visual foundation models (VFMs) such as SAM and DINOv2, and then perform category identification with vision-language models (VLMs) such as CLIP.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论开放词表变化检测（OVCD）旨在通过超越固定预定义类别的泛化能力，识别任意感兴趣的变化。我们将 OVCD 重新表述为两阶段流程：首先使用视觉基础模型（VFM，如 SAM 和 DINOv2）生成类别无关的变化候选区域，然后使用视觉语言模型（VLM，如 CLIP）进行类别识别。我们揭示类别识别错误是 OVCD 的主要瓶颈，主要归因于基于图文匹配的 VLM 对细粒度土地覆盖类别的表征能力有限。为解决此问题，我们提出 OpenDPR，一个免训练的视觉中心扩散引导原型检索框架。OpenDPR 利用扩散模型离线构建目标类别的多样化原型，并在推理时在视觉空间中与变化候选区域进行相似度检索。次要瓶颈在于变化定位，这是由于 VFM 本身缺乏变化先验。为弥补这一差距，我们设计了一个空间到变化的弱监督变化检测模块 S2C，以将其强大的空间建模能力适配于变化定位。将预训练的 S2C 集成到 OpenDPR 中，得到一个可选的弱监督变体 OpenDPR-W，它以极少的监督进一步改进 OVCD。在四个基准数据集上的实验结果表明，所提方法在两种监督模式下均达到最优性能。代码见 https://github.com/guoqi2002/OpenDPR。
英文原文摘要 1571 字符
Open-vocabulary change detection (OVCD) seeks to recognize arbitrary changes of interest by enabling generalization beyond a fixed set of predefined classes. We reformulate OVCD as a two-stage pipeline: first generate class-agnostic change proposals using visual foundation models (VFMs) such as SAM and DINOv2, and then perform category identification with vision-language models (VLMs) such as CLIP. We reveal that category identification errors are the primary bottleneck of OVCD, mainly due to the limited ability of VLMs based on image-text matching to represent fine-grained land-cover categories. To address this, we propose OpenDPR, a training-free vision-centric diffusion-guided prototype retrieval framework. OpenDPR leverages diffusion models to construct diverse prototypes for target categories offline, and to perform similarity retrieval with change proposals in the visual space during inference. The secondary bottleneck lies in change localization, due to the inherent lack of change priors in VFMs. To bridge this gap, we design a spatial-to-change weakly supervised change detection module named S2C to adapt their strong spatial modeling capabilities for change localization. Integrating the pretrained S2C into OpenDPR leads to an optional weakly supervised variant named OpenDPR-W, which further improves OVCD with minimal supervision. Experimental results on four benchmark datasets demonstrate that the proposed methods achieve state-of-the-art performance under both supervision modes. Code is available at https://github.com/guoqi2002/OpenDPR.