自监督与表征学习 · CVPR2026 ·2026.06.03

MARCO:探索语义对应的未知空间

MARCO: Navigating the Unseen Space of Semantic Correspondence

Claudia Cuttano、Gabriele Trivigno、Carlo Masone、Stefan Roth

与你有关的部分

用自蒸馏把稀疏关键点监督扩展成密集语义一致的对应,配合粗到细目标提升细粒度定位,为做稠密匹配/关键点迁移的视觉工作提供了一个可替换扩散骨干的轻量 DINOv2 训练框架。

原文依据 a self-distillation framework, which expands sparse supervision beyond annotated regions
可迁移到 semantic correspondencekeypoint transferdense matching

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

近期语义对应领域的进展依赖于双编码器架构,将 DINOv2 与扩散骨干网络相结合。这类方法虽然准确,但这些十亿参数级模型在训练关键点之外的泛化能力较差,暴露出基准性能与真实可用性之间的差距——在真实场景中,查询点很少与训练时见过的点相同。我们在 DINOv2 的基础上提出 MARCO,一个由新颖训练框架驱动的统一通用对应模型,同时增强细粒度定位与语义泛化能力。通过将提升空间精度的由粗到细目标与自蒸馏框架相结合——后者将稀疏监督扩展到标注区域之外——我们的方法把少量关键点转化为稠密且语义一致的对应。MARCO 在 SPair-71k、AP-10K 和 PF-PASCAL 上取得新的最优结果,且增益在细粒度定位阈值上进一步放大(+8.9 PCK@0.01),对未见关键点(+5.1,SPair-U)与未见类别(+4.7,MP-100)的泛化最强,同时比基于扩散的方法小 3 倍、快 10 倍。

英文原文摘要 1131 字符

Recent advances in semantic correspondence rely on dual-encoder architectures, combining DINOv2 with diffusion backbones. While accurate, these billion-parameter models generalize poorly beyond training keypoints, revealing a gap between benchmark performance and real-world usability, where queried points rarely match those seen during training. Building upon DINOv2, we introduce MARCO, a unified model for generalizable correspondence driven by a novel training framework that enhances both fine-grained localization and semantic generalization. By coupling a coarse-to-fine objective that refines spatial precision with a self-distillation framework, which expands sparse supervision beyond annotated regions, our approach transforms a handful of keypoints into dense, semantically coherent correspondences. MARCO sets a new state of the art on SPair-71k, AP-10K, and PF-PASCAL, with gains that amplify at fine-grained localization thresholds (+8.9 PCK@0.01), strongest generalization to unseen keypoints (+5.1, SPair-U) and categories (+4.7, MP-100), while remaining 3x smaller and 10x faster than diffusion-based approaches.

标签

会议 CVPR2026
页码 21649-21658
Bibkey
来源 CVF Open Access