图像分割 · CVPR2026 ·2026.06.03

选择性、正则化与校准:利用视觉基础模型进行跨域少样本语义分割

Selective, Regularized, and Calibrated: Harnessing Vision Foundation Models for Cross-Domain Few-Shot Semantic Segmentation

Junyuan Ma、Xunzhi Xiang、Wenbin Li、Qi Fan、Yang Gao

与你有关的部分

对做分割的人,这篇给出了一个在冻结 VFM 特征上做测试时适配的完整范式:用数据相关的 Exemplar Transfer Risk 自动挑层、再用先验引导正则化与像素级校准修正交互图,且只微调不到 2.7% 参数,可直接迁移到其他少样本/跨域密集预测任务。

原文依据 fine-tuning less than 2.7% of parameters at test time
可迁移到 跨域少样本语义分割少样本分割语义分割

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

视觉基础模型(VFM)已在各类视觉任务上取得强劲性能。然而,将 VFM 用于跨域少样本分割(CD-FSS)仍然具有挑战性:该任务在域偏移下仅用少量带标注的示例来分割新类别的物体。这一挑战主要由两个因素驱动:(1)相对于 VFM 预训练的规模而言,每个新类别的带标注示例十分有限,使模型在重训练过程中容易过拟合;(2)目标域偏移在预训练中未被充分表征,导致跨域不一致性与逐层敏感性。为解决这些问题,我们提出分层示例表征适配(HERA),这是一个基于 VFM 的三阶段“选择—正则化—校准”分割框架,能够从有限标签中有效学习,并在无需源数据重训练的情况下适配新域。我们首先设计分层选择(HLS),利用为每个候选层计算的数据相关的示例迁移风险(ETR)来自适应地识别信息量最大的 VFM 层。随后,先验引导正则化(PGR)对所选表征上的交互进行正则化,为后续阶段产生结构良好的局部信号。此外,像素级自适应校准(PAC)将所选表征与精化后的交互图相结合,以校准逐像素预测,产生一致的掩码。这些阶段共同构成分层“选择—正则化—校准”流程,在新域中引导冻结的 VFM 特征,同时在测试时仅微调不到 2.7% 的参数。大量实验表明,HERA 在多个 CD-FSS 基准上以超过 4.1 平均交并比的优势超越最优方法。

英文原文摘要 1736 字符

Vision foundation models (VFMs) have achieved strong performance across various vision tasks. However, it still remains challenging to apply VFMs for cross-domain few-shot segmentation (CD-FSS), which segments objects of novel classes under domain shifts using only a few labeled exemplars. The challenge is mainly driven by two factors: (1) limited labeled exemplars per novel class relative to the scale of VFM pre-training, making the model prone to overfitting during retraining, and (2) target-domain shifts underrepresented during pre-training, inducing cross-domain inconsistency and layerwise sensitivity. To address these issues, we propose Hierarchical Exemplar Representation Adaptation (HERA), a three-stage select-regularize-calibrate VFM-based segmentation framework that learns effectively from limited labels and adapts to novel domains without source-data retraining. We first design Hierarchical Layer Selection (HLS) to adaptively identify the most informative VFM layer using a data-dependent Exemplar Transfer Risk (ETR) computed for each candidate layer. Then, Prior-Guided Regularization (PGR) regularizes interactions on the selected representation, yielding well-structured local signals for the subsequent stage. Furthermore, Pixelwise Adaptive Calibration (PAC) combines the selected representation with the refined interaction maps to calibrate pixelwise predictions, producing consistent masks. Together, these stages form a hierarchical select-regularize-calibrate pipeline that guides frozen VFM features in new domains while fine-tuning less than 2.7% of parameters at test time. Extensive experiments show that HERA surpasses the state-of-the-art by more than 4.1 mIoU across multiple CD-FSS benchmarks.

标签

会议 CVPR2026
页码 12385-12395
Bibkey
来源 CVF Open Access