SAMIX：以语义适配器与参考选择策略强化 SAM2 用于混合监督分割
SAMIX: Reinforcing SAM2 with Semantic Adapter and Reference Selecting Policy for Mix-Supervised Segmentation
把 SAM2 通过轻量语义适配器改造成语义感知伪标签生成器，并用强化学习训练的 SPNet 为每张查询图像检索语义相关且互补的参考图像-掩码对作为密集上下文提示，这一「参考检索 + 可验证奖励」的伪标签提纯思路可直接迁移到任何需要从弱标注或无标注数据中挖掘高质量掩码的半监督/弱监督分割流程。
原文依据 SPNet is trained via reinforcement learning to actively explore reference combinations that maximize pseudo-label quality.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论混合监督图像分割旨在有效利用异构标注。近期基于提示的进展利用 Segment Anything Model（SAM）等基础模型，将弱标签视为空间提示来生成伪掩码。然而，这些方法严重依赖稀疏的空间先验，在模糊区域性能欠佳，并且由于缺乏可提示线索而忽视了无标注数据的潜力。本文提出 SAMIX，一种新颖框架，通过引入轻量语义适配器将 SAM2 改造为语义感知伪标签生成器 SA-SAM2。除受稀疏空间提示引导外，SA-SAM2 还能利用具有共享语义的有价值图像-掩码参考对所提供的密集上下文提示。SAMIX 的另一个核心组件是选择策略网络（SPNet），它自回归地为每张查询图像检索相关且互补的参考样本。与基于规则的选择不同，SPNet 通过强化学习训练，主动探索能使伪标签质量最大化的参考组合。在与掩码质量相关的定制化可验证奖励引导下，选择过程朝向语义信息丰富且多样的上下文。我们在两个通用数据集（PASCAL VOC 2012 和 Cityscapes）以及两个边界模糊的特定数据集（伪装目标检测和图像息肉分割）上开展了大量实验。在多种混合监督设置下，SAMIX 始终取得最优性能。代码见 https://github.com/Huster-Hq/SAMIX。
英文原文摘要 1629 字符
Mix-supervised image segmentation aims to effectively leverage heterogeneous annotations. Recent prompt-based advances utilize foundation models such as Segment Anything Model (SAM) to generate pseudo-masks by treating weak labels as spatial prompts. However, these methods rely heavily on sparse spatial priors, leading to suboptimal performance in ambiguous regions and overlooking the potential of unlabeled data due to the absence of promptable cues. In this paper, we propose SAMIX, a novel framework that adapts SAM2 into a semantic-aware pseudo-label generator SA-SAM2 by incorporating a lightweight semantic adapter. Beyond being guided by sparse spatial prompts, SA-SAM2 facilitates dense contextual prompts provided by valuable image-mask reference pairs with shared semantics. Another core component of SAMIX is the Selecting Policy Network (SPNet), which auto-regressively retrieves relevant and complementary reference samples for each query image. Unlike rule-based selections, SPNet is trained via reinforcement learning to actively explore reference combinations that maximize pseudo-label quality. Guided by customized and verifiable rewards associated with mask quality, the selection is directed toward semantically informative and diverse contexts. We conduct extensive experiments on two general datasets (PASCAL VOC 2012 and Cityscapes) and two specific datasets with ambiguous boundaries (camouflaged object detection and image polyp segmentation). Across diverse mix-supervision settings, SAMIX consistently achieves state-of-the-art performance. Codes are available at https://github.com/Huster-Hq/SAMIX.