MatchMask：面向标签稀缺语义分割的以掩码为中心的生成式数据增强
MatchMask: Mask-Centric Generative Data Augmentation for Label-Scarce Semantic Segmentation
提出梯度探针方法分析扩散模型各层作用，并设计轻量LoRA风格适配器与层自适应交叉注意力融合，实现少样本语义图像合成，可直接用于标签稀缺下的分割数据增强。
原文依据 we first propose a Gradient Probe Method to investigate the role of each layer in the diffusion model.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论当前的语义分割模型非常依赖数据，需要大量昂贵的像素级人工标注。生成式数据增强利用生成模型扩充训练集，提供了一种潜在的补救措施。本文提出MatchMask，一种新颖的以掩码为中心的生成式数据增强方法，专为标签稀缺的语义分割设计。通过利用有限的标注语义掩码，MatchMask生成多样、真实且良好对齐的图像-掩码对，从而提升语义分割模型的性能。具体而言，为了使现有的文本到图像模型适应少样本设置下的语义图像合成，我们首先提出梯度探针方法，研究扩散模型中每一层的作用。在此基础上，为关键层设计了轻量级的LoRA风格适配器以实现高效适配，并结合层自适应交叉注意力融合机制。同时，我们提出了一种鲁棒的相对过滤原则来抑制错误合成的区域。此外，所提方法在半监督设置下扩展为MatchMask++，以利用额外的无标签数据。在PASCAL VOC、COCO和ADE20K上的实验结果表明，MatchMask显著提升了分割模型的性能，在各种基准上超越了先前的数据增强技术，例如在PASCAL VOC上平均交并比从67.5%提升至74.3%。
英文原文摘要 1443 字符
Current semantic segmentation models are very data-hungry and require massive costly pixel-wise human annotations. Generative data augmentation, which scales the train set using generative models, provides a potential remedy. In this paper, we propose MatchMask, a novel mask-centric generative data augmentation approach tailored for label-scarce semantic segmentation. By leveraging a limited set of labeled semantic masks, MatchMask generates diverse, realistic, and well-aligned image-mask pairs, thereby enhancing the performance of semantic segmentation models. Specifically, to adapt existing text-to-image models for semantic image synthesis in the few-shot setting, we first propose a Gradient Probe Method to investigate the role of each layer in the diffusion model. On this basis, a lightweight LoRA-style adapter is designed for critical layers to enable efficient adaptation, coupled with a Layer-adaptive Cross-attention Fusion mechanism. Meanwhile, we present a robust relative filtering principle to suppress incorrectly synthesized regions. Moreover, the proposed approach is extended to MatchMask++ in the semi-supervised setting to take advantage of additional unlabeled data. Experimental results on PASCAL VOC, COCO and ADE20K demonstrate that MatchMask remarkably enhances the performance of segmentation models, surpassing prior data augmentation techniques in various benchmarks, e.g., 67.5%->74.3% mIoU on PASCAL VOC.