GenMask：通过直接掩码生成适配 DiT 的分割方法
GenMask: Adapting DiT for Segmentation via Direct Mask Generation
将分割视为生成任务，直接用 DiT 在 RGB 空间生成二值掩码，省去特征提取流程；其针对二值掩码的极端噪声时间步采样策略可复用于其他生成式密集预测任务。
原文依据 we introduce timesteps sampling strategy for binary masks that emphasizes extreme noise levels for segmentation
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近期分割方法利用预训练生成模型作为特征提取器，将分割视为通过间接特征检索的下游适配任务。这种隐式使用存在表征上的根本性错位，并且严重依赖间接特征提取流程，使工作流复杂化并限制适配。本文主张，分割任务不应采用间接适配，而应以生成方式直接训练。我们识别出这一统一表述的关键障碍：二值掩码的变分自编码器潜变量呈锐利分布、噪声鲁棒且线性可分，与自然图像潜变量不同。为弥合这一差距，我们引入针对二值掩码的时间步采样策略，对分割强调极端噪声水平，对图像生成采用适度噪声，从而实现和谐的联合训练。我们提出 GenMask，一种在原始生成目标下训练以在 RGB 空间生成黑白分割掩码及彩色图像的 DiT。GenMask 保留原始 DiT 架构，同时无需为分割任务定制的特征提取流程。实验上，GenMask 在指代和推理分割基准上达到最优性能，消融实验量化了各组件的贡献。
英文原文摘要 1367 字符
Recent approaches for segmentation have leveraged pretrained generative models as feature extractors, treating segmentation as a downstream adaptation task via indirect feature retrieval. This implicit use suffers from a fundamental misalignment in representation.It also depends heavily on indirect feature extraction pipelines, which complicate the workflow and limit adaptation.In this paper, we argue that instead of indirect adaptation, segmentation tasks should be trained directly in a generative manner.We identify a key obstacle to this unified formulation: VAE latents of binary masks are sharply distributed, noise robust, and linearly separable, distinct from natural image latents.To bridge this gap, we introduce timesteps sampling strategy for binary masks that emphasizes extreme noise levels for segmentation and moderate noise for image generation, enabling harmonious joint training.We present GenMask, a DiT trains to generate black-and-white segmentation masks as well as colorful images in RGB space under the original generative objective. GenMask preserves the original DiT architecture while removing the need of feature extraction pipelines tailored for segmentation tasks. Empirically, GenMask attains state-of-the-art performance on referring and reasoning segmentation benchmarks and ablations quantify the contribution of each component.