面向任务的数据合成与控制修正采样用于遥感语义分割
Task-Oriented Data Synthesis and Control-Rectify Sampling for Remote Sensing Semantic Segmentation
提出基于 MM-DiT 的文本-图像-掩码联合注意力控制方案与全微调策略，并设计控制修正流匹配（CRFM）在采样早期用语义损失动态调整方向，可直接用于提升少样本遥感语义分割的合成数据质量。
原文依据 we propose a control-rectify flow matching (CRFM) method, which dynamically adjusts sampling directions guided by semantic loss during the early high-plasticity stage
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论随着可控生成的快速发展，训练数据合成已成为扩充标注数据集、缓解遥感（RS）中人工标注负担的一种有前景的途径。然而，语义掩码控制的复杂性以及采样质量的不确定性，往往限制合成数据在下游语义分割任务中的效用。为应对这些挑战，我们提出了一个面向任务的数据合成框架（TODSynth），包含具有统一三重注意力的多模态扩散 Transformer（MM-DiT）以及一种由任务反馈引导的即插即用采样策略。基于强大的 DiT 生成基础模型，我们系统评估了不同的控制方案，表明文本-图像-掩码联合注意力方案结合图像与掩码分支的全微调，能显著提升遥感语义分割数据合成的有效性，尤其是在少样本和复杂场景下。此外，我们提出了控制修正流匹配（CRFM）方法，在早期高塑性阶段由语义损失引导动态调整采样方向，缓解生成图像的不稳定性，并弥合合成数据与下游分割任务之间的差距。大量实验表明，我们的方法持续优于最优的可控生成方法，为遥感语义分割生成更稳定、更面向任务的合成数据。
英文原文摘要 1490 字符
With the rapid progress of controllable generation, training data synthesis has become a promising way to expand labeled datasets and alleviate manual annotation in remote sensing (RS). However, the complexity of semantic mask control and the uncertainty of sampling quality often limit the utility of synthetic data in downstream semantic segmentation tasks. To address these challenges, we propose a task-oriented data synthesis framework (TODSynth), including a Multimodal Diffusion Transformer (MM-DiT) with unified triple attention and a plug-and-play sampling strategy guided by task feedback. Built upon the powerful DiT-based generative foundation model, we systematically evaluate different control schemes, showing that a text-image-mask joint attention scheme combined with full fine-tuning of the image and mask branches significantly enhances the effectiveness of RS semantic segmentation data synthesis, particularly in few-shot and complex-scene scenarios. Furthermore, we propose a control-rectify flow matching (CRFM) method, which dynamically adjusts sampling directions guided by semantic loss during the early high-plasticity stage, mitigating the instability of generated images and bridging the gap between synthetic data and downstream segmentation tasks. Extensive experiments demonstrate that our approach consistently outperforms state-of-the-art controllable generation methods, producing more stable and task-oriented synthetic data for RS semantic segmentation.