MM-OVSeg：面向遥感开放词表分割的多模态光学-SAR 融合
MM-OVSeg: Multimodal Optical-SAR Fusion for Open-Vocabulary Segmentation in Remote Sensing
对做开放词表分割的人，这篇给出了把 SAR 结构先验注入 VLM 文本对齐分割头的双编码器融合方案，可作为恶劣天气/光学缺失场景下扩展 OV 分割输入模态的参考；其跨模态表征统一流程也可迁移到其他传感器组合。
原文依据 a dual-encoder fusion module that integrates hierarchical features from multiple vision foundation models for text-aligned multimodal segmentation
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论开放词表分割能够依据开放集合的文本类别进行像素级识别，从而泛化到固定类别之外。尽管在遥感领域潜力巨大，该方向的进展仍主要局限于晴空光学数据，在云层或雾霾污染条件下表现不佳。本文提出 MM-OVSeg，一个面向恶劣天气条件下鲁棒开放词表分割的多模态光学-SAR 融合框架。MM-OVSeg 利用两种模态的互补优势——光学影像提供丰富的光谱语义，而合成孔径雷达（SAR）提供可穿透云层的结构线索。为应对跨模态域间差异以及当前视觉语言模型密集预测能力有限的问题，我们提出两项关键设计：用于多传感器表征对齐的跨模态统一过程，以及一个双编码器融合模块，该模块整合来自多个视觉基础模型的分层特征，以实现文本对齐的多模态分割。大量实验表明，MM-OVSeg 在多种云层条件下均取得更优的鲁棒性与泛化能力。源数据集与代码见 https://github.com/Jimmyxichen/MM-OVSeg。
英文原文摘要 1249 字符
Open-vocabulary segmentation enables pixel-level recognition from an open set of textual categories, allowing generalization beyond fixed classes. Despite great potential in remote sensing, progress in this area remains largely limited to clear-sky optical data and struggles under cloudy or haze-contaminated conditions. We present MM-OVSeg, a multimodal Optical-SAR fusion framework for resilient open-vocabulary segmentation under adverse weather conditions. MM-OVSeg leverages the complementary strengths of the two modalities--optical imagery provides rich spectral semantics, while synthetic aperture radar (SAR) offers cloud-penetrating structural cues. To address the cross-modal domain gap and the limited dense prediction capability of current vision-language models, we propose two key designs: a cross-modal unification process for multi-sensor representation alignment, and a dual-encoder fusion module that integrates hierarchical features from multiple vision foundation models for text-aligned multimodal segmentation. Extensive experiments demonstrate that MM-OVSeg achieves superior robustness and generalization across diverse cloud conditions. The source dataset and code are available at https://github.com/Jimmyxichen/MM-OVSeg.