城市场景分割中的开放词表域泛化
Open-Vocabulary Domain Generalization in Urban-Scene Segmentation
把状态空间模型（SSM）用于修正 VLM 在域偏移下的文本-图像相关性，这一思路可迁移到任何依赖 CLIP 类图文对齐的开放词表分割、检测任务中，用于缓解分布变化导致的图文错配。
原文依据 domain shifts often distort text-image correlations in pre-trained VLMs
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论语义分割中的域泛化（DG-SS）旨在使分割模型在未见环境中保持鲁棒性能。然而，传统 DG-SS 方法局限于固定的已知类别集合，限制了其在开放世界场景中的适用性。视觉语言模型（VLM）的最新进展推动了开放词表语义分割（OV-SS），使模型能够识别更广泛的概念。但这些模型对域偏移仍然敏感，在未见环境中部署时难以保持鲁棒性，这一问题在城市驾驶场景中尤为严重。为弥合这一差距，我们提出语义分割中的开放词表域泛化（OVDG-SS），这是一个同时应对未见域与未见类别的新设定。我们构建了自动驾驶领域首个 OVDG-SS 基准，针对此前未被探索的问题，覆盖合成到真实与真实到真实的泛化，涉及多样的未见域与未见类别。在 OVDG-SS 中，我们观察到域偏移常常扭曲预训练 VLM 中的文本-图像相关性，从而阻碍 OV-SS 模型的性能。为应对这一挑战，我们提出 S^2-Corr，一种由状态空间驱动的文本-图像相关性精炼机制，可缓解域引起的扭曲，并在分布变化下产生更一致的文本-图像相关性。在我们构建的基准上的大量实验表明，与现有 OV-SS 方法相比，所提方法取得了更优的跨域性能与效率。
英文原文摘要 1622 字符
Domain Generalization in Semantic Segmentation (DG-SS) aims to enable segmentation models to perform robustly in unseen environments. However, conventional DG-SS methods are restricted to a fixed set of known categories, limiting their applicability in open-world scenarios. Recent progress in Vision-Language Models (VLMs) has advanced Open-Vocabulary Semantic Segmentation (OV-SS) by enabling models to recognize a broader range of concepts. Yet, these models remain sensitive to domain shifts and struggle to maintain robustness when deployed in unseen environments, a challenge that is particularly severe in urban-driving scenarios. To bridge this gap, we introduce Open-Vocabulary Domain Generalization in Semantic Segmentation (OVDG-SS), a new setting that jointly addresses unseen domains and unseen categories. We introduce the first benchmark for OVDG-SS in autonomous driving, addressing a previously unexplored problem and covering both synthetic-to-real and real-to-real generalization across diverse unseen domains and unseen categories. In OVDG-SS, we observe that domain shifts often distort text-image correlations in pre-trained VLMs, which hinders the performance of OV-SS models. To tackle this challenge, we propose S^2-Corr, a state-space-driven text-image correlation refinement mechanism that can mitigate domain-induced distortions and produce a more consistent text-image correlation under distribution changes. Extensive experiments on our constructed benchmark demonstrate that the proposed method achieves superior cross-domain performance and efficiency compared to existing OV-SS approaches.