超越视野:外推式域自适应全景分割
Seeing Beyond: Extrapolative Domain Adaptive Panoramic Segmentation
针对透视训练、全景测试的跨视角几何偏移,提出带角度间隔的注意力调制与高阶图匹配适配器,可复用于任何需要视角不变语义表征的跨域分割或开放集识别任务。
原文依据 trains on local perspective views and tests on full 360^ \circ panoramic images
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论跨域全景语义分割因能为真实世界应用提供全面的 360° 场景理解而日益受到关注。然而,由于严重的几何视场(FoV)畸变以及跨域开放集语义的不一致,该任务仍极具挑战性。本文构建了一种开放集域自适应设定,并提出外推式域自适应全景分割(EDA-PSeg)框架,在局部透视视图上训练、在完整 360° 全景图像上测试,显式应对跨域的几何视场偏移以及由先前未见类别引发的语义不确定性。为此,我们提出欧拉间隔注意力(EMA),引入角度间隔以增强视角不变的语义表征,同时进行幅度与相位调制以提升对未见类别的泛化能力。此外,我们设计了图匹配适配器(GMA),构建高阶图关系以对齐跨视场偏移的共享语义,并通过结构自适应有效分离新类别。在相机偏移、天气条件与开放集场景下的四个基准数据集上的大量实验表明,EDA-PSeg 取得了最优性能,对多样视角几何具有稳健的泛化能力,并在多变环境条件下保持韧性。代码见 https://github.com/zyfone/EDA-PSeg。
英文原文摘要 1490 字符
Cross-domain panoramic semantic segmentation has attracted growing interest as it enables comprehensive 360^ \circ scene understanding for real-world applications. However, it remains particularly challenging due to severe geometric Field of View (FoV) distortions and inconsistent open-set semantics across domains. In this work, we formulate an open-set domain adaptation setting, and propose Extrapolative Domain Adaptive Panoramic Segmentation (EDA-PSeg) framework that trains on local perspective views and tests on full 360^ \circ panoramic images, explicitly tackling both geometric FoV shifts across domains and semantic uncertainty arising from previously unseen classes. To this end, we propose the Euler-Margin Attention (EMA), which introduces an angular margin to enhance viewpoint-invariant semantic representation, while performing amplitude and phase modulation to improve generalization toward unseen classes. Additionally, we design the Graph Matching Adapter (GMA), which builds high-order graph relations to align shared semantics across FoV shifts while effectively separating novel categories through structural adaptation. Extensive experiments on four benchmark datasets under camera-shift, weather-condition, and open-set scenarios demonstrate that EDA-PSeg achieves state-of-the-art performance, robust generalization to diverse viewing geometries, and resilience under varying environmental conditions. The code is available at https://github.com/zyfone/EDA-PSeg.