GeCo:面向域泛化语义分割的几何一致性正则化
GeCo: Geometry-Consistent Regularization for Domain Generalized Semantic Segmentation
针对在 DGSS 中用 PEFT 微调 VFM 时适配器过拟合源域、破坏预训练几何结构的问题,GeCo 用曲率引导扰动和测地线正则化在保持流形结构的前提下扩展表征,可直接迁移到其他基于 VFM 微调的密集预测任务(如深度估计、开放词表分割)的正则化设计。
原文依据 GeCo introduces curvature-guided perturbation to modulate feature variation according to local manifold complexity of the pretrained embedding space
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉基础模型(VFM)通过大规模预训练提供了丰富且可迁移的表征,然而在适配下游任务时,其高容量表征并未得到充分利用。在域泛化语义分割(DGSS)中,参数高效微调(PEFT)往往使适配器过拟合于源域统计特性和已见类别边界,导致表征退化,表现为域偏差与语义刚性。现有正则化策略通过随机扰动来缓解这一问题,但此类操作会破坏预训练的几何结构,造成语义漂移与不稳定的泛化性能。我们提出几何一致性正则化(GeCo),它在尊重结构的约束下将预训练表征空间向目标任务外推,从而在增强任务特定适配的同时保留 VFM 固有的泛化能力。GeCo 引入曲率引导扰动,依据预训练嵌入空间的局部流形复杂度来调节特征变化,实现结构对齐的表征扩展。作为补充,基于测地线的正则化约束预测沿平滑、与流形对齐的轨迹移动,确保语义连续性与稳定的决策行为。大量实验表明,GeCo 在闭集与开放集 DGSS 基准上均取得更优的泛化性能。代码已发布于 https://github.com/DZhaoXd/GeCo。
英文原文摘要 1508 字符
Vision Foundation Models (VFMs) provide rich and transferable representations through large-scale pretraining, yet their high-capacity representations remain underutilized when adapted to downstream tasks. In Domain Generalization Semantic Segmentation (DGSS), parameter-efficient fine-tuning (PEFT) often overfits adapters to source domain statistics and seen class boundaries, leading to representation degradation manifested as domain bias and semantic rigidity. Existing regularization strategies alleviate this through random perturbations, but such operations disrupt the pretrained geometric structure, causing semantic drift and unstable generalization. We propose Geometry-Consistent Regularization (GeCo), which extrapolates the pretrained representation space toward the target task under structure-respected constraints, thereby preserving the inherent generalization of VFMs while enhancing their task-specific adaptation. GeCo introduces curvature-guided perturbation to modulate feature variation according to local manifold complexity of the pretrained embedding space, enabling structure-aligned representation expansion. Complementarily, a geodesic-based regularization constrains prediction shifts along smooth, manifold-aligned trajectories, ensuring semantic continuity and stable decision behavior. Extensive experiments demonstrate that GeCo achieves superior generalization across both closed-set and open-set DGSS benchmarks. The code is available at https://github.com/DZhaoXd/GeCo.