图像分割 · CVPR2026 ·2026.06.03

PEARL:几何对齐语义的免训练开放词表语义分割

PEARL: Geometry Aligns Semantics for Training-Free Open-Vocabulary Semantic Segmentation

Gensheng Pei、Xiruo Jiang、Xinhao Cai、Tao Chen、Yazhou Yao、Byeungwoo Jeon

与你有关的部分

提出一种免训练、即插即用的两阶段推理框架:在自注意力块内用正交Procrustes对齐旋转key到query子空间,再用文本感知的拉普拉斯图求解细化像素logits,可复用于其他需要跨模态对齐的密集预测任务。

原文依据 The Procrustes alignment step performs an orthogonal projection inside the last self-attention block, rotating keys toward the query subspace via a stable polar iteration.
可迁移到 开放词表语义分割语义分割跨模态对齐

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

免训练开放词表语义分割(OVSS)有望在不重新训练的情况下快速适应新的标签集。然而,许多方法依赖繁重的后处理或孤立地处理文本和视觉,导致跨模态几何未被充分利用。其他方法引入辅助视觉骨干或多模型流水线,增加了复杂性和延迟,同时损害了设计简洁性。我们提出PEARL(Procrustes对齐与文本感知拉普拉斯传播),一种遵循“先对齐后传播”原则的紧凑两步推理方法。Procrustes对齐步骤在最后一个自注意力块内执行正交投影,通过稳定的极迭代将键旋转到查询子空间。文本感知的拉普拉斯传播随后通过置信度加权、文本引导的图求解在小网格上细化逐像素logits:文本提供数据信任信号和邻居门控,而图像梯度保留边界。在本工作中,我们的方法完全免训练、即插即用,仅使用固定常数,通过小型每头投影和少量共轭梯度步骤增加最小延迟。我们的方法PEARL在标准基准上,无需额外数据或辅助骨干,在带背景和不带背景两种协议下均取得优越性能,树立了免训练OVSS的新最优。

英文原文摘要 1448 字符

Training-free open-vocabulary semantic segmentation (OVSS) promises rapid adaptation to new label sets without retraining. Yet, many methods rely on heavy post-processing or handle text and vision in isolation, leaving cross-modal geometry underutilized. Others introduce auxiliary vision backbones or multi-model pipelines, which increase complexity and latency while compromising design simplicity.We present PEARL, \underline P rocrust\underline e s \underline a lignment with text-awa\underline r e \underline L aplacian propagation, a compact two-step inference that follows an align-then-propagate principle. The Procrustes alignment step performs an orthogonal projection inside the last self-attention block, rotating keys toward the query subspace via a stable polar iteration. The text-aware Laplacian propagation then refines per-pixel logits on a small grid through a confidence-weighted, text-guided graph solve: text provides both a data-trust signal and neighbor gating, while image gradients preserve boundaries. In this work, our method is fully training-free, plug-and-play, and uses only fixed constants, adding minimal latency with a small per-head projection and a few conjugate-gradient steps. Our approach, PEARL, sets a new state-of-the-art in training-free OVSS without extra data or auxiliary backbones across standard benchmarks, achieving superior performance under both with-background and without-background protocols.

标签

会议 CVPR2026
页码 17927-17937
Bibkey
来源 CVF Open Access