SegCompass：利用稀疏自编码器探索可解释对齐以增强推理分割
SegCompass: Exploring Interpretable Alignment with Sparse Autoencoders for Enhanced Reasoning Segmentation
用稀疏自编码器将思维链与视觉 token 映射到共享稀疏概念空间，为推理分割提供可解释、可微的对齐路径，可复用于需要透明推理的视觉-语言任务。
原文依据 we propose SegCompass, an end-to-end model that leverages a Sparse Autoencoder (SAE) to forge an explicit, interpretable, and differentiable alignment pathway.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论尽管大语言模型提供了强大的组合推理能力，现有的推理分割流程未能透明地将这种推理与视觉感知连接起来。当前的方法，如潜变量查询对齐，是端到端的，但却是不透明的“黑箱”。相反，文本定位读出仅仅是可读的，而非真正可解释的，通常充当一个无约束的事后步骤。为了弥合这一可解释性差距，我们提出了 SegCompass，一个端到端模型，利用稀疏自编码器（SAE）构建显式、可解释且可微的对齐路径。给定图像-指令对，SegCompass 首先生成思维链（CoT）轨迹。我们方法的核心是一个 SAE，它将 CoT 和视觉 token 映射到一个共享的高维稀疏概念空间。查询码本从该空间中选择显著概念，然后由槽映射器将这些概念空间定位为多槽热图，以引导最终的掩码解码器。整个模型联合训练，统一了推理路径的强化学习与标准分割监督。这种 SAE 驱动的接口提供了一个“白盒”连接，比潜变量查询显著更可追踪，比文本读出更连贯。在五个具有挑战性的基准上的大量实验表明，SegCompass 达到或超越了最优性能。关键的是，我们的视觉和定量分析显示，学习到的稀疏概念的质量与最终掩码准确性之间存在强相关性，证实了 SegCompass 通过其增强且可检查的对齐实现了优越的结果。代码可在 https://github.com/ZhenyuLU-Heliodore/SegCompass 获取。
英文原文摘要 1738 字符
While large language models provide strong compositional reasoning, existing reasoning segmentation pipelines fail to transparently connect this reasoning to visual perception. Current methods, such as latent query alignment, are end-to-end yet opaque "black boxes". Conversely, textual localization readout is merely readable, not truly interpretable, often functioning as an unconstrained post-hoc step. To bridge this interpretability gap, we propose SegCompass, an end-to-end model that leverages a Sparse Autoencoder (SAE) to forge an explicit, interpretable, and differentiable alignment pathway. Given an image-instruction pair, SegCompass first generates a chain-of-thought (CoT) trace. The core of our method is an SAE that maps both the CoT and visual tokens into a shared, high-dimensional sparse concept space. A query codebook selects salient concepts from this space, which are then spatially grounded by a slot mapper into a multi-slot heatmap that guides the final mask decoder. The entire model is trained jointly, unifying reinforcement learning for the reasoning path with standard segmentation supervision. This SAE-driven interface provides a "white-box" connection that is significantly more traceable than latent queries and more coherent than textual readouts. Extensive experiments on five challenging benchmarks demonstrate that SegCompass matches or surpasses state-of-the-art performance. Crucially, our visual and quantitative analyses show a strong correlation between the quality of the learned sparse concepts and final mask accuracy, confirming that SegCompass achieves superior results through its enhanced and inspectable alignment. Code is available at https://github.com/ZhenyuLU-Heliodore/SegCompass.