稀疏性作为钥匙：从潜变量结构中解锁分布外检测的新洞见
Sparsity as a Key: Unlocking New Insights from Latent Structures for Out-of-Distribution Detection
把 Top-k 稀疏自编码器套在 ViT 的 [CLS] 词元上，将稠密特征解耦成结构化潜空间，并用类激活剖面（CAPs）的能量散度做 OOD 打分；做 ViT 特征分析或需要可解释 OOD 判据的工作可直接复用这套「稀疏解耦 + 剖面偏离」流程。
原文依据 we present the first application of SAEs to the ViT [CLS] token for out-of-distribution (OOD) detection
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论稀疏自编码器（SAE）通过将稠密表示分解为稀疏的语义成分，在解释大语言模型（LLM）方面已取得显著成功。然而，其在分析视觉 Transformer（ViT）方面的潜力仍 largely 未被充分探索。在本工作中，我们首次将 SAE 应用于 ViT 的 [CLS] 词元以进行分布外（OOD）检测，解决了现有方法依赖纠缠特征表示的局限。我们提出一种新框架，利用 Top-k SAE 将稠密的 [CLS] 特征解耦为结构化的潜变量空间。通过这一分析，我们揭示出分布内（ID）数据呈现出一致的、类别特定的激活模式，我们将其形式化为类激活剖面（CAP）。我们的研究揭示了一个关键的结构不变量：ID 样本在 CAP 内保持稳定模式，而 OOD 样本则系统性地破坏这一结构。基于这一洞见，我们引入一种基于核心能量剖面散度的打分函数，以量化对理想激活剖面的偏离程度。我们的方法在多个基准上于 FPR95 指标——对安全敏感应用至关重要——取得强劲结果，同时获得有竞争力的 AUROC。总体而言，我们的发现表明，SAE 所揭示的稀疏、解耦特征可作为视觉模型中稳健 OOD 检测的强大且可解释的工具。
英文原文摘要 1453 字符
Sparse Autoencoders (SAEs) have demonstrated significant success in interpreting Large Language Models (LLMs) by decomposing dense representations into sparse, semantic components. However, their potential for analyzing Vision Transformers (ViTs) remains largely under-explored. In this work, we present the first application of SAEs to the ViT [CLS] token for out-of-distribution (OOD) detection, addressing the limitation of existing methods that rely on entangled feature representations. We propose a novel framework utilizing a Top-k SAE to disentangle the dense [CLS] features into a structured latent space. Through this analysis, we reveal that in-distribution (ID) data exhibits consistent, class-specific activation patterns, which we formalize as Class Activation Profiles (CAPs). Our study uncovers a key structural invariant: while ID samples preserve a stable pattern within CAPs, OOD samples systematically disrupt this structure. Leveraging this insight, we introduce a scoring function based on the divergence of core energy profiles to quantify the deviation from ideal activation profiles. Our method achieves strong results on the FPR95 metric--critical for safety-sensitive applications--across multiple benchmarks, while also achieving competitive AUROC. Overall, our findings demonstrate that the sparse, disentangled features revealed by SAEs can serve as a powerful, interpretable tool for robust OOD detection in vision models.