视觉语言模型的结构化图探测
Structural Graph Probing of Vision-Language Models
把每一层建模为神经元共激活相关图，用图结构定位跨模态的 hub 神经元并做因果扰动验证，为视觉研究者提供一种介于局部归因与完整电路复原之间的中层可解释性工具，可直接用于诊断 VLM 中视觉与语言信息如何逐层融合。
原文依据 representing each layer as a within-layer correlation graph derived from neuron-neuron co-activations
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉语言模型（VLM）已取得强劲的多模态性能，但计算如何在神经元群体层面组织起来仍鲜为人知。在本工作中，我们通过神经拓扑的视角研究 VLM，将每一层表示为由神经元-神经元共激活导出的层内相关图。这一视角使我们得以追问：群体层面的结构是否具有行为意义，它如何随模态与深度变化，以及它能否在干预下识别出具有因果影响力的内部组件。我们表明，相关拓扑携带可恢复的行为信号；此外，跨模态结构随深度逐步围绕一小组反复出现的枢纽神经元而巩固，对这些神经元的定向扰动会实质性地改变模型输出。因此，神经拓扑成为 VLM 可解释性中一个有意义的中间尺度：它比局部归因更丰富，比完整电路复原更易处理，并在经验上与多模态行为相关联。代码已公开于 https://github.com/he-h/vlm-graph-probing。
英文原文摘要 1079 字符
Vision-language models (VLMs) achieve strong multimodal performance, yet how computation is organized across populations of neurons remains poorly understood. In this work, we study VLMs through the lens of neural topology, representing each layer as a within-layer correlation graph derived from neuron-neuron co-activations. This view allows us to ask whether population-level structure is behaviorally meaningful, how it changes across modalities and depth, and whether it identifies causally influential internal components under intervention. We show that correlation topology carries recoverable behavioral signal; moreover, cross-modal structure progressively consolidates with depth around a compact set of recurrent hub neurons, whose targeted perturbation materially alters model output. Neural topology thus emerges as a meaningful intermediate scale for VLM interpretability: richer than local attribution, more tractable than full circuit recovery, and empirically tied to multimodal behavior. Code is publicly available at https://github.com/he-h/vlm-graph-probing.