通过 BrainSSD 解码三维感知：静态与动态视觉流脑电表征的协同融合
Decoding 3D Perception via BrainSSD: Synergistic Fusion of EEG Representations from Static and Dynamic Visual Streams
对做视觉表征学习的人，这篇提供了用 EEG 信号作为监督/对齐目标来反推 3D 物体表征的思路：静态视图与旋转序列两路信号经 PAC 引导融合后，再用多级对齐损失映射回视觉域，可作为研究「视觉模型表征是否与大脑 3D 感知一致」的探针与对齐范式。
原文依据 The resulting fused representations are subsequently mapped to the visual domain via a multi-level alignment loss.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论理解大脑如何从多方面的经验中构建连贯的三维视觉感知，仍是一个关键但尚未被充分探索的挑战。为研究这一问题，我们提出 BrainSSD，一个从脑电（EEG）信号中解码三维表征的新框架。BrainSSD 的核心是一种受神经科学启发的融合架构——分层相位-幅值耦合引导融合（HPACF），它协同整合来自两种不同观看范式的脑电信号：静态三维物体视图的短暂呈现，以及对物体进行完整旋转的持续观察。HPACF 体现了神经计算的两条关键原理，即通过多级交叉注意力实现的分层处理，以及通过使用相位-幅值耦合（PAC）的可微估计器来动态引导整合而实现的神经同步。所得的融合表征随后通过多级对齐损失被映射到视觉域。我们的框架在一系列脑电解码任务上建立了新的最优性能，实现了更强的判别能力与出色的生成保真度。此外，我们的静态-动态优势分析首次为大脑三维感知中的功能专门化提供了直接的视觉证据，揭示出对静态物体视图的神经响应主要支撑物体的整体结构与形态，而对旋转观察的响应对于解析其细粒度几何细节不可或缺。我们的工作为探究基于脑电的视觉解码提供了一个先进框架，并为大脑实现三维感知的协同策略提供了计算层面的洞见。
英文原文摘要 1696 字符
Understanding how the brain constructs coherent 3D visual percepts from multifaceted experiences remains a pivotal yet underexplored challenge. To investigate this, we introduce BrainSSD, a novel framework for decoding 3D representations from electroencephalography (EEG) signals. The core of BrainSSD is a neuro-inspired fusion architecture, Hierarchical Phase-Amplitude Coupling guided Fusion (HPACF), which synergistically integrates EEG from two distinct viewing paradigms: brief presentations of a static 3D object view, and sustained observation of the object undergoing full rotation. HPACF embodies two key principles of neural computation, namely hierarchical processing realized through multi-level cross-attention, and neural synchrony actualized by using a differentiable estimator of Phase-Amplitude Coupling (PAC) to dynamically guide the integration. The resulting fused representations are subsequently mapped to the visual domain via a multi-level alignment loss. Our framework establishes a new state-of-the-art across a range of EEG decoding tasks, achieving superior discriminative power and exceptional generative fidelity. Furthermore, our static-dynamic dominance analysis provides the first direct visual evidence for a functional specialization in the brain's 3D perception, revealing that neural responses to static object views primarily underpin the object's holistic structure and form, while responses to rotational observation are indispensable for resolving its fine-grained geometric details. Our work presents an advanced framework for probing EEG-based visual decoding and offers computational insights into the brain's synergistic strategies for 3D perception.