NeuroFlow：迈向神经活动驱动的统一视觉编码与解码
NeuroFlow: Toward Unified Visual Encoding and Decoding from Neural Activity
将视觉编码与解码统一为共享潜空间中的可逆流匹配过程，为视觉-神经跨模态生成提供新范式；其跨模态流匹配（XFM）可复用于图像与脑信号间的双向转换，NeuroVAE 的结构化潜空间设计对视觉表征学习有借鉴意义。
原文依据 Cross-modal Flow Matching (XFM) bypasses the typical paradigm of noise-to-data diffusion guided by a specific modality condition, instead learning a reversibly consistent flow model between visual and neural latent distributions.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉编码与解码模型是理解人类视觉感知神经机制的门户。通常，从刺激预测大脑活动的视觉编码模型与从大脑活动重建刺激的解码模型被视为不同任务，需要分别建模和训练。这种分离效率低下，且无法建模编码与解码过程之间的一致性。为解决这一局限，我们提出 NeuroFlow，这是首个在单一流模型中联合建模神经活动视觉编码与解码的统一框架。NeuroFlow 引入两个关键组件：(i) NeuroVAE 被设计为变分骨干网络，用于建模神经变异性，并建立紧凑、语义结构化的潜空间，以支持视觉与神经模态间的双向建模。(ii) 跨模态流匹配（XFM）绕过了由特定模态条件引导的典型噪声到数据扩散范式，转而学习视觉与神经潜变量分布之间可逆一致的流模型。视觉编码与解码首次被重新表述为共享潜空间内时间依赖的可逆过程，实现统一建模。实证结果表明，与任何孤立方法相比，NeuroFlow 在视觉编码与解码任务中取得更优的整体性能，且计算效率更高。我们进一步分析了引导模型实现编码-解码一致性的主要因素，并通过脑功能分析证明 NeuroFlow 捕捉到神经变异性背后一致的激活模式。NeuroFlow 标志着向神经活动驱动的统一视觉编码与解码迈出重要一步，为未来双向视觉脑机接口提供机制性见解。代码与项目页面已公开于 https://michaelmaiii.github.io/NeuroFlow-S。
英文原文摘要 1977 字符
Visual encoding and decoding models act as gateways to understanding the neural mechanisms underlying human visual perception. Typically, visual encoding models that predict brain activity from stimuli and decoding models that reproduce stimuli from brain activity are treated as distinct tasks, requiring separate models and training procedures. This separation is inefficient and fails to model the consistency between encoding and decoding processes. To address this limitation, we propose NeuroFlow, the first unified framework that jointly models visual encoding and decoding from neural activity within a single flow model. NeuroFlow introduces two key components: (i) NeuroVAE is designed as a variational backbone to model neural variability and establish a compact, semantically structured latent space for bidirectional modeling across visual and neural modalities. (ii) Cross-modal Flow Matching (XFM) bypasses the typical paradigm of noise-to-data diffusion guided by a specific modality condition, instead learning a reversibly consistent flow model between visual and neural latent distributions. For the first time, visual encoding and decoding are reformulated as a time-dependent, reversible process within a shared latent space for unified modeling. Empirical results demonstrate that NeuroFlow achieves superior overall performance in visual encoding and decoding tasks with higher computational efficiency compared to any isolated methods. We further analyze principal factors that steer the model toward encoding-decoding consistency and, through brain functional analyses, demonstrate that NeuroFlow captures consistent activation patterns underlying neural variability. NeuroFlow marks a major step toward unified visual encoding and decoding from neural activity, providing mechanistic insights that inform future bidirectional visual brain-computer interfaces. Code and project page are publicly available at https://michaelmaiii.github.io/NeuroFlow-S.