元学习上下文实现免训练跨被试大脑解码
Meta-Learning In-Context Enables Training-Free Cross Subject Brain Decoding
对做视觉表征研究的人，这篇提供了一个「用少量图像-脑响应样例做上下文学习、无需微调即可逆推个体编码器」的范式，可作为评估视觉骨干网络与大脑表征对齐程度的探针，也提示把元学习/上下文条件化用于跨域个体差异建模。
原文依据 generalizes to novel subjects without any fine-tuning
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论从脑信号中进行视觉解码是计算机视觉与神经科学交叉领域的关键挑战，需要能够连接神经表征与视觉计算模型的方法。该领域的一个总体目标是实现可泛化的跨被试模型。实现这一目标的主要障碍是个体间神经表征存在巨大差异，迄今这需要对每个被试单独训练定制模型或分别微调。为应对这一挑战，我们提出一种元优化方法，用于从 fMRI 中进行语义视觉解码，且无需任何微调即可泛化到新被试。只需以来自新个体的少量图像-大脑激活样例作为条件，我们的模型即可快速推断其独特的神经编码模式，从而实现稳健高效的视觉解码。我们的方法显式地针对新被试编码模型的上下文学习进行优化，并通过层级推理、逆推编码器来完成解码。首先，针对多个脑区，我们通过在多个刺激与响应上构建上下文来估计每个体素的视觉响应编码器参数。其次，我们构建由多个体素上的编码器参数与响应值组成的上下文，以执行聚合的功能逆推。我们在多种视觉骨干网络上展示了强大的跨被试与跨扫描仪泛化能力，且无需重新训练或微调。此外，我们的方法既不需要解剖对齐，也不需要刺激重叠。这项工作朝着面向非侵入式大脑解码的可泛化基础模型迈出了关键一步。
英文原文摘要 1655 字符
Visual decoding from brain signals is a key challenge at the intersection of computer vision and neuroscience, requiring methods that bridge neural representations and computational models of vision. A field-wide goal is to achieve generalizable, cross-subject models. A major obstacle towards this goal is the substantial variability in neural representations across individuals, which has so far required training bespoke models or fine-tuning separately for each subject. To address this challenge, we introduce a meta-optimized approach for semantic visual decoding from fMRI that **generalizes to novel subjects without any fine-tuning**. By simply conditioning on a small set of image-brain activation examples from the new individual, our model rapidly infers their unique neural encoding patterns to facilitate robust and efficient visual decoding. Our approach is explicitly optimized for in-context learning of the new subject's encoding model and performs decoding by hierarchical inference, inverting the encoder. First, for multiple brain regions, we estimate the per-voxel visual response encoder parameters by constructing a context over multiple stimuli and responses. Second, we construct a context consisting of encoder parameters and response values over multiple voxels to perform aggregated functional inversion. We demonstrate strong cross-subject and cross-scanner generalization across diverse visual backbones without retraining or fine-tuning. Moreover, our approach requires neither anatomical alignment nor stimulus overlap. This work is a critical step towards a generalizable foundation model for non-invasive brain decoding.