Duala：面向跨被试fMRI解码的主体与刺激双层对齐
Duala: Dual-Level Alignment of Subjects and Stimuli for Cross-Subject fMRI Decoding
提出刺激级语义对齐与关系一致性约束，以及基于分布的特征扰动机制，可迁移到其他跨域/跨被试的视觉表征对齐任务，如跨摄像头行人重识别中的域适应。
原文依据 Duala introduces a semantic alignment and relational consistency strategy that preserves intra-class similarity and inter-class separability
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论跨被试视觉解码旨在从不同个体的脑活动中重建视觉体验，从而构建更具可扩展性和实用性的脑机接口。然而，现有方法在适应数据有限的新被试时往往性能下降，因为它们难以同时保持刺激的语义一致性和脑响应的对齐。为解决这些挑战，我们提出Duala，一个双层对齐框架，旨在基于fMRI的跨被试视觉解码中实现刺激级一致性和被试级对齐。（1）在刺激层面，Duala引入语义对齐和关系一致性策略，保持类内相似性和类间可分性，在适应过程中维持清晰的语义边界。（2）在被试层面，开发了一种基于分布的特征扰动机制，以捕获全局和个体特异性变化，从而适应个体神经表征而不过拟合。在自然场景数据集（NSD）上的实验表明，Duala有效改善了跨被试对齐。值得注意的是，即使仅用约一小时的fMRI数据进行微调，Duala也能达到超过81.1%的图像到大脑检索准确率，并在检索和重建方面持续优于现有的微调策略。我们的代码可在https://github.com/ShumengLI/Duala获取。
英文原文摘要 1456 字符
Cross-subject visual decoding aims to reconstruct visual experiences from brain activity across individuals, enabling more scalable and practical brain-computer interfaces. However, existing methods often suffer from degraded performance when adapting to new subjects with limited data, as they struggle to preserve both the semantic consistency of stimuli and the alignment of brain responses. To address these challenges, we propose Duala, a dual-level alignment framework designed to achieve stimulus-level consistency and subject-level alignment in fMRI-based cross-subject visual decoding. (1) At the stimulus level, Duala introduces a semantic alignment and relational consistency strategy that preserves intra-class similarity and inter-class separability, maintaining clear semantic boundaries during adaptation. (2) At the subject level, a distribution-based feature perturbation mechanism is developed to capture both global and subject-specific variations, enabling adaptation to individual neural representations without overfitting. Experiments on the Natural Scenes Dataset (NSD) demonstrate that Duala effectively improves alignment across subjects. Remarkably, even when fine-tuned with only about one hour of fMRI data, Duala achieves over 81.1% image-to-brain retrieval accuracy and consistently outperforms existing fine-tuning strategies in both retrieval and reconstruction. Our code is available at https://github.com/ShumengLI/Duala.