fMRI-LM：迈向面向语言对齐 fMRI 理解的通用基础模型
fMRI-LM: Towards a Universal Foundation Model for Language-Aligned fMRI Understanding
把连续神经信号离散化为语言对齐 token 的分词器思路，可迁移到视频/时序视觉信号与语言模型的对齐；三阶段（分词器→联合预训练→多任务指令微调）流程为视觉-语言基础模型提供了可复用的训练范式。
原文依据 we learn a neural tokenizer that maps fMRI into discrete tokens embedded in a language-consistent space
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多模态大语言模型（LLM）的最新进展已实现跨图像、音频与视频的统一推理，但将这一能力扩展到脑成像领域仍基本未被探索。弥合这一差距对于连接神经活动与语义认知、发展跨模态大脑表征至关重要。为此，我们提出 fMRI-LM，一个通过三阶段框架将功能磁共振成像（fMRI）与语言相连接的基础模型。在第一阶段，我们学习一个神经分词器，将 fMRI 映射为嵌入语言一致空间的离散 token。在第二阶段，预训练 LLM 被适配为联合建模 fMRI token 与文本，将大脑活动视为一个可被时间预测并可用语言描述的序列。为克服自然 fMRI-文本配对数据的缺乏，我们构建了一个大型描述性语料库，将多种基于成像的特征转化为结构化文本描述符，以刻画 fMRI 信号的低层组织。在第三阶段，我们进行多任务、多范式的指令微调，使 fMRI-LM 具备高层语义理解能力，支持多样化的下游应用。在多个基准上，fMRI-LM 取得了强劲的零样本与少样本性能，并可通过参数高效微调（LoRA）高效适配，为面向 fMRI 结构与语义理解的可扩展、语言对齐通用模型奠定了基础。
英文原文摘要 1431 字符
Recent advances in multimodal large language models (LLMs) have enabled unified reasoning across images, audio, and video, but extending such capability to brain imaging remains largely unexplored. Bridging this gap is essential to link neural activity with semantic cognition and to develop cross-modal brain representations. To this end, we present fMRI-LM, a foundational model that bridges functional MRI (fMRI) and language through a three-stage framework. In Stage 1, we learn a neural tokenizer that maps fMRI into discrete tokens embedded in a language-consistent space. In Stage 2, a pretrained LLM is adapted to jointly model fMRI tokens and text, treating brain activity as a sequence that can be temporally predicted and linguistically described. To overcome the lack of natural fMRI-text pairs, we construct a large descriptive corpus that translates diverse imaging-based features into structured textual descriptors, capturing the low-level organization of fMRI signals. In Stage 3, we perform multi-task, multi-paradigm instruction tuning to endow fMRI-LM with high-level semantic understanding, supporting diverse downstream applications. Across various benchmarks, fMRI-LM achieves strong zero-shot and few-shot performance, and adapts efficiently with parameter-efficient tuning (LoRA), establishing a scalable pathway toward a language-aligned, universal model for structural and semantic understanding of fMRI.