自然图像自编码器能否紧凑地词元化 fMRI 体积以进行长程动力学建模？
Can Natural Image Autoencoders Compactly Tokenize fMRI Volumes for Long-Range Dynamics Modeling?
把预训练的 2D 自然图像自编码器当作 fMRI 体数据的词元化器，将每个 3D 体积压成少量连续词元，从而用普通 Transformer 编码器在有限显存下做长程时空建模；这套「自然图像编码器跨域复用 + 掩码词元建模预训练」的思路可直接迁移到其他高维医学/科学体数据的序列化建模。
原文依据 tokenizes fMRI volumes using a pre-trained 2D natural image autoencoder
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论由于四维信号的高维性，对功能磁共振成像（fMRI）中的长程时空动力学进行建模仍是一项关键挑战。此前的基于体素的模型虽然展现出优异的性能和可解释能力，但受制于高昂的显存需求，因而只能覆盖有限的时间窗口。为解决这一问题，我们提出 TABLeT（二维自编码脑潜变量 Transformer），一种利用预训练的二维自然图像自编码器对 fMRI 体积进行词元化的新方法。每个三维 fMRI 体积被压缩为一组紧凑的连续词元，使得仅用显存有限的简单 Transformer 编码器即可进行长序列建模。在包括 UK-Biobank（UKB）、Human Connectome Project（HCP）和 ADHD-200 数据集在内的大规模基准上，TABLeT 在多项任务中优于现有模型，同时在相同输入下相较最优的基于体素的方法在计算与显存效率上取得显著提升。此外，我们开发了一种自监督掩码词元建模方法来预训练 TABLeT，提升了模型在多种下游任务上的性能。我们的发现为脑活动的可扩展且可解释的时空建模提供了一条有前景的途径。代码已公开于 https://github.com/beotborry/TABLeT。
英文原文摘要 1385 字符
Modeling long-range spatiotemporal dynamics in functional Magnetic Resonance Imaging (fMRI) remains a key challenge due to the high dimensionality of the four-dimensional signals. Prior voxel-based models, although demonstrating excellent performance and interpretation capabilities, are constrained by prohibitive memory demands and thus can only capture limited temporal windows. To address this, we propose TABLeT (Two-dimensionally Autoencoded Brain Latent Transformer), a novel approach that tokenizes fMRI volumes using a pre-trained 2D natural image autoencoder. Each 3D fMRI volume is compressed into a compact set of continuous tokens, enabling long-sequence modeling with a simple Transformer encoder with limited VRAM. Across large-scale benchmarks including the UK-Biobank (UKB), Human Connectome Project (HCP), and ADHD-200 datasets, TABLeT outperforms existing models in multiple tasks, while demonstrating substantial gains in computational and memory efficiency over the state-of-the-art voxel-based method given the same input. Furthermore, we develop a self-supervised masked token modeling approach to pre-train TABLeT, which improves the model's performance for various downstream tasks. Our findings suggest a promising approach for scalable and interpretable spatiotemporal modeling of brain activity. Our code is available at https://github.com/beotborry/TABLeT.