MultiModalPFN：将先验数据拟合网络扩展到多模态表格学习
MultiModalPFN: Extending Prior-Data Fitted Networks for Multimodal Tabular Learning
把图像/文本编码器输出通过模态投影器（多头门控 MLP + 交叉注意力采样器）转成表格兼容 token，使 TabPFN 式先验数据拟合网络能直接吃视觉特征；做视觉+表格联合预测（如医学影像配临床指标）时可复用这套跨模态 token 对齐与注意力失衡缓解思路。
原文依据 The modality projectors serve as the critical bridge, transforming non-tabular embeddings into tabular-compatible tokens for unified processing.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近期，TabPFN 作为表格数据的基础模型受到关注。然而，它难以整合图像和文本等异构模态，而这些模态在医疗、营销等领域十分常见，从而限制了其适用性。为此，我们提出多模态先验数据拟合网络（MMPFN），将 TabPFN 扩展为以统一方式处理表格与非表格模态。MMPFN 由各模态编码器、模态投影器和预训练基础模型组成。模态投影器是关键桥梁，将非表格嵌入转换为与表格兼容的 token 以进行统一处理。为此，我们引入多头门控 MLP 和交叉注意力采样器，从非表格输入中提取更丰富的上下文，同时缓解多模态学习中的注意力失衡问题。在医学和通用多模态数据集上的大量实验表明，MMPFN 持续优于有竞争力的最优方法，并有效利用非表格模态与表格特征。这些结果凸显了将先验数据拟合网络扩展到多模态设定的前景，为异构数据学习提供了一个可扩展且有效的框架。
英文原文摘要 1274 字符
Recently, TabPFN has gained attention as a foundation model for tabular data. However, it struggles to integrate heterogeneous modalities such as images and text, which are common in domains like healthcare and marketing, thereby limiting its applicability. To address this, we present the Multi-Modal Prior-data Fitted Network (MMPFN), which extends TabPFN to handle tabular and non-tabular modalities in a unified manner. MMPFN comprises per-modality encoders, modality projectors, and pre-trained foundation models. The modality projectors serve as the critical bridge, transforming non-tabular embeddings into tabular-compatible tokens for unified processing. To this end, we introduce a multi-head gated MLP and a cross-attention sampler that extract richer context from non-tabular inputs while mitigates attention imbalance issue in multimodal learning. Extensive experiments on medical and general-purpose multimodal datasets demonstrate that MMPFN consistently outperforms competitive state-of-the-art methods and effectively exploits non-tabular modalities alongside tabular features. These results highlight the promise of extending prior-data fitted networks to the multimodal setting, offering a scalable and effective framework for heterogeneous data learning.