MDCS-MoAME：面向癌症生存预测的多方向复合扫描与注意力及 Mamba 混合专家
MDCS-MoAME: Multi-directional Composite Scanning with Mixture of Attention and Mamba Experts for Cancer Survival Prediction
把 Mamba 的多方向复合扫描策略用在千兆像素全切片图像上，为处理超长序列的病理图像提供了一种可复用的序列化与建模思路；注意力与 Mamba 混合专家动态路由也可迁移到其他多模态视觉任务。
原文依据 we introduce a multi-directional composite scanning (MDCS) strategy to both WSIs and genomic profiles
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论融合病理图像与基因组图谱的多模态学习方法显著提升了生存预测任务的准确性。然而，由于传统扫描策略在数据序列化方面的局限，以及模态本身复杂且异质的特性，以往方法往往难以有效处理长程的千兆像素全切片图像（WSIs）与稀疏的基因组图谱。受 Mamba 与混合专家（MoE）近期进展的启发，我们提出一种新的多方向复合扫描策略与注意力及 Mamba 混合专家（MDCS-MoAME），用于癌症生存预测。具体而言，我们针对 WSIs 与基因组图谱引入多方向复合扫描（MDCS）策略，并使用 Mamba 编码器在区域、图像块与基因三个层级上处理模态内表征，从而充分利用每个模态的内在信息。为进一步捕获异质的模态间表征，我们引入注意力与 Mamba 混合专家（MoAME），它动态选择定制化的专家来建模复杂的模态间相关性，灵活地关注模态之间的交互。最后，我们引入对齐约束以重新校准模态间交互并减少模态内与模态间的表征冗余，增强其对全面生存分析的判别能力。在五个公开数据集上的实验结果表明，我们的方法优于现有方法，取得了最优性能。
英文原文摘要 1605 字符
Multi-modal learning approaches that integrate pathological images with genomic profiles have significantly enhanced the accuracy of survival prediction tasks. However, previous methods often struggle to effectively process long-range gigapixel whole slide images (WSIs) and sparse genomic profiles due to the limitations of conventional scanning strategies to serialize data and the complex and heterogeneous nature of the modalities. Inspired by recent advancements in Mamba and mixture of experts (MoE), we propose a novel multi-directional composite scanning strategy with mixture of attention and Mamba experts (MDCS-MoAME) for cancer survival prediction. Specifically, we introduce a multi-directional composite scanning (MDCS) strategy to both WSIs and genomic profiles, and use the Mamba encoder to process intra-modal representations at the region, patch, and gene level, ensuring sufficient utilization of the intrinsic information within each modality. To further capture heterogeneous inter-modal representations, we introduce mixture of attention and Mamba experts (MoAME), which dynamically selects tailored experts to model complex inter-modal correlations, flexibly focusing on the interactions between modalities. Finally, we introduce alignment constraints to recalibrate inter-modal interactions and reduce intra- and inter-modal representation redundancy, enhancing its discriminative power for comprehensive survival analysis. Experimental results on five publicly available datasets demonstrate that our method outperforms existing approaches, achieving state-of-the-art performance.