重新审视2D基础模型用于可扩展的3D医学图像分类
Revisiting 2D Foundation Models for Scalable 3D Medical Image Classification
将2D基础模型通过轻量插件适配到3D医学分类,并系统评估多种适配策略,为视觉领域利用大规模2D预训练模型解决3D任务提供了可复用的范式。
原文依据 introduce AnyMC3D, a scalable 3D classifier adapted from 2D FMs
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论3D医学图像分类对现代临床工作流程至关重要。医学基础模型已成为扩展到新任务的一种有前景的方法,然而当前研究存在三个关键缺陷:数据机制偏差、次优适应以及任务覆盖不足。本文针对这些缺陷,提出AnyMC3D,一种从2D基础模型适配而来的可扩展3D分类器。它通过仅向单个冻结骨干网络添加轻量插件(每个任务约1M参数)即可高效扩展到新任务。此外,该通用框架还支持多视图输入、辅助像素级监督以及可解释热图生成。我们建立了涵盖多种病理、解剖结构和模态的12个任务的综合基准,并系统评估了最优的3D分类技术。分析揭示了若干关键见解:(1)有效适应对于释放基础模型潜力至关重要;(2)通用基础模型在适当适应后可与医学专用基础模型媲美;(3)基于2D的方法在3D分类上超越3D架构。我们首次证明了使用单一可扩展框架在多种应用中达到最优性能的可行性(例如在***挑战赛中获得第一名),从而无需单独的任务特定3D模型。
英文原文摘要 1371 字符
3D medical image classification is essential to modern clinical workflows. Medical foundation models (FMs) have emerged as a promising approach for scaling to new tasks, yet current research suffers from three critical pitfalls: data-regime bias, suboptimal adaptation, and insufficient task coverage. In this paper, we address these pitfalls and introduce AnyMC3D, a scalable 3D classifier adapted from 2D FMs. It allows efficient scaling to new tasks by adding only lightweight plugins ( 1M parameters per task) to a single frozen backbone. Besides, this versatile framework also supports multi-view inputs, auxiliary pixel-level supervision, and interpretable heatmap generation. We establish a comprehensive benchmark of 12 tasks covering diverse pathologies, anatomies, and modalities and systematically evaluate state-of-the-art 3D classification techniques. Our analysis reveals several key insights: (1) effective adaptation is critical to unlock FM potential, (2) general-purpose FMs can match medical-specific FMs if properly adapted, and (3) 2D-based methods surpass 3D architectures for 3D classification. For the first time, we demonstrate the feasibility of achieving state-of-the-art performance across diverse applications using a single scalable framework (e.g., 1st place in the *** challenge), eliminating the need for separate task-specific 3D models.