FBTA:通过特征桥接与翻译对齐实现单GPU端到端千兆像素WSI分类
FBTA: Enabling Single-GPU End-to-End Gigapixel WSI Classification with Feature Bridging and Translation Alignment
提出一种在单张24GB GPU上端到端训练WSI分类的MIL框架,通过特征桥接和翻译对齐解决冻结特征提取器带来的语义鸿沟,可复用于其他需要端到端训练的大图像分类任务。
原文依据 FBTA is the first end-to-end MIL framework trainable on a single 24\,GB GPU
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论计算病理学中的全切片图像(WSI)包含数十亿像素,使得在单张商用GPU上端到端训练特征提取器和多实例学习(MIL)网络不可行。现有方法通常冻结特征提取器并在得到的冻结特征上训练MIL网络,这引入了语义鸿沟,限制了下游性能。为解决此问题,我们提出FBTA,一种用于WSI分类的特征桥接与翻译对齐框架。FBTA是首个可在单张24GB GPU上训练的端到端MIL框架,利用三种互补的特征包视图:端到端特征支持联合优化,冻结特征稳定训练,翻译特征支持实际推理。在包括TCGA-NSCLC(Shot20/50/100)和TCGA-STAD在内的多个数据集上的实验证明了FBTA的有效性和通用性,其在三种MIL架构和两种提取器上均持续提升性能。例如,以ResNet-50作为提取器时,FBTA在NSCLC-Shot50和TCGA-STAD数据集上分别将经典ABMIL的准确率提升13.1%和15.8%,并进一步将最优的MambaMIL在相同数据集上分别提升4.1%和9.2%。此外,FBTA对于结合自监督预训练策略和数据增强技术的MIL模型还能带来额外增益。这些结果表明FBTA是一种在千兆像素WSI上进行端到端MIL的可行且可扩展的框架。代码将公开。
英文原文摘要 1531 字符
Whole-slide images (WSIs) in computational pathology contain billions of pixels, making end-to-end training of feature extractors and multi-instance learning (MIL) networks infeasible on a single commodity GPU.Existing methods often freeze the feature extractor and train MIL networks on the resulting frozen features, which introduces a semantic gap that limits downstream performance. To address this issue, we propose FBTA, a Feature Bridging and Translation Alignment framework for WSI classification. FBTA is the first end-to-end MIL framework trainable on a single 24\,GB GPU, leveraging three complementary feature-bag views: end-to-end features enable joint optimization, frozen features stabilize training, and translated features support practical inference.Experiments on diverse datasets, including TCGA-NSCLC (Shot20/50/100) and TCGA-STAD, demonstrate the effectiveness and generality of FBTA, which consistently improves performance across three MIL architectures and two extractors. For example, with ResNet-50 as the extractor, FBTA improves the accuracy of the classic ABMIL by 13.1% and 15.8% on the NSCLC-Shot50 and TCGA-STAD datasets, respectively, and further enhances the state-of-the-art MambaMIL by 4.1% and 9.2% on the same datasets. Moreover, FBTA yields additional gains for MIL models that incorporate self-supervised pretraining strategies and data augmentation techniques.These results suggest FBTA is a feasible and scalable framework for end-to-end MIL on gigapixel WSIs. The code will be available.