将预训练视觉 Transformer 转变为端到端病理全切片图像模型以用于生存预测
Turning Pre-Trained Vision Transformers into End-to-End Histopathology Whole Slide Image Models for Survival Prediction
把预训练 ViT 直接改造成端到端全切片图像模型，省去两阶段流程中的离线特征提取与 MIL 聚合，且不增加参数；对做病理或超大分辨率图像分析的人，这是一种可复用的「免额外参数、直接吃整张组织区域」的改造思路。
原文依据 transforms a vanilla ViT into an end-to-end pre-trained WSI model without introducing additional parameters
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论传统的全切片图像（WSI）分析流程遵循两阶段过程。首先，使用图像编码器（如视觉 Transformer，ViT）对从 WSI 中裁剪出的一系列图块进行批量离线特征提取。其次，使用切片级标签训练多实例学习（MIL）模型，以获得任务特定的切片嵌入。然而，该流程存在若干局限：对图块编码器预训练权重的强依赖、缺乏来自原始图像的感受野，以及缺少任务无关的 WSI 表征。理想的改进是开发端到端的预训练 WSI 模型，但从零开始训练将面临训练成本高和计算复杂度大等挑战。在本工作中，我们解构了基于 ViT 的病理图像表征的关键步骤，并提出一种名为 E2E-ViT 的转换策略，可将普通 ViT 转换为端到端的预训练 WSI 模型，且不引入额外参数。E2E-ViT 直接输入 WSI 中的整个组织区域，以高效地将图像序列送入 Transformer 主干，实现来自原始感受野的信息交互并生成切片特征。通过多项生存预测任务，我们证明转换后的预训练 ViT 优于两阶段 MIL 模型和切片基础模型（SFM）。我们的工作提出了一种新的端到端学习范式，为下一代计算病理模型提供了有前景的方向。
英文原文摘要 1542 字符
Conventional whole slide image (WSI) analysis pipelines follow a two-stage process. First, an image encoder, such as a vision transformer (ViT), is used to perform batched offline feature extraction on a series of tiles cropped from the WSI. Second, a multiple instance learning (MIL) model is trained with slide-level labels to obtain task-specific slide embeddings. However, several limitations exist: strong reliance on pre-trained weights of the tile encoder, the absence of receptive fields from the original image, and a lack of task-independent WSI representations. An ideal improvement would be to develop an end-to-end pre-trained WSI model, but training it from scratch will face challenges such as high training costs and computational complexity. In this work, we deconstruct the key steps of ViT-based pathology image representation and propose a conversion strategy called E2E-ViT, which transforms a vanilla ViT into an end-to-end pre-trained WSI model without introducing additional parameters. E2E-ViT directly inputs the entire tissue region in WSIs to efficiently feed image sequences into the transformer backbone, achieving information interaction from the original receptive fields and generating slide features. Through multiple survival prediction tasks, we demonstrate that transformed pre-trained ViTs outperform two-stage MIL models and slide foundation models (SFM). Our work presents a new end-to-end learning paradigm that provides a promising direction for the next generation of computational pathology models.