MLLM-HWSI：面向层次化全切片图像理解的多模态大语言模型
MLLM-HWSI: A Multimodal Large Language Model for Hierarchical Whole Slide Image Understanding
将全切片图像分解为细胞、图像块、区域、WSI四个尺度，并用尺度特定投影器与层次对比+跨尺度一致性损失对齐病理语言，为视觉-语言模型提供可解释的多尺度证据推理框架，可迁移到其他需要多尺度理解的视觉领域。
原文依据 aligns visual features with pathology language at four distinct scales, cell as word, patch as phrase, region as sentence, and WSI as paragraph
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论全切片图像（WSI）呈现层次化结构，诊断信息源自细胞形态、区域组织架构和全局上下文。现有的计算病理学（CPath）多模态大语言模型（MLLM）通常将整个WSI压缩为单一嵌入，这阻碍了细粒度视觉定位，并忽略了病理学家如何综合不同尺度的证据。我们提出MLLM-HWSI，一种层次化WSI级MLLM，在四个不同尺度上将视觉特征与病理语言对齐：细胞作为词、图像块作为短语、区域作为句子、WSI作为段落，以支持可解释的、基于证据的推理。MLLM-HWSI通过尺度特定的投影器将每个WSI分解为多尺度嵌入，并联合施加（i）层次对比目标和（ii）跨尺度一致性损失，从而保持从细胞到WSI的语义连贯性。我们计算诊断相关的图像块，并使用轻量级细胞-细胞注意力融合（CCAF）Transformer将分割后的细胞嵌入聚合为每个图像块的紧凑细胞词元。投影后的多尺度词元与文本词元融合，输入指令微调的大语言模型，用于开放式推理、视觉问答、报告生成和描述生成任务。经过三阶段训练，MLLM-HWSI在六个CPath任务的13个WSI级基准上取得了新的最先进结果。通过将语言与多尺度视觉证据对齐，MLLM-HWSI提供了准确、可解释的输出，反映了诊断工作流程，并推进了整体WSI理解。代码见：https://github.com/BasitAlawode/HWSI-MLLM Github。
英文原文摘要 1648 字符
Whole Slide Images (WSIs) exhibit hierarchical structure, where diagnostic information emerges from cellular morphology, regional tissue organization, and global context. Existing Computational Pathology (CPath) Multimodal Large Language Models (MLLMs) typically compress an entire WSI into a single embedding, which hinders fine-grained grounding and ignores how pathologists synthesize evidence across different scales. We introduce MLLM-HWSI, a Hierarchical WSI-level MLLM that aligns visual features with pathology language at four distinct scales, cell as word, patch as phrase, region as sentence, and WSI as paragraph to support interpretable evidence-grounded reasoning. MLLM-HWSI decomposes each WSI into multi-scale embeddings with scale-specific projectors and jointly enforces (i) a hierarchical contrastive objective and (ii) a cross-scale consistency loss, preserving semantic coherence from cells to the WSI. We compute diagnostically relevant patches and aggregate segmented cell embeddings into a compact cellular token per-patch using a lightweight Cell-Cell Attention Fusion (CCAF) transformer. The projected multi-scale tokens are fused with text tokens and fed to an instruction-tuned LLM for open-ended reasoning, VQA, report, and caption generation tasks. Trained in three stages, MLLM-HWSI achieves new SOTA results on 13 WSI-level benchmarks across six CPath tasks. By aligning language with multi-scale visual evidence, MLLM-HWSI provides accurate, interpretable outputs that mirror diagnostic workflows and advance holistic WSI understanding. Code is available at: \href https://github.com/BasitAlawode/HWSI-MLLM Github .