医学影像与生物视觉 · CVPR2026 ·2026.06.03

MUSE:利用精确且多样的语义进行少样本全切片图像分类

MUSE: Harnessing Precise and Diverse Semantics for Few-Shot Whole Slide Image Classification

Jiahao Xu、Sheng Huang、Xin Zhang、Zhixiong Nan、Jiajun Dong、Nankun Mu

与你有关的部分

对做视觉研究的人,这篇论文提供了一个可复用的思路:把 LLM 生成的类级文本先验,通过基于 MoE 的样本级视觉-语义交互细化为逐样本先验,再用检索增强的多视图文本做随机语义监督,从而在少样本下缓解过拟合并提升视觉-语义对齐的精度与多样性;该「先细化再增广」的语义监督流程可迁移到其他少样本视觉-语言任务。

原文依据 retrieves and stochastically integrates multiple matched textual views during training
可迁移到 少样本图像分类视觉-语言对齐全切片图像分类

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

在计算病理学中,少样本全切片图像分类主要受制于专家标注切片的极度稀缺。近期的视觉-语言方法引入由大语言模型生成的文本语义,但将这些描述视为静态的类级先验,在所有样本间共享且缺乏样本级细化。这限制了视觉-语义对齐的多样性与精确性,削弱了有限监督下的泛化能力。为克服这一问题,我们提出随机多视图语义增强(MUSE)框架,先通过样本级自适应细化语义精度,再通过检索增强的多视图生成提升语义丰富度。具体而言,MUSE 引入样本级细粒度语义增强(SFSE),通过基于混合专家(MoE)的自适应视觉-语义交互为每个样本生成细粒度语义先验。在该先验引导下,随机多视图模型优化(SMMO)为每个类别构建由大语言模型生成的多样化病理描述知识库,并在训练过程中检索并随机整合多个匹配的文本视图。这些动态选取的文本作为增强的语义监督,对视觉-语言模型进行随机优化,从而提升鲁棒性并缓解过拟合。在三个基准 WSI 数据集上的实验表明,MUSE 在少样本设置下持续优于现有的视觉-语言基线,说明有效的少样本病理学习不仅需要更丰富的语义来源,还需要对其开展主动且样本感知的语义优化。

英文原文摘要 1668 字符

In computational pathology, few-shot whole slide image classification is primarily driven by the extreme scarcity of expert-labeled slides. Recent vision-language methods incorporate textual semantics generated by large language models, but treat these descriptions as static class-level priors that are shared across all samples and lack sample-wise refinement. This limits both the diversity and precision of visual-semantic alignment, hindering generalization under limited supervision. To overcome this, we propose the stochastic MUlti-view Semantic Enhancement (MUSE), a framework that first refines semantic precision via sample-wise adaptation and then enhances semantic richness through retrieval-augmented multi-view generation. Specifically, MUSE introduces Sample-wise Fine-grained Semantic Enhancement (SFSE), which yields a fine-grained semantic prior for each sample through MoE-based adaptive visual-semantic interaction. Guided by this prior, Stochastic Multi-view Model Optimization (SMMO) constructs an LLM-generated knowledge base of diverse pathological descriptions per class, then retrieves and stochastically integrates multiple matched textual views during training. These dynamically selected texts serve as enriched semantic supervisions to stochastically optimize the vision-language model, promoting robustness and mitigating overfitting. Experiments on three benchmark WSI datasets show that MUSE consistently outperforms existing vision-language baselines in few-shot settings, demonstrating that effective few-shot pathology learning requires not only richer semantic sources but also their active and sample-aware semantic optimization.

标签

会议 CVPR2026
页码 33911-33921
Bibkey
来源 CVF Open Access