鲁棒性与安全 · CVPR2026 ·2026.06.03

LATA:面向医学视觉语言模型保形不确定性的拉普拉斯辅助传导式自适应

LATA: Laplacian-Assisted Transductive Adaptation for Conformal Uncertainty in Medical VLMs

Behzad Bozorgtabar、Dwarikanath Mahapatra、Sudipta Roy、Muzammal Naseer、Imran Razzak、Zongyuan Ge

与你有关的部分

把保形预测的预测集效率与类别条件覆盖率差距作为可靠性指标,用图像 kNN 图上的 CCCP 均值场传导式平滑在无标签、无训练条件下收紧零样本 VLM 的预测集,可直接迁移到任何需要带覆盖保证的不确定性输出的视觉识别流程。

原文依据 a training- and label-free refinement that operates on the joint calibration and test pool by smoothing zero-shot probabilities over an image-image kNN graph
可迁移到 医学图像分类零样本识别不确定性估计

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

医学视觉语言模型(VLM)是医学影像的强零样本识别器,但其在域偏移下的可靠性取决于带有保证的校准不确定性。分裂保形预测(SCP)提供有限样本覆盖保证,然而预测集往往过大(效率低)且类别间覆盖不均衡——类别条件覆盖差距(CCV)偏高,在少样本、类别不平衡的情形下尤为突出;此外,朴素地适应校准标签会破坏可交换性,使保证失效。我们提出 LATA(拉普拉斯辅助传导式自适应),一种无需训练、无需标签的精细化方法:它在校准与测试的联合池上,通过少量 CCCP 均值场更新在图像—图像 kNN 图上平滑零样本概率,并以确定性变换保持 SCP 的有效性。我们进一步引入一种失效感知的保形得分,将其接入视觉—语言不确定性(ViLU)框架,提供实例级难度与标签合理性,从而在固定覆盖率下提升预测集效率与类别间均衡性。LATA 是黑盒方法(不更新 VLM)、计算轻量(窗口化传导、无反向传播),并包含一个可选的先验调节旋钮,既可严格无标签运行,也可在需要时以一次性使用校准边缘分布的标签知情变体运行。在三个医学 VLM 与九个下游任务上,LATA 在匹配或收紧目标覆盖率的同时,一致地降低预测集规模与 CCV,优于先前的传导式基线,并缩小了与使用标签的方法之间的差距,而计算开销远低。全面的消融与定性分析表明,LATA 在不损害可交换性的前提下锐化了零样本预测。

英文原文摘要 1729 字符

Medical vision-language models (VLMs) are strong zero-shot recognizers for medical imaging, but their reliability under domain shift hinges on calibrated uncertainty with guarantees. Split conformal prediction (SCP) offers finite-sample coverage, yet prediction sets often become large (low efficiency) and class-wise coverage unbalanced--high class-conditioned coverage gap (CCV), especially in few-shot, imbalanced regimes; moreover, naively adapting to calibration labels breaks exchangeability and voids guarantees. We propose LATA (Laplacian-Assisted Transductive Adaptation), a training- and label-free refinement that operates on the joint calibration and test pool by smoothing zero-shot probabilities over an image-image kNN graph using a small number of CCCP mean-field updates, preserving SCP validity via a deterministic transform. We further introduce a failure-aware conformal score that plugs into the vision-language uncertainty (ViLU) framework, providing instance-level difficulty and label plausibility to improve prediction set efficiency and class-wise balance at fixed coverage. LATA is black-box (no VLM updates), compute-light (windowed transduction, no backprop), and includes an optional prior knob that can run strictly label-free or, if desired, in a label-informed variant using calibration marginals once. Across three medical VLMs and nine downstream tasks, LATA consistently reduces set size and CCV while matching or tightening target coverage, outperforming prior transductive baselines and narrowing the gap to label-using methods, while using far less compute. Comprehensive ablations and qualitative analyses show that LATA sharpens zero-shot predictions without compromising exchangeability.

标签

会议 CVPR2026
页码 36311-36320
Bibkey
来源 CVF Open Access