证据变换网络:将预训练模型转化为证据模型以进行事后不确定性估计
Evidential Transformation Network: Turning Pretrained Models into Evidential Models for Post-hoc Uncertainty Estimation
对视觉研究者而言,ETN 提供了一种即插即用的后处理手段:只需在冻结的预训练分类器 logit 上学习一个样本相关的仿射变换,就能把任意现成模型转成可输出 Dirichlet 证据量的模型,用于 OOD 检测与置信度校准,无需重训或多次前向。
原文依据 a lightweight post-hoc module that converts a pretrained predictor into an evidential model
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论预训练模型已成为视觉与语言领域的标准配置,但它们通常无法提供可靠的置信度度量。现有的不确定性估计方法——例如深度集成与 MC dropout——往往计算开销过大,难以在实际中部署。证据深度学习(EDL)提供了一种更高效的替代方案,但它要求模型从一开始就被训练为输出证据量,而预训练网络很少满足这一条件。为使预训练模型能够进行 EDL 风格的不确定性估计,我们提出了证据变换网络(ETN),这是一个轻量级的事后模块,可将预训练预测器转换为证据模型。ETN 在 logit 空间中运作:它学习一个依赖于样本的 logit 仿射变换,并将变换后的输出解释为用于不确定性估计的 Dirichlet 分布参数。我们在图像分类与大语言模型问答基准上,于分布内和分布外两种设定下评估了 ETN。ETN 在保持准确率、仅增加极小计算开销的同时,相较事后基线方法稳定地提升了不确定性估计效果。
英文原文摘要 1198 字符
Pretrained models have become standard in both vision and language, yet they typically do not provide reliable measures of confidence. Existing uncertainty estimation methods--such as deep ensembles and MC dropout--are often too computationally expensive to deploy in practice. Evidential Deep Learning (EDL) offers a more efficient alternative, but it requires models to be trained to output evidential quantities from the start, which is rarely true for pretrained networks. To enable EDL-style uncertainty estimation in pretrained models, we propose the Evidential Transformation Network (ETN), a lightweight post-hoc module that converts a pretrained predictor into an evidential model. ETN operates in logit space: it learns a sample-dependent affine transformation of the logits and interprets the transformed outputs as parameters of a Dirichlet distribution for uncertainty estimation. We evaluate ETN on image classification and large language model question-answering benchmarks, under both in-distribution and out-of-distribution settings. ETN consistently improves uncertainty estimation over post-hoc baselines, while preserving accuracy and adding only minimal computational overhead.