面向多模态失败检测的自适应置信度正则化
Adaptive Confidence Regularization for Multimodal Failure Detection
为多模态视觉模型（如RGB+深度+光流的自动驾驶感知）提供了一种可复用的失败检测训练策略：用自适应置信度损失约束多模态预测置信度不低于单模态分支，并用特征交换合成失败样本作为难负例，可直接嵌入现有多模态融合框架提升拒识能力。
原文依据 the confidence of the multimodal prediction is significantly lower than that of at least one unimodal branch
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论在自动驾驶车辆与医学诊断等高风险领域部署多模态模型，不仅要求强大的预测性能，还需要可靠的失败检测机制。本文研究多模态场景下尚少被探索的失败检测问题，提出自适应置信度正则化（ACR），一个专门用于检测多模态失败的新框架。该方法基于一个关键观察：在大多数失败案例中，多模态预测的置信度显著低于至少一个单模态分支，我们将这一现象称为置信度退化。为缓解该问题，我们引入自适应置信度损失，在训练过程中对这类退化施加惩罚。此外，我们提出多模态特征交换，一种新的离群样本合成技术，用于生成具有挑战性且与失败相关的训练样本。通过使用这些合成失败样本进行训练，ACR 学会更有效地识别并拒绝不确定的预测，从而提升整体可靠性。在四个数据集、三种模态以及多种评测设置上的大量实验表明，ACR 取得了一致且稳健的性能提升。源代码将发布于 https://github.com/mona4399/ACR。
英文原文摘要 1302 字符
The deployment of multimodal models in high-stakes domains, such as self-driving vehicles and medical diagnostics, demands not only strong predictive performance but also reliable mechanisms for detecting failures. In this work, we address the largely unexplored problem of failure detection in multimodal contexts. We propose Adaptive Confidence Regularization (ACR), a novel framework specifically designed to detect multimodal failures. Our approach is driven by a key observation: in most failure cases, the confidence of the multimodal prediction is significantly lower than that of at least one unimodal branch, a phenomenon we term confidence degradation. To mitigate this, we introduce an Adaptive Confidence Loss that penalizes such degradations during training. In addition, we propose Multimodal Feature Swapping, a novel outlier synthesis technique that generates challenging, failure-aware training examples. By training with these synthetic failures, ACR learns to more effectively recognize and reject uncertain predictions, thereby improving overall reliability. Extensive experiments across four datasets, three modalities, and multiple evaluation settings demonstrate that ACR achieves consistent and robust gains. The source code will be available at https://github.com/mona4399/ACR.