一种具有跨模态和跨任务效用保证的视觉语言模型去偏闭式解
A Closed-Form Solution for Debiasing Vision-Language Models with Utility Guarantees Across Modalities and Tasks
提出一种无需训练、无需标注数据的闭式去偏方法，可同时去偏视觉和文本模态，在保持任务性能的同时实现帕累托最优公平性，可直接用于零样本图像分类、文本到图像检索和生成等视觉任务。
原文依据 Our method is training-free, requires no annotated data, and can jointly debias both visual and textual modalities across downstream tasks.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论尽管视觉语言模型（VLM）在多种下游任务中取得了显著性能，但近期研究表明它们可能从训练数据中继承社会偏见，并进一步传播到下游应用中。为解决这一问题，已有多种去偏方法被提出，但大多数方法旨在提升公平性，却没有理论保证模型效用得以保持。本文提出一种去偏方法，在跨模态空间中给出闭式解，在效用损失有界的情况下实现帕累托最优公平性。该方法无需训练、无需标注数据，并且能够跨下游任务同时对视觉和文本模态进行去偏。大量实验表明，在零样本图像分类、文本到图像检索和文本到图像生成等下游任务中，我们的方法在多种公平性指标和数据集上（包括群体公平性和交叉公平性）优于现有VLM去偏方法，同时保持任务性能。代码将在论文被接收后公开。
英文原文摘要 1126 字符
While Vision-Language Models (VLMs) have achieved remarkable performance across diverse downstream tasks, recent studies have shown that they can inherit social biases from the training data and further propagate them into downstream applications. To address this issue, various debiasing approaches have been proposed, yet most of them aim to improve fairness without having a theoretical guarantee that the utility of the model is preserved. In this paper, we introduce a debiasing method that yields a closed-form solution in the cross-modal space, achieving Pareto-optimal fairness with bounded utility losses. Our method is training-free, requires no annotated data, and can jointly debias both visual and textual modalities across downstream tasks. Extensive experiments show that our method outperforms existing methods in debiasing VLMs across diverse fairness metrics and datasets for both group and intersectional fairness in downstream tasks such as zero-shot image classification, text-to-image retrieval, and text-to-image generation while preserving task performance. Code will be made available upon acceptance.