TANGO：面向标签噪声下视觉语言模型鲁棒微调的文本锚定引导优化
TANGO: Text-Anchored Guided Optimization for Robust Fine-tuning Vision-Language Models under Label Noise
用文本模态构造与训练标签无关的语义锚点，替代线性分类器并指导样本筛选与标签纠错，为视觉研究者提供一种在脏标注数据上微调 VLM 的即插即用范式。
原文依据 using the text modality not just to identify noise, but to establish a source of ground truth
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论微调大规模视觉语言模型（VLM）对专用任务至关重要，但其性能常被真实数据集里普遍存在的标签噪声所削弱。传统的噪声标签学习方法通常依赖自指循环，即用模型自身的预测来纠正错误。尽管近期针对 VLM 的方法已开始利用跨模态信息辅助噪声检测，我们探索了另一条路径：不仅用文本模态识别噪声，更用它建立一个完全独立于训练数据中可能被污染标签的真值来源。为此，我们提出文本锚定引导优化（TANGO），一个以“语义锚点”为核心的框架——语义锚点是一组由多样化文本描述生成的纯净、不可变的参考点。基于这些锚点，我们的方法重构了噪声标签学习的两个关键方面。第一，我们用无参数的文本锚定分类器取代传统线性分类器，使预测成为干净锚点的直接加权共识。第二，我们引入锚点引导精炼机制，用这一外部真值校验每个样本的给定标签，为样本选择和标签纠错提供更鲁棒的信号。大量实验表明，该方法取得了有竞争力且往往是最优的性能。我们的代码将公开。
英文原文摘要 1470 字符
Fine-tuning large-scale Vision-Language Models (VLMs) is crucial for specialized tasks, but their performance is often undermined by the label noise prevalent in real-world datasets. Traditional approaches to learning with noisy labels typically rely on a self-referential loop, using a model's own predictions to correct errors. While recent VLM-specific methods have begun to leverage cross-modal information to aid in noise detection, we explore an alternative direction: using the text modality not just to identify noise, but to establish a source of ground truth that is fully independent of the training data's potentially corrupt labels. To this end, we propose Text-ANchored Guided Optimization (TANGO), a framework centered on "semantic anchors"--a set of pure, immutable reference points generated from diverse text descriptions. Building upon these anchors, our approach reframes two key aspects of learning with noisy labels. First, we replace the conventional linear classifier with a parameter-free Text-Anchored Classifier, making predictions a direct, weighted consensus of the clean anchors. Second, we introduce an Anchor-Guided Refinement mechanism that validates each sample's given label against this external ground truth, providing a more robust signal for sample selection and label correction. Extensive experiments demonstrate that this approach achieves competitive and often state-of-the-art performance. Our code will be publicly available.