VL-Eraser：面向视觉语言模型机器遗忘的真空蒸馏
VL-Eraser: Vacuum Distillation for Machine Unlearning in Vision-Language Models
VL-Eraser 将遗忘目标解耦到低秩适配器中，通过删除适配器参数实现遗忘，为视觉语言模型提供了一种可复用的事后知识移除手段，尤其适用于需要移除敏感内容同时保持跨模态对齐的场景。
原文依据 VL-Eraser reformulates unlearning in VLMs as a two-stage process: distillation and deletion.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论机器遗忘（MU）旨在从预训练模型中移除敏感或不良内容。现有 MU 方法通常通过逐步降低模型在不良数据上的性能来实现近似遗忘。尽管这些方法取得了成功，但其在多模态遗忘任务中的有效性仍很大程度上未被探索。本文首先进行深入分析，揭示传统 MU 方法倾向于破坏跨模态对齐，导致多模态场景下的遗忘不彻底。为解决这一挑战，我们提出 VL-Eraser，一种面向视觉语言模型遗忘的新颖范式。VL-Eraser 将视觉语言模型中的遗忘重新表述为两个阶段的过程：蒸馏与删除。具体而言，VL-Eraser 首先引入真空蒸馏，将不良知识从视觉语言模型的复杂参数中解耦并转移到低秩适配器（LoRA）中。蒸馏后，通过从原始模型中删除 LoRA 参数即可高效实现遗忘。在多个基准上的大量实验表明，与最优基线相比，VL-Eraser 在保持模型效用的同时实现了更优的遗忘性能。
英文原文摘要 1200 字符
Machine unlearning (MU) aims to remove sensitive or undesired content from pre-trained models. Existing MU methods are commonly characterized as gradually degrading model performance on undesired data to realize approximate forgetting. Despite their successes, the effectiveness in multimodal unlearning tasks remains largely unexplored. In this paper, we first conduct an in-depth analysis and reveal that traditional MU methods tend to disrupt cross-modal alignment, leading to incomplete forgetting in multimodal scenarios. To tackle this challenge, we propose VL-Eraser, a novel unlearning paradigm for VLM unlearning. VL-Eraser reformulates unlearning in VLMs as a two-stage process: distillation and deletion. Specifically, VL-Eraser first introduces a vacuum distillation that disentangles undesired knowledge from the intricate parameters of VLMs and transfers it into low-rank adapters (LoRA). After distillation, unlearning is efficiently achieved by deleting the LoRA parameters from the original model. Extensive experiments across multiple benchmarks demonstrate that VL-Eraser achieves superior unlearning performance while preserving utility compared to the state-of-the-art baselines.