掩码教师与强化学生：面向视觉语言模型的蒸馏方法
Masking Teacher and Reinforcing Student for Distilling Vision-Language Models
提出一种掩码渐进式强化学习蒸馏框架，通过逐步恢复教师模型容量和离线RL奖励机制，解决大模型到小模型蒸馏中的不稳定性问题，为视觉语言模型的高效压缩与部署提供可复用技术路径。
原文依据 we propose Masters (Masking teacher and reinforcing student), a mask-progressive reinforcement learning (RL) distillation framework.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论大规模视觉语言模型（VLM）近期在多模态理解方面取得了显著成就，但其庞大的规模使其难以部署在移动或边缘设备上。因此，需要紧凑且能力强的VLM，能够高效地从强大的大型教师模型中学习。然而，由于师生模型之间的巨大规模差距，从大型教师模型向小型学生模型蒸馏知识仍然具有挑战性：学生模型往往无法复现教师模型复杂的高维表示，导致学习不稳定和性能下降。为解决这一问题，我们提出了Masters（掩码教师与强化学生），一种掩码渐进式强化学习（RL）蒸馏框架。Masters首先对教师模型进行掩码并处理非主导权重，以降低不必要的复杂性，然后在训练过程中从掩码状态逐步恢复教师模型，以逐渐增加教师模型的容量。这一策略使学生模型能够以平滑且稳定的方式学习教师模型更丰富的表示。为进一步精炼知识迁移，Masters集成了一个离线RL阶段，包含两种互补的奖励：一种准确度奖励，衡量生成响应的正确性；一种蒸馏奖励，量化响应从教师模型到学生模型的可迁移性难易程度。与计算成本高昂且生成长响应的在线思考-回答RL范式不同，我们的离线RL利用来自掩码教师模型的预生成响应。这些响应提供了丰富而高效的指导，使学生模型无需思考-回答过程即可获得强大性能。在多样化的VLM基准上的大量实验表明，Masters优于现有的紧凑型VLM，并部分超越大型模型，同时效率远高于后者。此外，在蒸馏过程中逐渐增加教师模型的规模（例如从14B到38B）比单样本蒸馏（例如38B）产生更平滑的收敛和更强的泛化能力，揭示了一条通往高效且可部署VLM的可扩展路径。
英文原文摘要 2067 字符
Large-scale vision-language models (VLMs) have recently achieved remarkable multimodal understanding, but their massive size makes them impractical for deployment on mobile or edge devices. This raises the need for compact yet capable VLMs that can efficiently learn from powerful large teacher. However, distilling knowledge from large teacher to small student remains challenging due to their large size gap: the student often fails to reproduce the teacher's complex, high-dimensional representations, leading to unstable learning and degraded performance. To address this, we propose Masters (Masking teacher and reinforcing student), a mask-progressive reinforcement learning (RL) distillation framework. Masters first masks and non-dominant weights of the teacher to reduce unnecessary complexity, then progressively restores the teacher from mask to gradually increase the teacher capacity during training. This strategy allows the student to learn richer representations of teacher in a smooth and stable manner. To further refine knowledge transfer, Masters integrates an offline RL stage with two complementary rewards: an accuracy reward that measures the correctness of the generated responses, and a distillation reward that quantifies the ease of their responses' transferability from teacher to student. Unlike online think-answer RL paradigms that are computationally expensive and generate lengthy responses, our offline RL leverages pre-generated responses from masked teachers. These provide rich yet efficient guidance, enabling the students to achieve strong performance without requiring the think-answer process. Extensive experiments across diverse VLM benchmarks demonstrate that Masters outperforms existing compact VLMs and partially surpasses large ones, while being far more efficient. Moreover, gradually increasing the teacher sizes during distillation (e.g., from 14B to 38B) yields smoother convergence and stronger generalization than one-shot distillation (e.g., 38B), revealing a scalable path toward efficient and deployable VLMs.