导师-学生强化学习：面向鲁棒深度伪造检测的动态课程
Tutor-Student Reinforcement Learning: A Dynamic Curriculum for Robust Deepfake Detection
将深度伪造检测的训练过程建模为马尔可夫决策过程，用PPO智能体根据样本的视觉特征和历史学习动态（EMA损失、遗忘次数）动态分配损失权重，这种基于强化学习的动态课程策略可迁移到其他需要鲁棒泛化的视觉任务中。
原文依据 The Tutor, implemented as a Proximal Policy Optimization (PPO) agent, observes a rich state representation for each training sample, encapsulating not only its visual features but also its historical learning dynamics, such as EMA loss and forgetting counts.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论深度伪造检测的标准监督训练对所有样本赋予相同的重要性，这可能不利于学习鲁棒且泛化的特征。本文提出一种新颖的导师-学生强化学习（TSRL）框架，以动态优化训练课程。该方法将训练过程建模为马尔可夫决策过程，其中“导师”智能体学习指导“学生”（即深度伪造检测器）。导师由近端策略优化（PPO）智能体实现，它观察每个训练样本的丰富状态表示，该表示不仅包含视觉特征，还包含历史学习动态，如EMA损失和遗忘次数。基于该状态，导师通过为样本损失分配一个连续权重（0-1）来采取动作，从而动态地重新加权训练批次。导师根据学生的即时性能变化获得奖励，具体奖励从错误预测到正确预测的转变。该策略鼓励导师学习一种优先考虑高价值样本（如困难但可学习的样本）的课程，从而带来更高效且有效的训练过程。我们证明，与传统训练方法相比，这种自适应课程提高了学生针对未见操纵技术的泛化能力。代码见 https://github.com/wannac1/TSRL
英文原文摘要 1436 字符
Standard supervised training for deepfake detection treats all samples with uniform importance, which can be suboptimal for learning robust and generalizable features. In this work, we propose a novel Tutor-Student Reinforcement Learning (TSRL) framework to dynamically optimize the training curriculum. Our method models the training process as a Markov Decision Process where a "Tutor" agent learns to guide a "Student" (the deepfake detector). The Tutor, implemented as a Proximal Policy Optimization (PPO) agent, observes a rich state representation for each training sample, encapsulating not only its visual features but also its historical learning dynamics, such as EMA loss and forgetting counts. Based on this state, the Tutor takes an action by assigning a continuous weight (0-1) to the sample's loss, thereby dynamically re-weighting the training batch. The Tutor is rewarded based on the Student's immediate performance change, specifically rewarding transitions from incorrect to correct predictions. This strategy encourages the Tutor to learn a curriculum that prioritizes high-value samples, such as hard-but-learnable examples, leading to a more efficient and effective training process. We demonstrate that this adaptive curriculum improves the Student's generalization capabilities against unseen manipulation techniques compared to traditional training methods. Code is available at https://github.com/wannac1/TSRL