面向可泛化AI生成图像检测的图像自适应提示词学习
Towards Generalizable AI-Generated Image Detection via Image-Adaptive Prompt Learning
把提示词从训练后固定改为按测试图像动态生成,并用单样本多视图预测一致性在推理时优化自适应 token,为视觉基础模型在分布外伪造图像上的泛化提供了一套可复用的测试时自适应提示词方案。
原文依据 dynamically adjusts the prompts fed into the encoder according to each testing image, rather than fixing them after training
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论在AI生成图像检测领域,当前最前沿的方法通常通过部分参数微调来适配预训练基础模型。然而,这类方法往往难以泛化到来自未见生成器的伪造图像,因为微调后的模型只捕捉到训练数据中的有限模式,无法反映新生成器不断演变的特征。为克服这一局限,我们提出图像自适应提示词学习(IAPL),这是一种新范式:它根据每张测试图像动态调整输入编码器的提示词,而非在训练后将其固定。该设计显著增强了对多样化伪造图像的鲁棒性与适应性。动态提示词通过一个轻量级可学习缩放因子,将条件信息与测试时自适应 token 相结合。条件信息由条件信息学习器产生,该学习器利用基于 CNN 的特征提取器对伪造特定条件与通用条件进行建模。测试时自适应 token 在推理阶段针对单个样本进行优化,通过强制多个视图间的预测一致性,使参数与当前图像对齐。在最终决策时,选择预测置信度最高的最优输入。大量实验表明,IAPL 达到了最优性能,在广泛使用的 UniversalFakeDetect 和 GenImage 数据集上分别取得 95.61% 和 96.7% 的平均准确率。
英文原文摘要 1477 字符
In AI-generated image detection, current cutting-edge methods typically adapt pre-trained foundation models through partial-parameter fine-tuning. However, these approaches often struggle to generalize to forgeries from unseen generators, as the fine-tuned models capture only limited patterns from training data and fail to reflect the evolving traits of new ones. To overcome this limitation, we propose Image-Adaptive Prompt Learning (IAPL), a novel paradigm that dynamically adjusts the prompts fed into the encoder according to each testing image, rather than fixing them after training. This design significantly enhances robustness and adaptability to diverse forged images. The dynamic prompts integrate conditional information with test-time adaptive tokens through a lightweight learnable scaling factor. The conditional information is produced by a Conditional Information Learner, which leverages CNN-based feature extractors to model both forgery-specific and general conditions. The test-time adaptive tokens are optimized during inference on a single sample by enforcing prediction consistency across multiple views, ensuring that the parameters align with the current image. For the final decision, the optimal input with the highest prediction confidence is selected. Extensive experiments show that IAPL achieves state-of-the-art performance, with mean accuracies of 95.61% and 96.7% on the widely used UniversalFakeDetect and GenImage datasets, respectively.