ClusterMark:面向自回归图像生成器、基于视觉词元聚类的鲁棒水印
ClusterMark: Towards Robust Watermarking for Autoregressive Image Generators with Visual Token Clustering
对做自回归图像生成的人,这篇给出了在词元序列上嵌入可检测水印的具体做法:按视觉词元聚类划分红/绿集合来偏置下一词元预测,并证明聚类比逐词元划分更能抵抗图像扰动与再生攻击,可直接复用到 VQ 词元序列的生成框架中。
原文依据 we propose a watermarking approach based on visual token clustering, which assigns similar tokens to the same set (red or green)
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论面向潜扩散模型的生成中水印近来在标记生成图像以便于检测与溯源方面展现出很高的鲁棒性。然而,其在自回归(AR)图像模型上的应用仍缺乏探索。自回归模型通过自回归地预测一串视觉词元来生成图像,这些词元随后由 VQ-VAE 解码器解码为像素。受大语言模型上 KGW 水印的启发,我们考察了基于先前词元对下一词元预测进行偏置的词元级水印方案。我们发现,直接迁移这些方案在原理上是可行的,但在常见的图像扰动下,水印的可检测性会显著下降。作为补救,我们提出一种基于视觉词元聚类的水印方法,将相似的词元归入同一集合(红色或绿色)。我们在免训练设定下考察了词元聚类,也考察了其与更准确的微调词元预测器或簇预测器相结合的情形。总体而言,实验表明基于聚类的水印在保持图像质量的同时,大幅提升了对扰动与再生攻击的鲁棒性,优于一系列基线方法与同期工作。此外,我们的方法验证运行时间很快,与轻量级后处理水印技术相当。
英文原文摘要 1338 字符
In-generation watermarking for latent diffusion models has recently shown high robustness in marking generated images for easier detection and attribution. However, its application to autoregressive (AR) image models is underexplored. Autoregressive models generate images by autoregressively predicting a sequence of visual tokens that are then decoded into pixels using a VQ-VAE decoder. Inspired by KGW watermarking for large language models, we examine token-level watermarking schemes that bias the next-token prediction based on prior tokens. We find that a direct transfer of these schemes works in principle, but the detectability of the watermarks decreases considerably under common image perturbations. As a remedy, we propose a watermarking approach based on visual token clustering, which assigns similar tokens to the same set (red or green). We investigate token clustering in a training-free setting, as well as in combination with a more accurate fine-tuned token or cluster predictor. Overall, our experiments show that cluster-based watermarks greatly improve robustness against perturbations and regeneration attacks while preserving image quality, outperforming a set of baselines and concurrent works. Moreover, our methods offer fast verification runtime, comparable to lightweight post-hoc watermarking techniques.