VLIC：以视觉语言模型作为感知裁判实现人类对齐的图像压缩
VLIC: Vision-Language Models As Perceptual Judges for Human-Aligned Image Compression
把 VLM 的零样本二选一判断直接当作奖励信号，用偏好后训练（如 DPO/奖励加权）微调扩散式图像压缩器，替代传统在心理物理数据上标定的可微感知损失网络；这套「VLM 当裁判」的奖励设计与训练细节可迁移到超分、去噪、增强等任何需要人眼对齐的复原任务。
原文依据 state-of-the-art vision-language models (VLMs) can replicate binary human two-alternative forced choice (2AFC) judgments zero-shot
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论包含人类偏好的图像压缩性能评估普遍发现，MSE 这类朴素的失真函数与人类感知的对齐程度不足。为使压缩模型与人类感知对齐，已有工作采用了由在大规模人类心理视觉判断数据集上校准的神经网络构成的可微感知损失。我们发现，令人惊讶的是，最优的视觉语言模型（VLM）在被要求推理成对图像之间的差异时，能够零样本复现二元人类二选一强制选择（2AFC）判断。为利用 VLM 强大的零样本视觉推理能力，我们提出 VLIC（用于图像压缩的视觉语言模型），这是一个基于扩散的图像压缩系统，设计为以二元 VLM 判断进行后训练。VLIC 采用现有的基于偏好的扩散模型后训练技术，而非将 VLM 判断蒸馏到单独的感知损失网络中。我们表明，根据感知指标和大规模用户研究，在 VLM 判断上校准该系统可在人类对齐的视觉压缩上取得有竞争力或最优的性能（视数据集而定）。此外，我们对基于 VLM 的奖励设计与训练流程进行了详尽分析，并分享了重要洞见。
英文原文摘要 1404 字符
Evaluations of image compression performance which include human preferences have generally found that naive distortion functions such as MSE are insufficiently aligned to human perception.In order to align compression models to human perception, prior work has employed differentiable perceptual losses consisting of neural networks calibrated on large-scale datasets of human psycho-visual judgments. We show that, surprisingly, state-of-the-art vision-language models (VLMs) can replicate binary human two-alternative forced choice (2AFC) judgments zero-shot when asked to reason about the differences between pairs of images. Motivated to exploit the powerful zero-shot visual reasoning capabilities of VLMs, we propose Vision Language Models for Image Compression (VLIC), a diffusion-based image compression system designed to be post-trained with binary VLM judgments. VLIC leverages existing techniques for diffusion model post-training with preferences, rather than distilling the VLM judgments into a separate perceptual loss network. We show that calibrating this system on VLM judgments produces competitive or state-of-the-art performance on human-aligned visual compression depending on the dataset, according to perceptual metrics and large-scale user studies. We additionally conduct an extensive analysis of the VLM-based reward design and training procedure and share important insights.