UARE:统一视觉语言模型用于图像质量评估、复原与增强
UARE: A Unified Vision-Language Model for Image Quality Assessment, Restoration, and Enhancement
该工作提出用统一视觉语言模型联合建模图像质量评估与复原,并通过交错文本-图像数据微调对齐 IQA 信号与复原目标,为低层视觉任务提供了一种可复用的多任务协同训练范式。
原文依据 we propose UARE, to our knowledge the first Unified vision-language model for image quality Assessment, Restoration, and Enhancement
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论图像质量评估(IQA)与图像复原是低层视觉中的基础问题。尽管 IQA 与复原在概念上紧密关联,现有工作大多将它们孤立处理。近期统一多模态理解-生成模型的进展展示了有前景的结果,并表明更强的理解能力可以提升生成性能。这促使我们构建一个统一 IQA 与复原的单一模型,并显式研究 IQA 如何引导复原——这一设定在很大程度上尚未被充分探索,却极具价值。本文提出 UARE,据我们所知,这是首个用于图像质量评估、复原与增强的统一视觉语言模型。基于预训练的统一理解与生成模型,我们引入两阶段训练框架。首先,采用渐进式由易到难的调度,从单一类型退化扩展到高阶混合退化,使 UARE 能够处理多种退化。其次,我们利用交错文本-图像数据对质量理解与复原进行统一微调,将 IQA 信号与复原目标对齐。通过多任务协同训练,UARE 利用 IQA 提升复原与增强性能。在 IQA、复原与增强任务上的大量实验证明了 UARE 的有效性。代码与模型将公开。
英文原文摘要 1408 字符
Image quality assessment (IQA) and image restoration are fundamental problems in low-level vision. Although IQA and restoration are closely connected conceptually, most existing work treats them in isolation. Recent advances in unified multimodal understanding-generation models demonstrate promising results and indicate that stronger understanding can improve generative performance. This motivates a single model that unifies IQA and restoration and explicitly studies how IQA can guide restoration, a setting that remains largely underexplored yet highly valuable. In this paper, we propose UARE, to our knowledge the first Unified vision-language model for image quality Assessment, Restoration, and Enhancement. Built on pretrained unified understanding and generation models, we introduce a two-stage training framework. First, a progressive, easy-to-hard schedule expands from single-type distortions to higher-order mixed degradations, enabling UARE to handle multiple degradations. Second, we perform unified fine-tuning of quality understanding and restoration with interleaved text-image data, aligning IQA signals with restoration objectives. Through multi-task co-training, UARE leverages IQA to boost restoration and enhancement performance. Extensive experiments across IQA, restoration, and enhancement tasks demonstrate the effectiveness of UARE. The code and models will be made available.