VITAL：以视觉编码器为中心的视觉质量评估多模态大模型预训练
VITAL: Vision-Encoder-centered Pre-training for LMMs in Visual Quality Assessment
以视觉编码器为中心的生成式预训练流程，配合机器执行的标注-审查范式构建大规模视觉-语言对，为视觉质量评估等低层感知任务提供可零样本迁移的多模态大模型底座；其解码器仅需极少预热数据即可复现全量训练性能，为视觉-语言模型的高效扩展提供可复用方案。
原文依据 we propose a vision-encoder-centered generative pre-training pipeline and develop the VITAL-Series LMMs
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论构建一个稳健的视觉质量评估（VQualA）多模态大模型（LMM）需要兼具通用性、强大性和可迁移性。然而，现有的VQualA多模态大模型通常只关注单一任务，并依赖全参数微调，这使其容易对特定模态或任务类型过拟合，从而限制了其泛化能力和可迁移性。为解决这一问题，我们提出了一种以视觉编码器为中心的生成式预训练流程，并开发了VITAL系列多模态大模型。（1）我们采用机器执行的标注-审查范式，构建了超过450万个视觉-语言（VL）对——这是迄今为止最大的VQualA训练数据集。（2）我们采用多任务训练流程，同时提升模型的定量评分精度，并增强其在图像和视频两种模态上的质量解释能力。（3）在视觉编码器的基础上，我们实现了高效的模型库扩展：该模型库展现出强大的零样本性能，且每个配对的解码器仅需使用不到预训练数据1/1000的快速预热，即可达到与完全训练对应模型相当的性能。总体而言，我们的工作为迈向VQualA的基础多模态大模型奠定了基石。项目地址：https://github.com/jzhws/VITAL-Series。
英文原文摘要 1344 字符
Developing a robust visual quality assessment (VQualA) large multi-modal model (LMM) requires achieving versatility, powerfulness, and transferability. However, existing VQualA LMMs typically focus on a single task and rely on full-parameter fine-tuning, which makes them prone to overfitting on specific modalities or task types, thereby limiting their generalization capacity and transferability. To address this, we propose a vision-encoder-centered generative pre-training pipeline and develop the VITAL-Series LMMs. (1) We adopt a machine-executed annotation-scrutiny paradigm, constructing over 4.5M vision-language (VL) pairs-the largest VQualA training dataset to date. (2) We employ a multi-task training workflow that simultaneously enhances the model's quantitative scoring precision and strengthens its capability for quality interpretation across both image and video modalities. (3) Building upon the vision encoder, we realize an efficient model zoo extension: the model zoo exhibits strong zero-shot performance, and each paired decoder requires only a swift warm-up using less than 1/1000 of the pre-training data to achieve performance comparable to the fully trained counterpart. Overall, our work lays a cornerstone for advancing toward the foundation LMM for VQualA. The project is at https://github.com/jzhws/VITAL-Series.