Score2Instruct：通过自动维度评分扩展以视频质量为中心的指令
Score2Instruct: Scaling Up Video Quality-Centric Instructions via Automated Dimension Scoring
提出基于自动维度评分的指令生成流水线（SIG），将无标注视频的多维质量分数映射为文本等级并构造分层思维链，为视频 LMM 提供可扩展的质量指令数据；其渐进式微调策略与 S2I-Bench 可直接用于评估和提升视频 LMM 的质量评分与解释能力。
原文依据 we propose the Score-based Instruction Generation (SIG) pipeline
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论经典视频质量评估（VQA）方法生成一个数值分数来判断视频的感知视觉保真度和清晰度。然而，分数无法描述视频复杂的质量维度（例如噪声），限制了其适用性。受益于人类友好的语言输出，通过指令微调使视频大语言多模态模型（LMM）适应 VQA 有可能解决这一问题。该方法的核心在于以视频质量为中心的指令数据。先前的探索主要集中于图像领域，且其数据生成过程严重依赖人工质量标注和专有系统（例如 GPT-4），限制了数据的可扩展性和有效性。为应对这些挑战，我们提出了基于分数的指令生成（SIG）流水线。具体而言，SIG 首先对无标注视频的多个质量维度进行评分，并将分数映射为文本定义的等级。然后，它显式地引入分层思维链（CoT）来建模特定维度与整体质量之间的关联，模仿人类视觉系统（HVS）的推理过程。该自动化流水线消除了对专家编写的质量描述和专有系统的依赖，确保了数据可扩展性和生成效率。由此得到的 Score2Instruct（S2I）数据集包含超过 32 万个多样化的指令-响应对，为指令微调奠定了基础。此外，为了同时提升视频 LMM 的质量评分和解释能力，我们设计了一种渐进式微调策略，以充分释放 S2I 的潜力。基于 SIG，我们进一步构建了一个名为 S2I-Bench 的基准，包含 400 个开放式问题，以更好地评估视频 LMM 的质量解释能力。在 S2I-Bench 和现有基准上的实验结果表明，我们的方法在多个视频 LMM 上持续提升了质量评分和解释能力。代码和数据集将在 https://github.com/KeiChiTse/S2I 提供。
英文原文摘要 2042 字符
Classical video quality assessment (VQA) methods generate a numerical score to judge a video's perceived visual fidelity and clarity. Yet, a score fails to describe the video's complex quality dimensions (e.g., noise), restricting its applicability. Benefiting from the human-friendly linguistic output, adapting video large multimodal models (LMMs) to VQA via instruction tuning has the potential to address this issue. The core of the approach lies in the video quality-centric instruction data. Previous explorations mainly focus on the image domain, and their data generation processes heavily rely on human quality annotations and proprietary systems (e.g., GPT-4), limiting data scalability and effectiveness. To address these challenges, we propose the Score-based Instruction Generation (SIG) pipeline. Specifically, SIG first scores multiple quality dimensions of an unlabeled video and maps scores to text-defined levels. It then explicitly incorporates a hierarchical Chain-of-Thought (CoT) to model the correlation between specific dimensions and overall quality, mimicking the human visual system's (HVS) reasoning process. The automated pipeline eliminates the reliance on expert-written quality descriptions and proprietary systems, ensuring data scalability and generation efficiency. To this end, the resulting Score2Instruct (S2I) dataset contains over 320K diverse instruction-response pairs, laying the basis for instruction tuning. Moreover, to advance video LMMs' quality scoring and justification abilities simultaneously, we devise a progressive tuning strategy to fully unleash the power of S2I. Built upon SIG, we further curate a benchmark termed S2I-Bench with 400 open-ended questions to better evaluate the quality justification capacity of video LMMs. Experimental results on the S2I-Bench and existing benchmarks indicate that our method consistently improves quality scoring and justification capabilities across multiple video LMMs. The code and dataset will be available at https://github.com/KeiChiTse/S2I.