SCIEval：用大型多模态模型评估与基准化科学图像生成与解读的忠实度
SCIEval: Evaluating and Benchmarking the Faithfulness of Scientific Image Generation and Interpretation with Large Multimodal Models
提供了一套面向科学图像忠实度的评测模型与基准（SCIEval-Bench），其基于 CLIP 的模态内/跨模态对比学习增强科学图像感知、以及用监督式理由信号微调 LMM 做可解释性归因的做法，可直接迁移到任意生成图像/描述的可信度打分与细粒度错误定位任务中。
原文依据 we present SCIEval-Bench, a human-annotated evaluation benchmark consisting of 3,000 samples for scientific text-to-image and 3,000 samples for scientific image captioning
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论科学图像通常需要精确的数值表示和正确的物体属性。然而，当前的忠实度指标主要面向照片级真实的日常图像，因而难以适用于科学图像评估。为填补这一空白，我们提出一种新的评估模型 SCIEval（Scientific Image Evaluation），旨在从三个关键维度刻画忠实度：(i) 相关性，衡量文本与图像的整体对应关系；(ii) 准确性，考察科学物体的技术细节；(iii) 可解释性，将生成内容中不忠实的元素单独分离出来。针对这些维度，我们构建了一个专门的科学图文对数据集，用于训练三个评估模块。对于相关性与准确性模块，我们提出一种基于 CLIP 的策略，通过模态内与跨模态对比学习增强科学图像感知。同时，可解释性模块通过使用监督式理由信号微调一个高性能大型多模态模型（LMM）来实现。最后，我们提出 SCIEval-Bench，一个由人工标注的评估基准，包含 3,000 个科学文生图样本和 3,000 个科学图像描述样本。在 SCIEval-Bench 上的大量实验表明，我们的 SCIEval 模型显著优于包括 GPT-4o 在内的 24 个对比模型，与人类判断具有更高的相关性。
英文原文摘要 1477 字符
Scientific images often require accurate numerical representations and correct object attributes. However, current faithfulness metrics are primarily tailored toward photorealistic, real-life imagery, rendering them ill-suited for scientific image evaluation. To address this gap, we introduce a novel evaluation model, SCIEval (Scientific Image Evaluation), which aims to capture faithfulness through three key dimensions: (i) Relevance, measuring overall text-image correspondence; (ii) Accuracy, examining the technical details of scientific objects; and (iii) Explainability, which isolates unfaithful elements within the generated content. To address these dimensions, we curate a specialized dataset of scientific text-image pairs to train three evaluation modules. For the Relevance and Accuracy modules, we propose a CLIP-based strategy that enhances scientific image perception through intra- and cross-modal contrastive learning. Concurrently, the Explainability module is developed by fine-tuning a high-performance Large Multimodal Model (LMM) using supervised rationale signals. Finally, we present SCIEval-Bench, a human-annotated evaluation benchmark consisting of 3,000 samples for scientific text-to-image and 3,000 samples for scientific image captioning. Extensive experiments on SCIEval-Bench demonstrate that our SCIEvalmodel is significantly more reliable than 24 competing models--including GPT-4o--exhibiting a superior correlation with human judgments.