IF-Bench：面向红外图像的多模态大模型基准评测与生成式视觉提示增强
IF-Bench: Benchmarking and Enhancing MLLMs for Infrared Images with Generative Visual Prompting
提出免训练的生成式视觉提示方法，利用图像编辑模型将红外图像转换为语义和空间对齐的RGB图像，以缓解域分布偏移，可复用于其他跨域视觉-语言任务。
原文依据 we propose a training-free generative visual prompting (GenViP) method, which leverages advanced image editing models to translate infrared images into semantically and spatially aligned RGB counterparts
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近期多模态大语言模型（MLLMs）的进展已在各类基准上取得令人瞩目的成果。然而，其在理解红外图像方面的能力仍未被探索。为填补这一空白，我们推出IF-Bench，这是首个专为评估红外图像多模态理解而设计的高质量基准。IF-Bench包含来自23个红外数据集的499张图像和680个精心整理的视觉问答对，覆盖图像理解的10个基本维度。基于该基准，我们系统评估了40多个开源和闭源MLLMs，采用循环评估、双语评估和混合判断策略以提高结果的可靠性。我们的分析揭示了模型规模、架构和推理范式如何影响红外图像理解，为该领域提供了宝贵见解。此外，我们提出了一种免训练的生成式视觉提示（GenViP）方法，该方法利用先进的图像编辑模型将红外图像转换为语义和空间对齐的RGB对应图像，从而缓解域分布偏移。大量实验表明，我们的方法在广泛的MLLMs上持续带来显著的性能提升。
英文原文摘要 1310 字符
Recent advances in multimodal large language models (MLLMs) have led to impressive progress across various benchmarks. However, their capability in understanding infrared images remains unexplored. To address this gap, we introduce **IF-Bench**, the first high-quality benchmark designed for evaluating multimodal understanding of infrared images. IF-Bench consists of 499 images sourced from 23 infrared datasets and 680 carefully curated visual question-answer pairs, covering 10 essential dimensions of image understanding. Based on this benchmark, we systematically evaluate over 40 open-source and closed-source MLLMs, employing cyclic evaluation, bilingual assessment, and hybrid judgment strategies to enhance the reliability of the results. Our analysis reveals how model scale, architecture, and inference paradigms affect infrared image comprehension, providing valuable insights for this area. Furthermore, we propose a training-free generative visual prompting (**GenViP**) method, which leverages advanced image editing models to translate infrared images into semantically and spatially aligned RGB counterparts, thereby mitigating domain distribution shifts. Extensive experiments demonstrate that our method consistently yields significant performance improvements across a wide range of MLLMs.