OddGridBench:揭示多模态大语言模型缺乏细粒度视觉差异敏感性
OddGridBench: Exposing the Lack of Fine-Grained Visual Discrepancy Sensitivity in Multimodal Large Language Models
OddGridBench 用可控网格图把「细粒度视觉差异检测」拆成颜色/大小/旋转/位置等单属性变量,可直接用来诊断任意 MLLM 的低层感知短板;OddGrid-GRPO 的距离感知奖励 + 课程学习配方也可迁移到其他需要空间定位精度的视觉定位训练中。
原文依据 a single element differs from all others by one or multiple visual attributes such as color, size, rotation, or position
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多模态大语言模型(MLLMs)在广泛的视觉-语言任务上已取得显著性能。然而,其在低层视觉感知方面的能力,尤其是在检测细粒度视觉差异方面的能力,仍缺乏充分探索与系统分析。在本工作中,我们提出 OddGridBench,一个用于评估 MLLMs 视觉差异敏感性的可控基准。OddGridBench 包含 1400 余张基于网格的图像,其中单个元素在颜色、大小、旋转或位置等一个或多个视觉属性上与其余所有元素不同。实验表明,所有被评估的 MLLMs,包括 Qwen3-VL、InternVL3.5 等开源系列以及 Gemini-2.5-Pro、GPT-5 等专有系统,在视觉差异检测上的表现均远低于人类水平。我们进一步提出 OddGrid-GRPO,一个融合课程学习与距离感知奖励的强化学习框架。通过逐步控制训练样本的难度,并将空间邻近约束纳入奖励设计,OddGrid-GRPO 显著增强了模型的细粒度视觉判别能力。我们希望 OddGridBench 与 OddGrid-GRPO 能为推进多模态智能中的感知视觉定位与视觉差异敏感性奠定基础。所有资源将在论文录用后公开发布。
英文原文摘要 1400 字符
Multimodal large language models (MLLMs) have achieved remarkable performance across a wide range of vision-language tasks. However, their ability in low-level visual perception, particularly in detecting fine-grained visual discrepancies, remains underexplored and lacks systematic analysis.In this work, we introduce OddGridBench, a controllable benchmark for evaluating the visual discrepancy sensitivity of MLLMs. OddGridBench comprises over 1,400 grid-based images, where a single element differs from all others by one or multiple visual attributes such as color, size, rotation, or position.Experiments reveal that all evaluated MLLMs, including open-source families such as Qwen3-VL and InternVL3.5, and proprietary systems like Gemini-2.5-Pro and GPT-5, perform far below human levels in visual discrepancy detection. We further propose OddGrid-GRPO, a reinforcement learning framework that integrates curriculum learning and distance-aware reward. By progressively controlling the difficulty of training samples and incorporating spatial proximity constraints into the reward design, OddGrid-GRPO significantly enhances the model's fine-grained visual discrimination ability.We hope OddGridBench and OddGrid-GRPO will lay the groundwork for advancing perceptual grounding and visual discrepancy sensitivity in multimodal intelligence.All resources will be publicly released upon acceptance.