VisRes Bench：评估视觉语言模型的视觉推理能力
VisRes Bench: On Evaluating the Visual Reasoning Capabilities of VLMs
对做 VLM 或视觉推理的研究者，VisRes Bench 提供了按感知补全、单属性规则推理、多属性组合推理三级划分的受控评测协议，并系统施加模糊、纹理替换、遮挡、旋转等感知扰动，可用于诊断模型究竟是真正在做视觉推理还是依赖语言先验，也可作为训练或微调后的抽象视觉推理评测集。
原文依据 we find that state-of-the-art VLMs perform near random under subtle perceptual perturbations
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉语言模型（VLM）在视觉问答、图像描述等任务上取得了显著进展。然而，这些模型究竟是在进行视觉推理，还是依赖语言先验，仍不清楚。为此，我们提出 VisRes Bench，一个旨在研究无上下文语言监督的自然场景下视觉推理的基准。通过分析模型在三个复杂度层级上的行为，我们揭示了其在感知推理与关系视觉推理能力上的明显局限。VisRes 在其各层级上分离出不同的推理能力：第 1 层在模糊、纹理变化、遮挡和旋转等扰动下考察感知补全与全局图像匹配；第 2 层测试对单一属性（如颜色、数量、朝向）的规则推理；第 3 层则针对需要整合多种视觉属性的组合性推理。在超过 19,000 张受控任务图像上，我们发现最优 VLM 在细微感知扰动下表现接近随机，表明其模式识别之外的抽象能力有限。最后，我们讨论了 VisRes 如何为推进多模态研究中的抽象视觉推理提供一个统一框架（https://visres-bench.github.io）。
英文原文摘要 1317 字符
Vision-Language Models (VLMs) have achieved remarkable progress across tasks such as visual question answering and image captioning. Yet, the extent to which these models perform visual reasoning as opposed to relying on linguistic priors remains unclear. To address this, we introduce VisRes Bench, a benchmark designed to study visual reasoning in naturalistic settings without contextual language supervision. Analyzing model behavior across three levels of complexity, we uncover clear limitations in perceptual and relational visual reasoning capacities. VisRes isolates distinct reasoning abilities across its levels. Level 1 probes perceptual completion and global image matching under perturbations such as blur, texture changes, occlusion, and rotation; Level 2 tests rule-based inference over a single attribute (e.g., color, count, orientation); and Level 3 targets compositional reasoning that requires integrating multiple visual attributes. Across more than 19,000 controlled task images, we find that state-of-the-art VLMs perform near random under subtle perceptual perturbations, revealing limited abstraction beyond pattern recognition. We conclude by discussing how VisRes provides a unified framework for advancing abstract visual reasoning in multimodal research (https://visres-bench.github.io).