VLM 是感知还是回忆？用经典视觉错觉探测视觉感知与记忆
Do VLMs Perceive or Recall? Probing Visual Perception vs. Memory with Classic Visual Illusions
提出 VI-Probe 框架，通过分级扰动和匹配视觉控制（无错觉诱导）来分离视觉感知与语言记忆，并引入极性翻转一致性、模板固定指数和错觉乘数等指标，为评估 VLM 的视觉感知敏感性提供了可复用的探测方法。
原文依据 introduces VI-Probe, a controllable visual-illusion framework with graded perturbations and matched visual controls (without illusion inducer) that disentangles visually grounded perception from language-driven recall
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论大型视觉语言模型（VLM）在原始图像上常能“正确”回答经典视觉错觉，然而当错觉因素反转时，即使视觉变化对人类显而易见，它们仍坚持相同回答。这引出一个根本问题：VLM 是感知到了视觉变化，还是仅仅回忆了记忆中的模式？尽管已有若干研究注意到这一现象，但其根本原因仍不清楚。为从观察走向系统理解，本文提出 VI-Probe，一个可控的视觉错觉框架，具有分级扰动和匹配的视觉控制（无错觉诱导因素），用于分离基于视觉的感知与语言驱动的回忆。与先前关注平均准确率的工作不同，我们使用极性翻转一致性、模板固定指数以及针对匹配控制归一化的错觉乘数来测量稳定性和敏感性。跨不同模型家族的实验表明，响应持续性源于异质原因而非单一机制。例如，GPT-5 表现出记忆覆盖，Claude-Opus-4.1 表现出感知-记忆竞争，而 Qwen 变体则提示视觉处理局限。我们的发现挑战了单一原因观点，并推动基于探测的评估，以同时测量知识和对照视觉变化的敏感性。数据和代码见 https://sites.google.com/view/vi-probe/。
英文原文摘要 1434 字符
Large Vision-Language Models (VLMs) often answer classic visual illusions "correctly" on original images, yet persist with the same responses when illusion factors are inverted, even though the visual change is obvious to humans. This raises a fundamental question: do VLMs perceive visual changes or merely recall memorized patterns? While several studies have noted this phenomenon, the underlying causes remain unclear. To move from observations to systematic understanding, this paper introduces VI-Probe, a controllable visual-illusion framework with graded perturbations and matched visual controls (without illusion inducer) that disentangles visually grounded perception from language-driven recall. Unlike prior work that focuses on averaged accuracy, we measure stability and sensitivity using Polarity-Flip Consistency, Template Fixation Index, and an illusion multiplier normalized against matched controls. Experiments across different families reveal that response persistence arises from heterogeneous causes rather than a single mechanism. For instance, GPT-5 exhibits memory override, Claude-Opus-4.1 shows perception-memory competition, while Qwen variants suggest visual-processing limits. Our findings challenge single-cause views and motivate probing-based evaluation that measures both knowledge and sensitivity to controlled visual change. Data and code are available at https://sites.google.com/view/vi-probe/.