视觉语言模型 · CVPR2026 ·2026.06.03

何时思考、何时观察:不确定性引导的回看

When to Think and When to Look: Uncertainty-Guided Lookback

Jing Bi、Filippos Bellos、Junjia Guo、Yayuan Li、Chao Huang、Yunlong Tang 等 11 人

与你有关的部分

对做 LVLM 视觉推理的人,这篇给出了一个可直接套用的无训练解码策略:用不确定性信号触发显式回看图像的短提示,并配合宽度搜索,在固定模型和词元预算下提升 MMMU 等基准,尤其补强标准 thinking 较弱的类别。

原文依据 we propose uncertainty-guided lookback, a training-free decoding strategy that combines an uncertainty signal with adaptive lookback prompts and breadth search
可迁移到 多模态推理视觉问答视觉定位

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

测试时的“思考”(即生成显式的中间推理链)已被证明能提升大语言模型的性能,近期也在大型视觉语言模型(LVLM)上展现出显著收益。然而,尽管结果令人鼓舞,目前仍缺乏关于思考究竟如何影响视觉推理的系统性分析。我们提供了首个此类分析,对 LVLM 的思考进行了大规模、受控的对比,在充裕的词元预算与多轮解码条件下,于 MMMUval 上评测了 InternVL3.5 与 Qwen3-VL 系列的 10 个变体。我们表明,思考更多并不总是更好:长推理链常常产生冗长且错误的轨迹,这些轨迹忽略图像,表现还不如同一模型在标准指令模式下的运行结果。更深入的分析揭示,某些明确回指图像的简短“回看”短语在成功轨迹中显著富集,并与更好的视觉定位相关。基于这一洞见,我们提出不确定性引导的回看,这是一种无需训练的解码策略,将不确定性信号与自适应回看提示以及宽度搜索相结合。我们的方法提升了 MMMU 的整体性能,在标准思考表现较弱的类别上带来最大增益,并优于若干强解码基线,在固定模型系列与词元预算下刷新了当前最优结果。我们进一步表明该解码策略具有泛化性,在另外五个基准上取得一致提升,其中包括两个广泛的多模态评测套件以及面向数学的视觉推理数据集。

英文原文摘要 1594 字符

Test-time "thinking" (i.e., generating explicit intermediate reasoning chains) is known to boost performance in large language models and has recently shown strong gains for large vision-language models (LVLMs). However, despite these promising results, there is still no systematic analysis of how thinking actually affects visual reasoning. We provide the first such analysis with a large-scale, controlled comparison of thinking for LVLMs, evaluating 10 variants from the InternVL3.5 and Qwen3-VL families on MMMUval under generous token budgets and multi-pass decoding. We show that more thinking is not always better: long chains often yield long-wrong trajectories that ignore the image and underperform the same models run in standard instruct mode. A deeper analysis reveals that certain short "lookback" phrases, which explicitly refer back to the image, are strongly enriched in successful trajectories and correlate with better visual grounding. Building on this insight, we propose uncertainty-guided lookback, a training-free decoding strategy that combines an uncertainty signal with adaptive lookback prompts and breadth search. Our method improves overall MMMU performance, delivers the largest gains in categories where standard thinking is weak, and outperforms several strong decoding baselines, setting a new state of the art under fixed model families and token budgets. We further show that this decoding strategy generalizes, yielding consistent improvements on five additional benchmarks, including two broad multimodal suites and math-focused visual reasoning datasets.

标签

会议 CVPR2026
页码 5104-5113
Bibkey
来源 CVF Open Access