不要展示像素，展示线索：通过感知程序解锁语言模型中的视觉工具推理
Don't Show Pixels, Show Cues: Unlocking Visual Tool Reasoning in Language Models via Perception Programs
提出将视觉工具输出（如深度、光流、对应关系）重写为紧凑、结构化的语言原生摘要，使多模态大模型能直接解析和推理，从而在BLINK等感知任务上大幅提升性能，为视觉-语言模型中的工具使用提供了免训练、模型无关的新范式。
原文依据 We introduce Perception Programs (P^2), a training-free, model-agnostic method that rewrites tool outputs into compact, structured, language-native summaries that MLLMs can directly parse and reason over.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多模态语言模型（MLLMs）越来越多地与视觉工具（如深度、光流、对应关系）配对以增强视觉推理。然而，尽管能够访问这些工具生成的视觉线索，MLLMs往往无法充分利用它们。现有方法通常将原始工具输出馈入模型，但这些密集的像素级表示与LLMs的语言原生推理优势不匹配，导致因依赖语言先验而感知能力薄弱。我们认为，在视觉工具能够提供必要视觉线索的问题中，瓶颈不在于更多的工具调用或更大的MLLMs，而在于工具输出的表示方式。我们引入了感知程序（P^2），一种免训练、模型无关的方法，将工具输出重写为紧凑、结构化、语言原生的摘要，使MLLMs能够直接解析和推理。在BLINK的六个以感知为中心的任务中，P^2相比基础模型和原始工具增强基线持续带来大幅提升。以GPT-5 Mini作为基础模型，P^2将其在多视图推理上的准确率从41.35%提升至86.47%，在相对深度上从52.42提升至81.45，并在各任务上实现19.66的整体平均增益，创下新的最优结果。即使在较小的MLLMs上，例如InternVL3.5-4B和Qwen3VL-4B，与基于图像的工具变体相比，P^2在各任务上带来21-25的绝对增益，超越了先前的智能体、监督和基于强化学习的工具使用方法，且无需任何训练或模型修改。
英文原文摘要 1529 字符
Multimodal language models (MLLMs) are increasingly paired with vision tools (e.g., depth, flow, correspondence) to enhance visual reasoning. However, despite access to these tool-generated visual cues, MLLMs often fail to benefit fully from them. Existing approaches typically feed raw tool outputs into the model, but these dense, pixel-level representations are misaligned with the language-native reasoning strengths of LLMs, leading to weak perception due to reliance on language priors. We argue that, in problems where vision tools can provide the necessary visual cues, the bottleneck is not more tool calls or larger MLLMs, it is how tool outputs are represented. We introduce Perception Programs (P^2), a training-free, model-agnostic method that rewrites tool outputs into compact, structured, language-native summaries that MLLMs can directly parse and reason over. Across six perception-centric tasks in BLINK, P^2 consistently yields large improvements over base models and raw tool-augmented baselines. With GPT-5 Mini as the base model, P^2 raises its accuracy from 41.35% to 86.47% on multi-view reasoning, from 52.42 to 81.45 on relative depth, and achieves a 19.66 overall average gain across tasks, setting new state-of-the-art results. Even on smaller MLLMs, e.g., InternVL3.5-4B and Qwen3VL-4B, we observe 21-25 absolute gains from P^2 across tasks when comparing to image-based tool variants, surpassing prior agentic, supervised, and RL-based tool-use methods, without any training or model modifications.