视觉语言模型 · CVPR2026 ·2026.06.03

感知证明:具备组合共形保证的可认证工具使用多模态推理

Proof-of-Perception: Certified Tool-Using Multimodal Reasoning with Compositional Conformal Guarantees

Arya Fayyazi、Haleh Akrami

与你有关的部分

把每个感知/逻辑节点包成共形预测集,为多模态推理链提供逐步校准的不确定性,可用来决定何时追加工具调用、何时提前停止,从而抑制视觉问答中的误差累积与幻觉。

原文依据 Each perception or logic node outputs a conformal set \Gamma^ (t) _\delta(x), yielding calibrated, stepwise uncertainty
可迁移到 多模态问答图表理解文档理解多图像推理

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

我们提出感知证明(Proof-of-Perception,PoP),一个将多模态推理转化为带显式可靠性保证的可执行图的工具使用框架。每个感知或逻辑节点输出一个共形集 \Gamma^ (t) _\delta(x),从而给出经过校准的逐步不确定性;一个轻量控制器利用这些证书在预算约束下分配计算——仅在需要时通过额外的工具调用进行扩展,否则提前停止。这使得答案建立在可验证的证据之上,减少误差累积与幻觉,并实现有原则的精度—计算权衡。在文档、图表和多图像问答基准上,PoP 在更高效地使用计算的同时,相较强链式思维、ReAct 风格和思维程序基线提升了性能与可靠性。代码见 https://github.com/AryaFayyazi/PoP 。

英文原文摘要 894 字符

We present Proof-of-Perception (PoP), a tool-using framework that casts multimodal reasoning as an executable graph with explicit reliability guarantees. Each perception or logic node outputs a conformal set \Gamma^ (t) _\delta(x), yielding calibrated, stepwise uncertainty; a lightweight controller uses these certificates to allocate compute under a budget--expanding with extra tool calls only when needed and stopping early otherwise. This grounds answers in verifiable evidence, reduces error compounding and hallucinations, and enables principled accuracy-compute trade-offs. Across document, chart, and multi-image QA benchmarks, PoP improves performance and reliability over strong chain-of-thought, ReAct-style, and program-of-thought baselines while using computation more efficiently. Code is available at \href https://github.com/AryaFayyazi/PoP https://github.com/AryaFayyazi/PoP .

标签

会议 CVPR2026
页码 5144-5153
Bibkey
来源 CVF Open Access