视觉语言模型 · CVPR2026 ·2026.06.03

VIRO:面向指代表达理解的带验证鲁棒高效神经符号推理

VIRO: Robust and Efficient Neuro-Symbolic Reasoning with Verification for Referring Expression Comprehension

Hyejin Park、Junhyuk Kwon、Suha Kwak、Jungseul Ok

与你有关的部分

在神经符号 REC 的推理链中嵌入算子级轻量验证器,使每个算子(物体存在性、空间关系)执行后自检并可弃权,为视觉推理系统提供抑制级联误检、处理无目标情形的可复用机制。

原文依据 we introduce Verification-Integrated Reasoning Operators (VIRO), a neuro-symbolic framework that embeds lightweight operator-level verifiers within reasoning steps
可迁移到 Referring Expression Comprehension视觉定位组合式视觉推理

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

指代表达理解(REC)旨在定位与自然语言查询相对应的图像区域。近期的神经符号 REC 方法利用大语言模型(LLM)和视觉语言模型(VLM)进行组合式推理,将查询分解为结构化程序并逐步执行。此类方法虽然实现了可解释推理与强大的零样本泛化能力,但假设中间推理步骤是准确的。然而,这一假设会导致级联错误:错误检测与无效关系沿推理链传播,即使图像中不存在目标,也会产生高置信度的误报。为解决这一局限,我们提出验证集成推理算子(VIRO),这是一个在推理步骤中嵌入轻量级算子级验证器的神经符号框架。每个算子执行并验证其输出,例如物体存在性或空间关系,使系统能够通过验证感知的弃权机制稳健地处理无目标情形。我们的框架达到最优性能,在有目标与无目标设置下取得 61.1% 的平衡准确率,并展示了对真实世界第一人称数据的泛化能力。VIRO 还表现出高可靠性,程序失败率至多为 0.3%,单查询运行时间高效,并通过解耦的程序生成与执行实现可扩展性。

英文原文摘要 1462 字符

Referring Expression Comprehension (REC) aims to localize the image region corresponding to a natural language query. Recent neuro-symbolic REC approaches leverage large language models (LLMs) and vision-language models (VLMs) to perform compositional reasoning, decomposing queries into structured programs and executing them step-by-step. While such approaches achieve interpretable reasoning and strong zero-shot generalization, they assume that intermediate reasoning steps are accurate. However, this assumption causes cascading errors: false detections and invalid relations propagate through the reasoning chain, yielding high-confidence false positives even when no target is present in the image. To address this limitation, we introduce Verification-Integrated Reasoning Operators (VIRO), a neuro-symbolic framework that embeds lightweight operator-level verifiers within reasoning steps. Each operator executes and validates its output, such as object existence or spatial relationships, allowing the system to robustly handle no-target cases through verification-aware abstention. Our framework achieves state-of-the-art performance, reaching 61.1% balanced accuracy across target-present and no-target settings, and demonstrates generalization to real-world egocentric data. VIRO also shows high reliability with a program failure rate of at most 0.3%, efficient per-query runtime, and scalability through decoupled program generation and execution.

标签

会议 CVPR2026
页码 33426-33435
Bibkey
来源 CVF Open Access