视觉语言模型 · CVPR2026 ·2026.06.03

NeuroRule:通过可微规则归纳桥接视觉与逻辑

NeuroRule: Bridging Vision and Logic with Differentiable Rule Induction

Muhammad Zarar、Mingzheng Zhang、Xiaowang Zhang、Zhiyong Feng

与你有关的部分

将视觉特征映射为概率符号表示,并用可微规则学习引擎自动发现一阶逻辑规则,为关系预测提供透明解释;这种神经符号桥接思路可迁移到其他需要组合推理的视觉任务(如视觉问答、指代分割)。

原文依据 a neural-symbolic bridge that maps visual features to probabilistic symbolic representations
可迁移到 场景图生成视觉关系检测视觉问答

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

场景图生成(SGG)旨在通过检测物体及其成对关系来结构化地表示视觉场景。尽管已取得显著进展,当前模型由于纯神经、基于流水线的特性,编码的视觉知识存在视觉上下文模糊和逻辑推断的隐式关系问题。这一局限凸显了超越识别存在何种关系、进而解释关系为何存在以及如何通过逻辑规则链进行组合推理的必要性。为应对这些挑战,我们提出 NeuroRule,首个神经引导的规则归纳网络,将 Mask2Former 的像素级精确视觉理解与可微规则归纳引擎相结合。所提方法能够直接从视觉数据中自动学习组合逻辑规则,同时为关系预测提供透明的解释。NeuroRule 引入三项关键创新:(1)神经符号桥,将视觉特征映射为概率符号表示;(2)可微规则学习机制,无需人工设计即可自动发现可解释的一阶逻辑规则;(3)组合链式规则系统,通过端到端可训练流水线传播置信度分数,实现复杂推理。在包括 Visual Genome(VG)、Panoptic Scene Graph(PSG)和 OpenPSG 在内的基准数据集上的大量实验表明,NeuroRule 达到了最优性能。我们的方法显著提升了少样本关系抽取,同时在其基于规则的解释中保持了完全的可解释性。

英文原文摘要 1656 字符

Scene Graph Generation (SGG) aims to structurally represent visual scenes by detecting objects and their pairwise relationships. Despite significant progress, current models encode visual knowledge with ambiguous visual context and logically inferred implicit relations due to their purely neural, pipeline-based nature. This limitation underscores the need to advance beyond identifying what relations exist to explaining why they exist and how they can be compositionally reasoned about through logical rule chaining. To address these challenges, we introduce NeuroRule, the first Neurally-Guided Rule Induction Network that integrates Mask2Former pixel-precise visual understanding with a differentiable rule induction engine. Our proposed method enables automatic learning of compositional logical rules directly from visual data while providing transparent explanations for relational predictions. NeuroRule introduces three key innovations: (1) a neural-symbolic bridge that maps visual features to probabilistic symbolic representations; (2) a differentiable rule-learning mechanism that automatically discovers interpretable first-order logic rules without manual engineering; and (3) a compositional chain rule system that enables complex inference while propagating confidence scores through an end-to-end trainable pipeline. Extensive experiments on the benchmark datasets, including Visual Genome (VG), Panoptic Scene Graph (PSG), and OpenPSG, demonstrate that NeuroRule achieves state-of-the-art performance. Our method significantly improves few-shot relation extraction while maintaining full interpretability in its rule-based explanations.

标签

会议 CVPR2026
页码 11654-11663
Bibkey
来源 CVF Open Access