视觉语言模型 · CVPR2026 ·2026.06.03

第二视角图像能否充当一种语言?面向 X 射线违禁品检测的几何与语义跨模态推理

Can a Second-View Image Be a Language? Geometric and Semantic Cross-Modal Reasoning for X-ray Prohibited Item Detection

Chuang Peng、Renshuai Tao、Zhongwei Ren、Xianglong Liu、Yunchao Wei

与你有关的部分

把第二视角图像当作「类语言模态」与文本语义联合对齐,并用结构化思维链(top→side→conclusion)组织跨视角几何与跨模态语义的推理,这一设定可迁移到任何多视角+文本的检测或问答任务中。

原文依据 treating the second-view images as a "language-like modality"
可迁移到 X-ray 违禁品检测跨视角推理图像描述生成视觉问答

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

自动 X 射线违禁品检测对安检至关重要,已被广泛研究。传统方法依赖视觉模态,面对复杂威胁时往往力不从心。尽管近期研究引入语言来引导单视角图像,但实际检查中人工安检员通常使用双视角图像。这引出一个问题:第二视角能否提供类似语言模态的约束?在本工作中,我们提出 DualXrayBench,这是首个同时包含多视角与多模态的 X 射线检查综合基准。它支持八项用于检验跨视角推理的任务。在 DualXrayBench 中,我们构建了一个双视角描述语料库,包含 12 个类别共 45,613 张双视角图像及对应描述文本。基于这些数据,我们提出几何(跨视角)—语义(跨模态)推理器(GSR),这是一个多模态模型,联合学习跨视角几何与跨模态语义之间的对应关系,将第二视角图像视为一种「类语言模态」。为实现这一点,我们构建了 GSXray 数据集,其中包含结构化的思维链序列:俯视图、侧视图、结论。在 DualXrayBench 上的全面评测表明,GSR 在所有 X 射线任务上均取得显著提升,为真实场景的 X 射线检查提供了新视角。

英文原文摘要 1337 字符

Automatic X-ray prohibited items detection is vital for security inspection and has been widely studied. Traditional methods rely on visual modal, often struggling with complex threats. While recent studies incorporate language to guide single-view images, human inspectors typically use dual-view images in practice. This raises the question: can the second view provide constraints similar to a language modality? In this work, we introduce DualXrayBench, the first comprehensive benchmark for X-ray inspection that includes multiple views and modalities. It supports eight tasks designed to test cross-view reasoning. In DualXrayBench, we introduce a dual-view caption corpus consisting of 45,613 dual-view images across 12 categories with corresponding captions. Building upon these data, we propose the Geometric (cross-view)-Semantic (cross-modality) Reasoner (GSR), a multimodal model that jointly learns correspondences between cross-view geometry and cross-modal semantics, treating the second-view images as a "language-like modality". To enale this, we construct the GSXray dataset, with structured Chain-of-Thought sequences: top , side, conclusion. Comprehensive evaluations on DualXrayBench demonstrate that GSR achieves significant improvements across all X-ray tasks, offering a new sight for real-world X-ray inspection.

标签

会议 CVPR2026
页码 26176-26186
Bibkey
来源 CVF Open Access