连接像素与词：面向多模态媒体验证的掩码感知局部语义融合
Bridging Pixels and Words: Mask-Aware Local Semantic Fusion for Multimodal Media Verification
把掩码-标签对作为视觉与文本之间的细粒度对齐单元，并用双向查询机制检测局部语义冲突，这一思路可直接迁移到需要像素级视觉定位的跨模态任务（如指代分割、开放词表定位、图文检索中的局部匹配）。
原文依据 explicitly bridges words and pixels via mask-label pairs, enabling the model to perform precise reasoning over fine-grained cross-modal correspondences
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论随着虚假新闻造成的危害日益增长，检测并定位多模态媒体篡改（DGM4）任务受到更多关注。现有多模态方法忽视了视觉与文本模态之间的细粒度语义对齐，从而限制了其检测复杂而隐蔽的跨模态篡改的能力。为应对这一挑战，本文提出 MaLSF，一种新颖的掩码感知局部语义融合框架，通过掩码-标签对显式地连接词与像素，使模型能够对细粒度跨模态对应关系进行精确推理。MaLSF 通过两项关键创新捕获跨模态局部语义：1）双向跨模态验证模块（BCV），通过双向查询机制识别掩码区域与关联标签之间的语义冲突；2）层次语义聚合模块（HSA），将多粒度局部语义自适应地聚合为解耦特征，以用于任务特定的验证。此外，为提取细粒度掩码-标签对，本文引入一组多样化的掩码-标签对提取解析器。所提模型在多个数据集上进行了评估，在 DGM4 与多模态虚假新闻检测两项任务上均取得最优性能。大量消融实验与可视化结果进一步验证了其有效性与可解释性。
英文原文摘要 1392 字符
As the harm caused by fake news grows, the task of detecting and grounding multi-modal media manipulation (DGM4) is gaining more attention. Existing multimodal methods overlook fine-grained semantic alignment between visual and textual modalities, thereby limiting their ability to detect sophisticated and subtle cross-modal manipulations. To address this challenge, we present MaLSF, a novel Mask-aware Local Semantic Fusion framework that explicitly bridges words and pixels via mask-label pairs, enabling the model to perform precise reasoning over fine-grained cross-modal correspondences. MaLSF captures cross-modal local semantics through two key innovations: 1) A Bidirectional Cross-modal Verification Module (BCV) that identifies semantic conflicts between masked regions and associated labels via a bidirectional query mechanism; 2) A Hierarchical Semantic Aggregation (HSA) Module that adaptively aggregates multi-granularity local semantics into decoupled features for task-specific verification. In addition, to extract fine-grained mask-label pairs, we introduce a set of diverse mask-label pair extraction parsers. The proposed model is evaluated on multiple datasets and achieves state-of-the-art performance on both the DGM4 and multimodal fake news detection tasks. Extensive ablation studies and visualization results further verify its effectiveness and interpretability.