基于混合专家的特征解耦用于开放词表场景图生成
Mixture-of-Experts based Feature Decoupling for Open Vocabulary Scene Graph Generation
MoE-FD 通过门控网络自适应解耦物体与关系特征，并用迭代交叉注意力建模语义交互，可直接用于开放词表场景图生成中提升新类别判别力与图文对齐精度。
原文依据 we propose the MoE Feature Decoupling (MoE-FD) framework for Open Vocabulary Scene Graph Generation
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近年来，场景图生成虽已取得显著进展，但主流方法仍受限于预定义的物体与关系类别，难以泛化到开放的真实场景。受开放词表目标检测启发，近期工作将场景图生成扩展到开放词表领域。然而，这些模型通常依赖现成的视觉语言模型，缺乏判别性属性提取，且物体-关系语义交互有限，导致新类别被误分类。为解决这些问题，我们提出面向开放词表场景图生成的混合专家特征解耦（MoE-FD）框架。MoE-FD 通过多个专家自适应地学习物体与关系的特征解耦，并利用门控网络权重优先关注关键特征。此外，它采用迭代交叉注意力建模物体与关系之间的语义交互，增强关系三元组关联与视觉-语义对齐。MoE-FD 的主要贡献有三点：（1）基于混合专家的特征解耦框架，自适应增强物体与关系的判别性特征表示；（2）物体与关系之间的语义交互建模，以强化关系三元组关联与图文对齐精度；（3）大量实验证明 MoE-FD 在 Visual Genome 数据集上的有效性。
英文原文摘要 1441 字符
In recent years, while Scene Graph Generation has advanced significantly, mainstream methods remain constrained by predefined object and relationship categories, limiting generalization to open real-world scenarios. Inspired by open vocabulary object detection, recent efforts have expanded SGG to the open vocabulary domain. However, these models often rely on off-the-shelf VLMs, lacking discriminative attribute extraction and suffering from limited object-relationship semantic interaction, which leads to misclassification of novel categories. To address these issues, we propose the MoE Feature Decoupling (MoE-FD) framework for Open Vocabulary Scene Graph Generation. MoE-FD adaptively learns feature decoupling for objects and relationships via multiple experts, prioritizing critical features through gating network weights. Moreover, it models semantic interactions between objects and relationships using iterative cross-attention, enhancing relationship triple associations and visual-semantic alignment. The main contributions of MoE-FD are threefold: (1) A MoE-based feature decoupling framework that adaptively enhances discriminative feature representation for objects and relations. (2) Semantic interaction modeling between objects and relations to strengthen relationship triple associations and image-text alignment accuracy. (3) Extensive experiments demonstrate the effectiveness of MoE-FD on the Visual Genome dataset.