Robo-SGG：利用面向布局的归一化与恢复可提升鲁棒场景图生成
Robo-SGG: Exploiting Layout-Oriented Normalization and Restitution Can Improve Robust Scene Graph Generation
针对损坏图像下的场景图生成，提出利用对域偏移鲁棒的布局信息（实例归一化+布局导向恢复+布局嵌入编码器）来增强视觉特征鲁棒性，该即插即用模块可集成到任意基线SGG模型中，为处理域偏移提供了一种可复用的思路。
原文依据 we leverage layout information, which represents the global structure of an image and is robust to domain shift, to enhance the robustness of SGG methods under corruption.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论我们提出Robo-SGG，一个用于鲁棒场景图生成（SGG）的即插即用模块。与标准SGG不同，鲁棒SGG旨在对多种损坏图像进行推理，其核心挑战是干净图像与损坏图像之间的域偏移。现有SGG方法因视觉特征偏移（如损坏干扰或遮挡）而性能下降。为获得鲁棒的视觉特征，我们利用布局信息——它表示图像的全局结构且对域偏移具有鲁棒性——来增强SGG方法在损坏条件下的鲁棒性。具体而言，我们采用实例归一化（IN）来缓解域特定变化，并通过所提出的布局导向恢复来恢复鲁棒的结构特征（即对象之间的位置和语义关系）。此外，对于损坏图像，我们引入布局嵌入编码器（LEE），通过门控机制自适应地融合布局和视觉特征，增强对象和谓词的位置与语义表示的鲁棒性。值得注意的是，我们提出的Robo-SGG可以轻松集成到任何基线SGG模型中。大量实验表明，通过将我们提出的Robo-SGG集成到最优方法中，我们在VG-C基准上的PredCls、SGCls和SGDet任务中，mR@50分别实现了6.3%、11.1%和8.0%的相对提升，并在损坏场景图生成基准（VG-C和GQA-C）上达到了新的最优性能。我们的源代码可在https://github.com/MICLAB-BUPT/Robo-SGG获取。
英文原文摘要 1623 字符
We propose Robo-SGG, a plug-and-play module for robust scene graph generation (SGG). Unlike standard SGG, robust SGG aims to perform inference on a diverse range of corrupted images, with the core challenge being the domain shift between clean and corrupted images. Existing SGG methods suffer from degraded performance due to shifted visual features (e.g., corruption interference or occlusions). To obtain robust visual features, we leverage layout information, which represents the global structure of an image and is robust to domain shift, to enhance the robustness of SGG methods under corruption. Specifically, we employ Instance Normalization (IN) to alleviate the domain-specific variations and recover the robust structural features (i.e., the positional and semantic relationships among objects) by the proposed Layout-Oriented Restitution. Furthermore, for corrupted images, we introduce a Layout-Embedded Encoder (LEE) that adaptively fuses layout and visual features via a gating mechanism, enhancing the robustness of positional and semantic representations for objects and predicates. Note that our proposed Robo-SGG can be easily integrated into any baseline SGG model. Extensive experiments demonstrate that by integrating our proposed Robo-SGG into the state-of-the-art method, we achieve relative improvements of 6.3%, 11.1%, and 8.0% in mR@50 for PredCls, SGCls, and SGDet tasks on the VG-C benchmark, respectively, and achieve new state-of-the-art performance in the corrupted scene graph generation benchmarks (VG-C and GQA-C). Our source code is available at https://github.com/MICLAB-BUPT/Robo-SGG.