精简的开放词表人-物交互检测
Streamlined Open-Vocabulary Human-Object Interaction Detection
对做开放词表检测的人，这篇给出了一个可复用的思路：不再拼接传统检测器与 VLM 两套表征，而是把交互查询和骨干图像 token 一起送进 DINOv3 的文本对齐视觉头，让交叉注意力在同一个表征空间内完成，从而绕开跨模型特征融合的鸿沟；同时全冻结骨干、只训练少量参数的做法可直接迁移到其他开放词表定位任务。
原文依据 we propose first feeding both the interaction queries and the backbone image tokens into the vision head
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论开放词表人-物交互（HOI）检测旨在定位并识别图像中的所有人物交互，包括训练阶段未见的类别。现有方法通常依赖传统 HOI 检测器与视觉语言模型（VLM）的协作来识别未见过的 HOI 类别。然而，由于跨模型表征之间存在显著鸿沟，该范式下的特征融合颇具挑战。为解决这一问题，我们提出 SL-HOI，一个仅基于强大的 DINOv3 模型的精简开放词表 HOI 检测框架。我们的设计利用了 DINOv3 各组成部分的互补优势：其骨干网络用于细粒度定位，其文本对齐的视觉检测头用于开放词表交互分类。此外，为促进交互查询与视觉检测头输出之间顺畅的交叉注意力，我们提出将交互查询与骨干图像 token 一并送入视觉检测头，从而有效弥合二者的表征鸿沟。我们方法中所有 DINOv3 参数均被冻结，仅添加少量可学习参数，从而能够快速适配 HOI 检测任务。大量实验表明，SL-HOI 在 SWiG-HOI 与 HICO-DET 两个基准上均取得最优性能，验证了该精简模型架构的有效性。代码见 https://github.com/MPI-Lab/SL-HOI。
英文原文摘要 1453 字符
Open-vocabulary human-object interaction (HOI) detection aims to localize and recognize all human-object interactions in an image, including those unseen during training. Existing approaches usually rely on the collaboration between a conventional HOI detector and a Vision-Language Model (VLM) to recognize unseen HOI categories. However, feature fusion in this paradigm is challenging due to significant gaps in cross-model representations. To address this issue, we introduce SL-HOI, a StreamLined open vocabulary HOI detection framework based solely on the powerful DINOv3 model. Our design leverages the complementary strengths of DINOv3's components: its backbone for fine-grained localization and its text-aligned vision head for open-vocabulary interaction classification. Moreover, to facilitate smooth cross-attention between the interaction queries and the vision head's output, we propose first feeding both the interaction queries and the backbone image tokens into the vision head, effectively bridging their representation gaps. All DINOv3 parameters in our approach are frozen, with only a small number of learnable parameters added, allowing a fast adaptation to the HOI detection task. Extensive experiments show that SL-HOI achieves state-of-the-art performance on both the SWiG-HOI and HICO-DET benchmarks, demonstrating the effectiveness of our streamlined model architecture. Code is available at https://github.com/MPI-Lab/SL-HOI.