基于专家协作的伪装感知图像-文本检索
Camouflage-aware Image-Text Retrieval via Expert Collaboration
针对伪装场景中目标与背景高度相似导致跨模态对齐困难的问题，提出双分支视觉编码器与置信度条件图注意力机制，可复用于其他低对比度或复杂背景下的视觉-语言检索任务。
原文依据 we propose a camouflage-expert collaborative network (CECNet), which features a dual-branch visual encoder
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论伪装场景理解（CSU）因其广泛的实践意义而受到显著关注。然而，在该领域中，鲁棒的图像-文本跨模态对齐仍未得到充分探索，阻碍了对伪装场景及其相关应用的深入理解。为此，我们聚焦典型的图像-文本检索任务，提出了一项新任务，称为“伪装感知图像-文本检索”（CA-ITR）。我们首先构建了一个专用的伪装图像-文本检索数据集（CamoIT），包含约10.5K个样本，并带有多个粒度的文本标注。在CamoIT上进行的基准测试结果揭示了CA-ITR对现有前沿检索技术所构成的潜在挑战，这些挑战主要源于目标的伪装特性以及复杂的图像内容。作为解决方案，我们提出了一种伪装专家协作网络（CECNet），其特点是具有双分支视觉编码器：一个分支捕获整体图像表示，而另一个分支则结合一个专用模型来注入伪装目标的表示。我们引入了一种新颖的置信度条件图注意力（C²GA）机制，以利用分支间的互补性。对比实验表明，CECNet在CA-ITR整体准确率上提升了约29%，超越了七个代表性检索模型。数据集和代码将在https://github.com/jiangyao-scu/CA-ITR上提供。
英文原文摘要 1451 字符
Camouflaged scene understanding (CSU) has attracted significant attention due to its broad practical implications. However, in this field, robust image-text cross-modal alignment remains under-explored, hindering deeper understanding of camouflaged scenarios and their related applications. To this end, we focus on the typical image-text retrieval task, and formulate a new task dubbed "camouflage-aware image-text retrieval" (CA-ITR). We first construct a dedicated camouflage image-text retrieval dataset (CamoIT), comprising ~10.5K samples with multi-granularity textual annotations. Benchmark results conducted on CamoIT reveal the underlying challenges of CA-ITR for existing cutting-edge retrieval techniques, which are mainly caused by objects' camouflage properties as well as those complex image contents. As a solution, we propose a camouflage-expert collaborative network (CECNet), which features a dual-branch visual encoder: one branch captures holistic image representations, while the other incorporates a dedicated model to inject representations of camouflaged objects. A novel confidence-conditioned graph attention (C\textsuperscript 2 GA) mechanism is incorporated to exploit the complementarity across branches. Comparative experiments show that CECNet achieves ~29% overall CA-ITR accuracy boost, surpassing seven representative retrieval models. The dataset and code will be available at https://github.com/jiangyao-scu/CA-ITR.