跨模态模糊对齐网络用于文本-航拍行人检索及大规模基准
Cross-modal Fuzzy Alignment Network for Text-Aerial Person Retrieval and A Large-scale Benchmark
针对无人机视角下视觉信息退化导致的跨模态对齐难题，提出用模糊隶属度函数动态量化词元级可靠性，并引入地面视角图像作为桥接代理，为跨模态检索中的噪声抑制和模态鸿沟弥合提供了可复用的对齐思路。
原文依据 we propose a novel Cross-modal Fuzzy Alignment Network (CFANet), which quantifies the token-level reliability by fuzzy logic to achieve accurate fine-grained alignment
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论文本-航拍行人检索旨在根据目击者描述在无人机拍摄的图像中识别目标，支撑智能交通与公共安全应用。与地面视角的文本-图像行人检索相比，无人机拍摄图像因视角和飞行高度的剧烈变化常导致视觉信息退化，使其与文本描述的语义对齐极具挑战。为解决该问题，我们提出一种新颖的跨模态模糊对齐网络（CFANet），通过模糊逻辑量化词元级可靠性以实现准确的细粒度对齐，并引入地面视角图像作为桥接代理以进一步弥合航拍图像与文本描述之间的鸿沟，用于文本-航拍行人检索。具体而言，我们设计了模糊词元对齐模块，采用模糊隶属度函数动态建模词元级关联强度并抑制不可观测或噪声词元的影响，可缓解因视觉线索缺失导致的语义不一致，显著增强词元级语义对齐的鲁棒性。此外，为进一步弥合航拍图像与文本描述之间的鸿沟，我们设计了上下文感知动态对齐模块，将地面视角代理作为桥接引入文本-航拍对齐，并自适应组合直接对齐与代理辅助对齐以提升鲁棒性。另外，我们利用思维链将文本生成分解为属性解析、初始描述和精炼，构建了名为AERI-PEDES的大规模基准数据集，从而提升文本准确性与语义一致性。在AERI-PEDES和TBAPR上的实验证明了我们方法的优越性。代码和数据集将公开释放。
英文原文摘要 1834 字符
Text-aerial person retrieval aims to identify targets in UAV-captured images from eyewitness descriptions, supporting intelligent transportation and public security applications. Compared to ground-view text-image person retrieval, UAV-captured images often suffer from degraded visual information due to drastic variations in viewing angles and flight altitudes, making semantic alignment with textual descriptions very challenging. To address this issue, we propose a novel Cross-modal Fuzzy Alignment Network (CFANet), which quantifies the token-level reliability by fuzzy logic to achieve accurate fine-grained alignment and incorporates ground-view images as a bridge agent to further mitigate the gap between aerial images and text descriptions, for text-aerial person retrieval. In particular, we design the Fuzzy Token Alignment module that employs the fuzzy membership function to dynamically model token-level association strength and suppress the influence of unobservable or noisy tokens. It can alleviate the semantic inconsistencies caused by missing visual cues and significantly enhance the robustness of token-level semantic alignment. Moreover, to further mitigate the gap between aerial images and text descriptions, we design a Context-Aware Dynamic Alignment module to incorporates the ground-view agent as a bridge in text-aerial alignment and adaptively combine direct alignment and agent-assisted alignment to improve the robustness. In addition, we construct a large-scale benchmark dataset called AERI-PEDES by using a chain-of-thought to decompose text generation into attribute parsing, initial captioning, and refinement, thus boosting textual accuracy and semantic consistency. Experiments on AERI-PEDES and TBAPR demonstrate the superiority of our method. The code and dataset will be publicly released.