CrossVL:面向跨视角视觉语言检测的复杂度感知特征路由与配对课程学习
CrossVL: Complexity-Aware Feature Routing and Paired Curriculum for Cross-View Vision-Language Detection
CPA 用多模态统计量估计场景复杂度并据此把视觉特征路由到不同通路,这一「按复杂度动态选择融合路径」的思路可直接迁移到任意 VLM 检测器的视觉-语言融合层;PCL 用同步地空图像对的语义一致性做早期稳定监督、再逐步转向随机采样,为跨域/跨视角微调提供了一种可复用的课程调度策略。
原文依据 CPA estimates scene complexity from multimodal statistics and routes visual features through multiple pathways to obtain view-specific representations.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉语言模型(VLM)能够实现文本引导的目标检测,但在地面与空中视角于高度、尺度和空间布局上存在差异的跨视角场景下性能严重退化。这些几何变化导致视角之间出现系统性的复杂度差异,例如地面视角图像包含密集且高度遮挡的结构,而空中图像则稀疏且呈全局组织。固定的 VLM 融合机制无法应对这种差异。我们提出 CrossVL,一个结合复杂度感知通路聚合(CPA)与配对课程学习(PCL)的框架,用于增强 VLM 的跨视角检测。CPA 从多模态统计量中估计场景复杂度,并将视觉特征路由经过多条通路以获得视角特定的表征。PCL 利用同步的地面-空中图像对的语义一致性来提供稳定的早期监督,随后逐步转向随机采样。在 MAVREC 上,CrossVL 将 Florence-2 的空中平均精度均值从 58.66% 提升至 61.03%,并将地面-空中性能差距从 8.63 个百分点缩小至 6.65 个百分点,同时在随机种子间的方差上实现了 3.3 倍的降低。CPA 提供了稳定的复杂度感知特征聚合,PCL 改善了优化动态。二者共同表明,协同的架构与训练适配对于鲁棒的跨视角 VLM 检测至关重要。
英文原文摘要 1390 字符
Vision-language models (VLMs) enable text-guided object detection but degrade severely under cross-view scenarios where ground and aerial viewpoints differ in altitude, scale, and spatial layout. These geometric changes introduce systematic complexity variations between viewpoints, e.g., ground view images contain dense and highly occluded structures, while aerial images are sparse and globally organized. Fixed VLM fusion mechanisms cannot handle this discrepancy. We propose CrossVL, a framework combining Complexity-Aware Pathway Aggregation (CPA) and Paired Curriculum Learning (PCL) for enhanced cross-view detection for VLM. CPA estimates scene complexity from multimodal statistics and routes visual features through multiple pathways to obtain view-specific representations. PCL leverages semantic consistency of synchronized ground-aerial pairs to provide stable early supervision and then gradually shifts toward randomized sampling. On MAVREC, CrossVL improves Florence-2's aerial mAP from 58.66% to 61.03% and reduces the ground-aerial performance gap from 8.63pp to 6.65pp, while also achieving a 3.3x reduction in variance across random seeds. CPA provides stable complexity-aware feature aggregation, and PCL enhances optimization dynamics. Together, they demonstrate that coordinated architectural and training adaptations are crucial for robust cross-view VLM detection.