视觉语言模型 · CVPR2026 ·2026.06.03

AVION:从离线教师到提示微调网络的遥感视觉-语言指令

AVION: Aerial Vision-Language Instruction from Offline Teacher to Prompt-Tuned Network

Yu Hu、Jianyang Gu、Hao Liu、Yue Cao、Jozsef Hamari、Zheng Liu 等 7 人

与你有关的部分

用大语言模型生成并经验证筛选的文本原型作为教师信号,蒸馏到视觉-语言双塔的轻量可学习 prompt 中,推理时学生独立运行;这套「离线教师+prompt 微调」范式可直接迁移到其他专业成像域(医学、遥感、工业)的 CLIP 类模型适配。

原文依据 The student module integrates lightweight and learnable prompts into both vision and language encoders, guided by the teacher to align embeddings
可迁移到 零样本/少样本图像分类跨模态检索视觉-语言模型领域适配

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

将视觉-语言模型适配到遥感影像仍具挑战,原因主要有两点:文本表示的语义覆盖有限,以及视觉特征的自适应能力不足。这些问题在航空场景中尤为突出,因为此类场景涉及多样的视觉外观和细粒度的目标区分。我们提出 AVION,一个专为视觉-语言模型遥感适配而设计的知识蒸馏框架。教师模块通过从大语言模型收集描述,并利用遥感图像特征验证其有效性,从而构建语义丰富的文本原型。学生模块在视觉与语言编码器中同时引入轻量、可学习的提示,并在教师的引导下对齐嵌入及其跨模态关系。训练完成后,学生在推理阶段独立运行。在六个光学遥感基准上的实验表明,AVION 提升了少样本分类与基类准确率,且不损害对新类别的泛化能力;同时提升了跨模态检索的平均召回率,而额外可训练参数极少。

英文原文摘要 1176 字符

Adapting vision-language models to remote sensing imagery remains challenging due to two key factors: limited semantic coverage in textual representations and insufficient adaptability of visual features. These issues are particularly significant in aerial scenes, which involve various visual appearances and fine-grained object distinctions. We propose AVION, a knowledge distillation framework tailored for remote sensing adaptation of vision-language models. The teacher module constructs semantically rich textual prototypes by collecting descriptions from a large language model and verifying validity using remote sensing image features. The student module integrates lightweight and learnable prompts into both vision and language encoders, guided by the teacher to align embeddings and their cross-modal relationships. Once trained, the student operates independently during inference. Experiments on six optical remote sensing benchmarks show that AVION improves few-shot classification and base-class accuracy without degrading generalization to novel categories. It also enhances mean recall for cross-modal retrieval, with minimal additional trainable parameters.

标签

会议 CVPR2026
页码 10106-10115
Bibkey
来源 CVF Open Access