面向卫星影像的光谱蒸馏表征与指令增强 LLM 对齐
Spectrally Distilled Representations Aligned with Instruction-Augmented LLMs for Satellite Imagery
用冻结的多光谱教师模型向 RGB 学生模型蒸馏光谱先验，再把视觉空间对齐到指令增强 LLM 的嵌入空间，使推理阶段只需 RGB 输入即可获得光谱感知的跨模态表征；这套「教师蒸馏 + LLM 对齐」的两阶段范式可直接迁移到遥感或其他多模态缺失场景的视觉语言预训练。
原文依据 Spectral Representation Distillation transfers spectral priors from a frozen multi-spectral teacher to an RGB student via a lightweight projector
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉语言基础模型（VLFM）有望为对地观测带来零样本与检索式理解能力。然而，实际运行的卫星系统往往缺乏完整的多光谱覆盖，使得仅用 RGB 进行推理对于可扩展部署极具吸引力；但 VLFM 在卫星影像上的采用仍受两个因素制约：（1）多光谱输入信息丰富，却因波段冗余与失配而难以被一致地利用；（2）CLIP 式文本编码器限制了语义表达能力，并削弱了细粒度对齐。我们提出 SATtxt，一种在推理时仅需 RGB 输入、却保留了训练阶段所学光谱线索并产生丰富跨模态表征的 VLFM。该框架包含两个阶段。首先，光谱表征蒸馏通过一个轻量投影器，将光谱先验从冻结的多光谱教师模型迁移到 RGB 学生模型。其次，光谱锚定的指令增强 LLM 对齐将蒸馏得到的视觉空间与具有强表达能力的 LLM 嵌入空间连接起来。在多个卫星基准上，SATtxt 相较基线在零样本分类上平均提升 4.2%，检索提升 5.9%，线性探测提升 2.7%，开放词表分割提升 2.8%，为对地观测展示了一条通向光谱感知视觉语言学习的高效路径。项目主页：https://ikhado.github.io/sattxt/
英文原文摘要 1389 字符
Vision-language foundation models (VLFMs) promise zero-shot and retrieval understanding for Earth observation. While operational satellite systems often lack full multi-spectral coverage, making RGB-only inference highly desirable for scalable deployment, the adoption of VLFMs for satellite imagery remains hindered by two factors: (1) multi-spectral inputs are informative but difficult to exploit consistently due to band redundancy and misalignment; and (2) CLIP-style text encoders limit semantic expressiveness and weaken fine-grained alignment. We present SATtxt, a VLFM that operates with RGB inputs at inference while retaining spectral cues learned during training and producing rich cross-modal representations. Our framework comprises two stages. First, Spectral Representation Distillation transfers spectral priors from a frozen multi-spectral teacher to an RGB student via a lightweight projector. Second, Spectrally Grounded Alignment with Instruction-Augmented LLMs bridges the distilled visual space and an expressive LLM embedding space. Across satellite benchmarks, SATtxt improves zero-shot classification on average by 4.2%, retrieval by 5.9%, linear probing by 2.7%, and open-vocabulary segmentation by 2.8% over baselines, showing an efficient path toward spectrum-aware vision-language learning for Earth observation. Project page: https://ikhado.github.io/sattxt/