超越共享信息：从中间层恢复丢失的独特信息以提升多模态地理基础模型
Beyond What's Shared: Recovering Lost Unique Information from Intermediate Layers to Boost Multimodal Geo-Foundation Models
对做多模态对比学习（如 CLIP 式图文对齐）的人，这篇给出一个几乎零成本的改进：把中间层与最后一层特征融合，可恢复被对齐目标压掉的模态特有信息；其逐层分析（模态间隙、表征相似度、互信息）也可直接迁移到视觉-语言模型的特征选择与层融合策略上。
原文依据 fusing intermediate (more unique) and final (more shared) representations yields consistent gains on diverse geospatial tasks
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论学习地理位置的通用表征已成为人口估计与环境监测等地理空间任务的基础。为获得此类表征，多模态地理基础模型常采用对比学习将卫星影像与地理坐标对齐，并隐含假设跨模态（共享）信息足以支撑下游任务。然而，鉴于任务范围广泛，任务相关信息可能位于共享空间之外，因此保留模态特有（独特）特征可提升任务性能。已有方法通过额外的训练目标或外部模型来保留独特信息，增加了训练复杂度。受「浅层捕获通用输入特征、深层趋于任务特定」这一传统认识的启发，我们假设对比学习模型的中间层比经对齐优化的最后一层保留了更多模态特有结构。通过对模态间隙、表征相似度与互信息的逐层三重分析，我们验证了这一趋势，并发现融合中间层（更具独特性）与最后一层（更具共享性）表征可在多种地理空间任务上带来一致增益。我们的发现揭示了对比学习模型中被低估的信息多样性，并表明简单的逐层融合是获得更丰富地理嵌入的一条高效路径。
英文原文摘要 1393 字符
Learning general-purpose representations of geographic locations has become essential to geospatial tasks such as population estimation and environmental monitoring. To obtain such representations, multimodal geo-foundation models often use contrastive learning (CL) to align satellite imagery with geo-coordinates, implicitly assuming that cross-modal (shared) information suffices for downstream tasks. However, given the breadth of tasks, task-relevant information may lie beyond the shared space, so retaining modality-specific (unique) features can improve task performance. Prior methods retain unique information through extra training objectives or external models, increasing training complexity. Motivated by the conventional wisdom that earlier layers capture general input features while later layers become task-specific, we hypothesize that intermediate layers in CL models retain more modality-specific structure than the alignment-optimized final layer. Through a trifecta layerwise analysis of modality gap, representation similarity, and mutual information, we validate this trend and find that fusing intermediate (more unique) and final (more shared) representations yields consistent gains on diverse geospatial tasks. Our findings reveal underutilized information diversity in CL models and show that simple layerwise fusion is an efficient path to richer geo-embeddings.