TerraScope:面向地球观测的像素级视觉推理
TerraScope: Pixel-Grounded Visual Reasoning for Earth Observation
把像素级掩码嵌入推理链,让 VLM 在输出答案的同时给出可验证的分割证据,这一「推理链内嵌掩码」的监督构造方式可迁移到需要空间定位的视觉问答与指代任务;其光学/SAR 自适应融合与多时相序列建模也为遥感 VLM 提供了可复用的输入组织范式。
原文依据 a large-scale dataset containing 1 million samples with pixel-level masks embedded in reasoning chains
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉语言模型(VLM)在地球观测(EO)中已展现出潜力,但它们在需要将复杂空间推理建立在精确像素级视觉表征之上的任务中仍表现不佳。为解决这一问题,我们提出 TerraScope,一个统一的 VLM,能够实现像素级地理空间推理,具备两项关键能力:(1)模态灵活推理:它可处理单模态输入(光学或 SAR),并在两种模态同时可用时自适应地将不同模态融合进推理过程;(2)多时相推理:它整合时序序列以进行跨多个时间点的变化分析。此外,我们构建了 Terra-CoT,一个大规模数据集,包含 100 万个样本,其推理链中嵌入了来自多源数据的像素级掩码。我们还提出 TerraScope-Bench,这是首个面向像素级地理空间推理的基准,包含六个子任务,同时评估答案准确性与掩码质量,以确保实现真正的像素级推理。实验表明,TerraScope 在像素级地理空间推理上显著优于现有 VLM,同时提供可解释的视觉证据。
英文原文摘要 1164 字符
Vision-language models (VLMs) have shown promise in earth observation (EO), yet they struggle with tasks that require grounding complex spatial reasoning in precise pixel-level visual representations. To address this problem, we introduce TerraScope, a unified VLM that delivers pixel-grounded geospatial reasoning with two key capabilities: (1) modality-flexible reasoning: it handles single-modality inputs (optical or SAR) and adaptively fuses different modalities into the reasoning process when both are available; (2) multi-temporal reasoning: it integrates temporal sequences for change analysis across multiple time points. In addition, we curate Terra-CoT, a large-scale dataset containing 1 million samples with pixel-level masks embedded in reasoning chains across multiple sources. We also propose TerraScope-Bench, the first benchmark for pixel-grounded geospatial reasoning with six sub-tasks that evaluates both answer accuracy and mask quality to ensure authentic pixel-grounded reasoning. Experiments show that TerraScope significantly outperforms existing VLMs on pixel-grounded geospatial reasoning while providing interpretable visual evidence.