基于 ScaleFormer 的跨尺度全色锐化与 PanScale 基准
Cross-Scale Pansharpening via ScaleFormer and the PanScale Benchmark
把跨分辨率泛化重构成跨序列长度泛化，用可变长度图像块序列加 RoPE 外推到未见尺度，这一思路可直接迁移到超分、去噪等需要在不同分辨率间泛化的底层视觉任务。
原文依据 ScaleFormer reframes generalization across image resolutions as generalization across sequence lengths
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论全色锐化的目标是通过融合全色图像的空间细节与低分辨率多光谱数据的光谱丰富性，生成高分辨率多光谱图像。然而，现有方法大多在受限的低分辨率设定下进行评估，限制了其向真实高分辨率场景的泛化能力。为弥合这一差距，我们系统性地研究了跨尺度全色锐化在数据、算法与计算三方面的挑战。我们首先提出 PanScale，这是首个大规模跨尺度全色锐化数据集，并配套 PanScale-Bench——一个用于评估在不同分辨率与尺度下泛化能力的综合基准。为实现尺度泛化，我们提出 ScaleFormer，一种面向多尺度全色锐化的新架构。ScaleFormer 将跨图像分辨率的泛化重新表述为跨序列长度的泛化：它将图像 token 化为分辨率相同但长度与图像尺度成比例变化的图像块序列。一个尺度感知的图像块化模块使得能够从固定尺寸裁剪中训练出对这种变化鲁棒的模型。随后，ScaleFormer 将图像块内空间特征学习与图像块间序列依赖建模解耦，并引入旋转位置编码以增强对未见尺度的外推能力。大量实验表明，我们的方法在融合质量与跨尺度泛化方面优于当前最优方法。数据集与源代码见 https://github.com/caoke-963/ScaleFormer。
英文原文摘要 1506 字符
Pansharpening aims to generate high-resolution multi-spectral images by fusing the spatial detail of panchromatic images with the spectral richness of low-resolution MS data. However, most existing methods are evaluated under limited, low-resolution settings, limiting their generalization to real-world, high-resolution scenarios. To bridge this gap, we systematically investigate the data, algorithmic, and computational challenges of cross-scale pansharpening. We first introduce PanScale, the first large-scale, cross-scale pansharpening dataset, accompanied by PanScale-Bench, a comprehensive benchmark for evaluating generalization across varying resolutions and scales. To realize scale generalization, we propose ScaleFormer, a novel architecture designed for multi-scale pansharpening. ScaleFormer reframes generalization across image resolutions as generalization across sequence lengths: it tokenizes images into patch sequences of the same resolution but variable length proportional to image scale. A Scale-Aware Patchify module enables training for such variations from fixed-size crops. ScaleFormer then decouples intra-patch spatial feature learning from inter-patch sequential dependency modeling, incorporating Rotary Positional Encoding to enhance extrapolation to unseen scales. Extensive experiments show that our approach outperforms SOTA methods in fusion quality and cross-scale generalization. The datasets and source code are available at https://github.com/caoke-963/ScaleFormer.