通过基于解混的丰度融合学习增强未配准高光谱图像超分辨率
Enhancing Unregistered Hyperspectral Image Super-Resolution via Unmixing-based Abundance Fusion Learning
针对未配准参考图像的超分辨率问题，提出基于解混的丰度融合框架，利用可变形聚合和交叉注意力实现空间-光谱信息解耦与融合，可迁移到其他多模态图像融合任务。
原文依据 we propose an unmixing-based fusion framework that decouples spatial-spectral information to simultaneously mitigate the impact of unregistered fusion
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论未配准高光谱图像（HSI）超分辨率（SR）通常旨在利用未配准的高分辨率参考图像增强低分辨率HSI。本文提出一种基于解混的融合框架，通过解耦空间-光谱信息，同时减轻未配准融合的影响并提升SR模型的可学习性。具体而言，我们首先利用奇异值分解进行初始光谱解混，保留原始端元，并将后续网络专门用于增强初始丰度图。为了利用未配准参考图像的空间纹理，我们引入一种由粗到细的可变形聚合模块，该模块首先使用粗金字塔预测器估计像素级光流和相似度图，然后进行精细的亚像素级细化，以实现参考特征的可变形聚合。聚合后的特征随后通过一系列空间-通道丰度交叉注意力模块进行细化。此外，我们提出一种空间-通道调制融合模块，利用动态门控权重融合编码器-解码器特征，生成高质量、高分辨率的HSI。在模拟和真实数据集上的实验结果证实，所提方法达到了最优的超分辨率性能。代码将发布于 https://github.com/yingkai-zhang/UAFL。
英文原文摘要 1400 字符
Unregistered hyperspectral image (HSI) super-resolution (SR) typically aims to enhance a low-resolution HSI using an unregistered high-resolution reference image. In this paper, we propose an unmixing-based fusion framework that decouples spatial-spectral information to simultaneously mitigate the impact of unregistered fusion and enhance the learnability of SR models. Specifically, we first utilize singular value decomposition for initial spectral unmixing, preserving the original endmembers while dedicating the subsequent network to enhancing the initial abundance map. To leverage the spatial texture of the unregistered reference, we introduce a coarse-to-fine deformable aggregation module, which first estimates a pixel-level flow and a similarity map using a coarse pyramid predictor. It further performs fine sub-pixel refinement to achieve deformable aggregation of the reference features. The aggregative features are then refined via a series of spatial-channel abundance cross-attention blocks. Furthermore, a spatial-channel modulated fusion module is presented to merge encoder-decoder features using dynamic gating weights, yielding a high-quality, high-resolution HSI. Experimental results on simulated and real datasets confirm that our proposed method achieves state-of-the-art super-resolution performance. The code will be available at https://github.com/yingkai-zhang/UAFL.