协同展开的联合光谱图像重建与语义分割
Joint Spectral Image Reconstruction and Semantic Segmentation with Cooperative Unfolding
将重建与分割两个任务耦合进同一个深度展开框架，并用跨聚合超令牌注意力在两者之间传递表征，为「复原任务与高层任务联合优化」提供了可复用的展开式设计思路。
原文依据 we propose the first Cooperative Reconstruction-Segmentation Deep Unfolding Network (CRSDUN) to solve the reconstruction and segmentation tasks in parallel
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论编码孔径快照光谱成像（CASSI）是一种新兴的高光谱图像（HSI）获取技术，可服务于下游语义分割任务。由于 CASSI 系统本身具有病态性，典型解决方案被迫采用「先重建、后分割」的两阶段流程，即把二者视为两个独立任务。然而，我们观察到这两个任务相互关联，且在表征学习上相互促进，因此将它们分离会限制整体精度与效率。为此，我们提出首个协同重建-分割深度展开网络（CRSDUN），以并行方式求解重建与分割任务。为使二者相互促进，我们引入跨聚合超令牌注意力（CASTA）机制，以增强 HSI 重建与语义分割之间的表征交互。在合成与真实 HSI 重建-分割数据集上的大量实验表明，我们的方法在光谱重建与语义分割两方面均达到最优。代码见 https://github.com/zjhe02/CRSDUN 。
英文原文摘要 1207 字符
Coded Aperture Snapshot Spectral Imaging (CASSI) is an emerging hyperspectral image (HSI) acquisition technique for downstream semantic segmentation. Due to the ill-posedness nature of CASSI systems, typical solutions are compelled to conduct a two-stage reconstruction-then-segmentation pipeline, namely viewing them as two separate tasks. However, we observe that such two tasks are interrelated and mutually reinforcing for representation learning, and thus separating them limits the overall accuracy and efficiency. To this end, we propose the first Cooperative Reconstruction-Segmentation Deep Unfolding Network (CRSDUN) to solve the reconstruction and segmentation tasks in parallel. To make the two mutually reinforcing, we introduce the Cross-Aggregated Super-Token Attention (CASTA) mechanism to enhance the representation interactions between HSI reconstruction and semantic segmentation. Extensive experiments on both synthetic and real-world HSI reconstruction-segmentation datasets demonstrate that our method achieves state-of-the-art in both spectral reconstruction and semantic segmentation. The code is available at \href https://github.com/zjhe02/CRSDUN https://github.com/zjhe02/CRSDUN .