多粒度感知语义原型扫描与三词元提示学习驱动的高阶 RWKV 全色锐化
Multigrain-aware Semantic Prototype Scanning and Tri-Token Prompt Learning Embraced High-Order RWKV for Pan-Sharpening
将 RWKV 的栅格扫描替换为基于局部敏感哈希的语义原型扫描，并用全局/原型/寄存器三词元提示词注入语义先验，可直接迁移到其他线性复杂度视觉骨干的全局建模与词元重排设计中。
原文依据 we introduce a semantic-driven scanning strategy that leverages locality-sensitive hashing to group semantically related regions
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论本文提出一种面向全色锐化的多粒度感知语义原型扫描范式，其建立在高阶 RWKV 架构与源自语义聚类的三词元提示机制之上。具体而言，本方法包含三个关键部分：1）多粒度感知语义原型扫描。尽管 RWKV 为 Transformer 提供了一种高效的线性复杂度替代方案，但其常规的双向栅格扫描仍是语义无关的，且易受位置偏置影响。为解决该问题，我们引入一种语义驱动的扫描策略，利用局部敏感哈希对语义相关区域进行分组并构建多粒度语义原型，从而实现上下文感知的词元重排序与更连贯的全局交互。2）三词元提示学习。我们设计了一种由全局词元、聚类导出的原型词元以及可学习寄存器词元组成的三词元提示机制。全局词元与原型词元为 RWKV 建模提供互补的语义先验，而寄存器词元有助于抑制含噪且易产生伪影的中间表示。3）可逆 Q-Shift。为对抗空间细节的损失，我们在值通路上应用中心差分卷积以注入高频信息，并引入可逆的多尺度 Q-Shift 操作，以实现高效且无损的特征变换，而无需依赖参数繁重的感受野扩展。实验结果表明了本方法的优越性。
英文原文摘要 1506 字符
In this work, we propose a Multigrain-aware Semantic Prototype Scanning paradigm for pan-sharpening, built upon a high-order RWKV architecture and a tri-token prompting mechanism derived from semantic clustering. Specifically, our method contains three key components: 1) Multigrain-aware Semantic Prototype Scanning. Although RWKV offers an efficient linear-complexity alternative to Transformers, its conventional bidirectional raster scanning is still semantic-agnostic and prone to positional bias. To address this issue, we introduce a semantic-driven scanning strategy that leverages locality-sensitive hashing to group semantically related regions and construct multi-grain semantic prototypes, enabling context-aware token reordering and more coherent global interaction. 2) Tri-token Prompt Learning. We design a tri-token prompting mechanism consisting of a global token, cluster-derived prototype tokens, and a learnable register token. The global and prototype tokens provide complementary semantic priors for RWKV modeling, while the register token helps suppress noisy and artifact-prone intermediate representations. 3) Invertible Q-Shift. To counteract spatial details, we apply center difference convolution on the value pathway to inject high-frequency information, and introduce an invertible multi-scale Q-shift operation for efficient and lossless feature transformation without parameter-heavy receptive field expansion. Experimental results demonstrate the superiority of our method.