GFRRN:弥合单图反射去除中的鸿沟
GFRRN: Explore the Gaps in Single Image Reflection Removal
用可学习 Mona 层做 PEFT 对齐预训练特征与复原模型特征,并用标签生成器统一合成/真实反射标签,可直接迁移到去雨、去雾等存在合成-真实域差的低层复原任务。
原文依据 we first adopt the parameter efficient fine-tuning (PEFT) strategy by integrating several learnable Mona layers into the pre-trained model
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论具有特征交互机制的先验双流方法在单图反射去除(SIRR)上已取得显著性能。然而,它们往往受困于:(1)预训练模型特征与反射去除模型特征之间的语义理解鸿沟;(2)合成数据与真实世界训练数据之间的反射标签不一致。在本工作中,我们首先采用参数高效微调(PEFT)策略,通过向预训练模型集成若干可学习的 Mona 层来对齐训练方向。随后,设计了一个标签生成器,以统一合成数据与真实世界数据的反射标签。此外,提出了基于高斯分布的自适应频率学习模块(G-AFLB),以自适应地学习并融合频率先验;并采用动态代理注意力(DAA)作为基于窗口的注意力的替代方案,通过对窗口之间(inter-)与单个窗口内部(intra-)的重要性程度进行动态建模。这些组件构成了我们提出的无鸿沟反射去除网络(GFRRN)。大量实验证明了我们的 GFRRN 的有效性,其性能优于最优的 SIRR 方法。
英文原文摘要 1239 字符
Prior dual-stream methods with the feature interaction mechanism have achieved remarkable performance in single image reflection removal (SIRR). However, they often struggle with (1) semantic understanding gap between the features of pre-trained models and those of reflection removal models, and (2) reflection label inconsistencies between synthetic and real-world training data. In this work, we first adopt the parameter efficient fine-tuning (PEFT) strategy by integrating several learnable Mona layers into the pre-trained model to align the training directions. Then, a label generator is designed to unify the reflection labels for both synthetic and real-world data. In addition, a Gaussian-based Adaptive Frequency Learning Block (G-AFLB) is proposed to adaptively learn and fuse the frequency priors, and a Dynamic Agent Attention (DAA) is employed as an alternative to window-based attention by dynamically modeling the significance levels across windows (inter-) and within an individual window (intra-). These components constitute our proposed Gap-Free Reflection Removal Network (GFRRN). Extensive experiments demonstrate the effectiveness of our GFRRN, achieving superior performance against state-of-the-art SIRR methods.