定制化融合:面向自适应多任务感知红外-可见光图像融合的闭环动态网络
Customized Fusion: A Closed-Loop Dynamic Network for Adaptive Multi-Task-Aware Infrared-Visible Image Fusion
把下游任务的反馈以闭环方式回传到融合网络,并用基向量库+架构自适应注入按任务定制网络结构,为多任务图像融合提供了一种无需重训即可切换任务语义的融合框架,可迁移到其他多任务底层视觉问题。
原文依据 a Closed-Loop Dynamic Network (CLDyN) that can adaptively respond to the semantic requirements of diverse downstream tasks
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论红外-可见光图像融合旨在整合互补信息以实现鲁棒的视觉理解,但现有融合方法难以同时适应多个下游任务。为解决该问题,本文提出闭环动态网络(CLDyN),能够自适应地响应多种下游任务的语义需求,实现面向任务定制的图像融合。具体而言,CLDyN 引入闭环优化机制,通过需求驱动的语义补偿(RSC)模块建立语义传输链,实现下游任务对融合网络的显式反馈。RSC 模块利用基向量库(BVB)与架构自适应语义注入(A2SI)模块,根据任务需求定制网络架构,从而实现任务特定的语义补偿,使融合网络无需重新训练即可主动适应多种任务。为促进语义补偿,本文引入奖励-惩罚策略,依据任务性能变化对 RSC 模块进行奖励或惩罚。在 M3FD、FMB 和 VT5000 数据集上的实验表明,CLDyN 不仅保持了较高的融合质量,还展现出较强的多任务适应能力。代码已发布于 https://github.com/YR0211/CLDyN。
英文原文摘要 1337 字符
Infrared-visible image fusion aims to integrate complementary information for robust visual understanding, but existing fusion methods struggle with simultaneously adapting to multiple downstream tasks. To address this issue, we propose a Closed-Loop Dynamic Network (CLDyN) that can adaptively respond to the semantic requirements of diverse downstream tasks for task-customized image fusion. Specifically, CLDyN introduces a closed-loop optimization mechanism that establishes a semantic transmission chain to achieve explicit feedback from downstream tasks to the fusion network through a Requirement-driven Semantic Compensation (RSC) module. The RSC module leverages a Basis Vector Bank (BVB) and an Architecture-Adaptive Semantic Injection (A2SI) block to customize the network architecture according to task requirements, thereby enabling task-specific semantic compensation and allowing the fusion network to actively adapt to diverse tasks without retraining. To promote semantic compensation, a reward-penalty strategy is introduced to reward or penalize the RSC module based on task performance variations. Experiments on the M3FD, FMB, and VT5000 datasets demonstrate that CLDyN not only maintains high fusion quality but also exhibits strong multi-task adaptability. The code is available at https://github.com/YR0211/CLDyN.