DIMOS:解耦实例级运动目标分割
DIMOS: Disentangling Instance-level Moving Object Segmentation
双解耦特征提取把图像与事件模态各自拆成外观流和运动流,再配合多粒度跨模态对齐做融合;这套「先解耦再对齐」的流程可直接迁移到其他事件-图像融合任务(如事件辅助的检测、深度估计),也为小目标运动分割提供了可复用的特征增密思路。
原文依据 we first propose a dual-disentangling feature extraction framework that separates and extracts appearance and motion information within both image and event modalities
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论运动实例分割(MIS)因在交通监控、自动驾驶和动物追踪中的广泛应用而受到越来越多的关注。事件相机记录异步亮度变化,具有高时间分辨率和动态范围,因而对运动信息高度敏感。通过融合事件与图像特征,来自事件的运动线索可以补充图像中的空间细节,从而提升 MIS 的性能。然而,现有多模态 MIS 方法仍难以分割小型运动实例,因为事件相机在有限分辨率下往往产生稀疏特征。此外,事件特征将外观属性与运动线索纠缠在一起,进一步限制了有效的跨模态融合。为应对这些挑战,我们首先提出一种双解耦特征提取框架,在图像与事件两种模态内部将外观信息与运动信息分离并分别提取,从而提升特征密度。随后,我们引入多粒度跨模态对齐,在分布与语义层面将跨模态一致的特征对齐,从而在融合时获得丰富的空间与时序细节。实验结果表明,我们的方法在多模态 MIS 上取得了最优性能,尤其是在快速运动和低光照等挑战性条件下的小型实例上。
英文原文摘要 1420 字符
Moving instance segmentation (MIS) attracts increasing attention due to its broad applications in traffic surveillance, autonomous driving, and animal tracking. Event cameras record asynchronous brightness changes, providing high temporal resolution and dynamic range, which makes them highly sensitive to motion information. By fusing event and image features, motion cues from events can complement spatial details from images, enhancing the performance of MIS. However, current multimodal MIS methods still struggle to segment small moving instances, as event cameras often yield sparse features under limited resolution. Moreover, event features entangle appearance attributes with motion cues, which further restricts effective cross-modal fusion. To address these challenges, we first propose a dual-disentangling feature extraction framework that separates and extracts appearance and motion information within both image and event modalities, thereby improving feature density. Subsequently, a multi-granularity cross-modal alignment is introduced to align distributionally and semantically consistent features across modalities, enabling more effective fusion with rich spatial and temporal details. The experiment results demonstrate that our method achieves state-of-the-art performance in multimodal MIS, especially for small instances under challenging conditions such as fast motion and low-light settings.