ELVIS:增强低光照以用于暗光视频实例分割
ELVIS: Enhance Low-Light for Video Instance Segmentation in the Dark
把低光照视频的时空退化建模成可无监督合成的数据管线,再用免标定的退化估计网络加增强解码器检测头把退化从内容特征中解耦,可直接迁移到任何在暗光视频上失效的时序感知模型(VIS、跟踪、光流)做域自适应。
原文依据 an unsupervised synthetic low-light video pipeline that models both spatial and temporal degradations
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论由于噪声、模糊及其他不利条件,面向低光照内容的视频实例分割(VIS)对人类和机器而言都极具挑战性。大规模标注数据集的缺乏以及现有合成管线(尤其在建模时序退化方面)的局限,进一步阻碍了进展。此外,现有 VIS 方法对低光照视频中的退化并不鲁棒,因此即便经过微调也表现不佳。本文提出 ELVIS(Enhance Low-Light for Video Instance Segmentation),一个使最优 VIS 模型能够域自适应到低光照场景的框架。ELVIS 由三部分组成:一个对空间与时序退化同时建模的无监督合成低光照视频管线、一个免标定的退化轮廓估计网络(VDP-Net),以及一个将退化与内容特征解耦的增强解码器检测头。ELVIS 在合成的低光照 YouTube-VIS 2019 数据集上最高提升 +3.7AP,并在真实低光照视频上以至少 +2.8AP 优于两阶段基线。代码与数据集见:https://joannelin168.github.io/research/ELVIS
英文原文摘要 1196 字符
Video instance segmentation (VIS) for low-light content remains highly challenging for both humans and machines alike, due to noise, blur and other adverse conditions. The lack of large-scale annotated datasets and the limitations of current synthetic pipelines, particularly in modeling temporal degradations, further hinder progress. Moreover, existing VIS methods are not robust to the degradations found in low-light videos and, consequently, perform poorly even after finetuning. In this paper, we introduce ELVIS (Enhance Low-Light for Video Instance Segmentation), a framework that enables domain adaptation of state-of-the-art VIS models to low-light scenarios. ELVIS is comprised of an unsupervised synthetic low-light video pipeline that models both spatial and temporal degradations, a calibration-free degradation profile estimation network (VDP-Net) and an enhancement decoder head that disentangles degradations from content features. ELVIS improves performances by up to +3.7AP on the synthetic low-light YouTube-VIS 2019 dataset and beats two-stage baselines by at least +2.8AP on real low-light videos. Code and dataset available at: https://joannelin168.github.io/research/ELVIS