利用快速旋转辐条状中性密度滤光片从多衰减脉冲重建高帧率与高动态范围视频
HFR and HDR Video from Multi-Attenuated Spikes Using a Rapidly Rotating SpokeND Filter
用旋转辐条状中性密度滤光片让脉冲相机每个像素周期性获得多档衰减,从而在单传感器上同时恢复空间一致的增益与高帧率 HDR 视频;这套「空间-时序联合衰减 + 增益恢复」思路可迁移到事件相机/脉冲相机的 HDR 与高速成像重建。
原文依据 placing a rapidly rotating spoke-pattern neutral density (SpokeND) filter in front of the spike camera
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论同时具备高动态范围(HDR)与高速运动的场景对传统相机而言仍难以捕捉。现有的交替曝光方法会加剧时序分辨率损失,因而不适用于高速场景。因此,当前方案通常要么通过固定的空间变化衰减水平牺牲空间分辨率,要么采用多传感器配置来维持时序分辨率。本文利用超高速脉冲相机对入射光实现空间与时序衰减,从而以单传感器重建高帧率(HFR)与 HDR 视频。我们通过在脉冲相机前放置快速旋转的辐条图案中性密度(SpokeND)滤光片来实现这一点,使每个像素在保持全空间分辨率的同时周期性捕获多衰减脉冲。基于这些多衰减脉冲,我们提出 ReST-Net,其包含 ReGain 与 ReFine 两个模块。ReGain 模块通过学习从多衰减脉冲中恢复相对增益来重建空间一致的帧,ReFine 模块则消除时序波动以生成时序一致的 HDR 视频。在合成数据与真实数据上的大量实验表明,我们的方法能够重建高达 2000 FPS 的 HDR 视频。
英文原文摘要 1345 字符
Capturing scenes with both high dynamic range (HDR) and high-speed motion remains challenging for conventional cameras. Existing alternating-exposure approaches exacerbate temporal resolution loss, making them unsuitable for high-speed scenes. Consequently, current solutions typically compromise either spatial resolution through fixed spatial-varying attenuation levels or employ multi-sensor configurations to maintain temporal resolution. In this paper, we leverage an ultra-high speed spike camera to enable spatial and temporal attenuation of incident light, thereby reconstructing high frame rate (HFR) and HDR video with a single sensor. We achieve this by placing a rapidly rotating spoke-pattern neutral density (SpokeND) filter in front of the spike camera, enabling each pixel to periodically capture multi-attenuated spikes while maintaining full spatial resolution. Building on these multi-attenuated spikes, we propose ReST-Net, which comprises the ReGain and ReFine modules. The ReGain module reconstructs spatially consistent frames by learning to recover relative gain from the multi-attenuated spikes, and the ReFine module removes temporal fluctuations to produce temporally consistent HDR videos. Extensive experiments on synthetic and real-world data demonstrate that our method can reconstruct HDR video at up to 2000 FPS.