DarkShake-DVS:低光照与相机抖动条件下基于事件相机的人体动作识别
DarkShake-DVS: Event-based Human Action Recognition under Low-light and Shaking Camera Conditions
用事件相机+IMU的非线性warping做运动补偿,把6-DoF抖动下的低光事件流重建为稳定输入,再接四阶段混合时空特征提取;对做事件视觉、视频动作识别和抖动视频稳定化的研究者提供了可复用的补偿前端与配套基准。
原文依据 an EIS module that reduces motion blur via a non-linear warping function to reconstruct a motion-compensated input
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论人体动作识别(HAR)是一项基础计算机视觉任务,具有多样的现实应用。实际部署常涉及低光照环境与无约束的6自由度相机运动,这些条件会降低视觉质量、破坏时序一致性,并损害现有方法的可靠性。事件相机具有高弱光敏感度和微秒级时序分辨率,配合惯性测量单元(IMU),是一种有前景的解决方案。然而,当前研究面临两个关键挑战:缺乏一个整合低光照条件、6自由度运动与同步IMU数据的基准;以及缺乏有效的运动补偿技术。为解决这些问题,我们提出Event-IMU Stabilized HAR(EIS-HAR),包含两个模块。第一个是EIS模块,通过非线性warping函数减少运动模糊,以重建运动补偿后的输入。第二个是HAR模块,采用四阶段混合架构,高效提取时空特征以实现准确的动作识别。为缓解数据稀缺,我们提出DarkShake-DVS,这是首个大规模基于事件相机的HAR基准,包含18,041段在低光照和剧烈6自由度运动下采集的真实世界视频片段,并辅以同步IMU数据。在三个数据集上的大量实验表明,EIS-HAR持续优于最优方法。
英文原文摘要 1395 字符
Human Action Recognition (HAR) is a fundamental computer vision task with diverse real-world applications. Practical deployments often involve low-light environments and unconstrained 6-DoF camera motion, conditions that degrade visual quality, disrupt temporal coherence, and compromise reliability of existing methods. Event cameras, with high low-light sensitivity and microsecond-level temporal resolution, paired with an inertial measurement unit (IMU), present a promising solution. However, current research faces two key challenges: absence of a benchmark integrating low-light conditions, 6-DoF motion, and synchronized IMU data; and lack of effective motion compensation techniques. To address these, we propose Event-IMU Stabilized HAR (EIS-HAR), with two modules. The first is an EIS module that reduces motion blur via a non-linear warping function to reconstruct a motion-compensated input. The second is a HAR module with a four-stage hybrid architecture to efficiently extract spatiotemporal features for accurate action recognition. To alleviate data scarcity, we introduce DarkShake-DVS, the first large-scale event-based HAR benchmark that includes 18,041 real-world clips captured in low light and intense 6-DoF motion, supplemented by synchronized IMU data. Extensive experiments on three datasets demonstrate consistent superiority of EIS-HAR over state-of-the-art methods.