ARES:统一非对称 RGB-事件双目以实现概率化场景流估计
ARES: Unifying Asymmetric RGB-Event Stereo for Probabilistic Scene Flow Estimation
用事件相机与 RGB 构成非对称双目,并通过跨视角约束下的多模态上下文注意力构建统一对应空间,再以时序视差后验融合做概率化场景流推断,为高速动态场景的三维运动估计提供了可复用的融合与概率建模思路。
原文依据 a hybrid setup where an event camera captures fine grained temporal dynamics and an RGB camera provides rich spatial structure
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论在动态高速场景中估计稠密三维运动仍然具有挑战性,原因在于运动模糊、光照变化以及传统相机有限的时间分辨率。我们提出 ARES,一个面向非对称 RGB-事件双目的统一框架,通过由事件相机捕获细粒度时序动态、由 RGB 相机提供丰富空间结构的混合配置来解决上述问题。为整合这些异构模态,我们提出多模态上下文注意力,这是一种基于 Transformer 的融合机制,在跨视角约束下关注空间与时序上下文,并为视差与光流估计形成统一的对应空间。在该共享表示的基础上,我们提出时序视差后验融合,这是一个概率框架,通过建模视差后验的演化来推断视差变化并恢复度量一致的场景流。在稀疏监督与稠密自一致性线索下训练,我们的 ARES 在多种驾驶场景中实现了几何一致且时序稳定的三维运动估计。实验表明,ARES 在 RGB-事件双目方法的场景流估计中取得了最先进的性能,为统一非对称多模态双目感知建立了一条有原则的路径。代码见项目网站。
英文原文摘要 1395 字符
Estimating dense three dimensional motion in dynamic high speed scenes remains challenging due to motion blur, illumination variation, and the limited temporal resolution of conventional cameras. We introduce ARES, a unified framework for Asymmetric RGB-Event Stereo that addresses these issues through a hybrid setup where an event camera captures fine grained temporal dynamics and an RGB camera provides rich spatial structure. To integrate these heterogeneous modalities, we propose Multimodal Contextual Attention, a transformer based fusion mechanism that attends to spatial and temporal contexts under cross view constraints and forms a unified correspondence space for disparity and optical flow estimation. Building on this shared representation, we introduce Temporal Disparity Posterior Fusion, a probabilistic framework that models the evolution of disparity posteriors to infer disparity change and recover metrically coherent scene flow. Trained with sparse supervision and dense self consistency cues, our ARES achieves geometrically consistent and temporally stable three dimensional motion estimation across diverse driving scenarios. Experiments show that ARES attains state of the art performance in scene flow estimation among RGB-event stereo methods, establishing a principled path toward unified asymmetric multimodal stereo sensing. Code available at the project website.