MoBind：面向细粒度 IMU-视频姿态对齐的运动绑定
MoBind: Motion Binding for Fine-Grained IMU-Video Pose Alignment
把视频中提取的2D骨骼序列当作视觉侧表征，与IMU信号做分层对比对齐，为视觉研究者提供了一个绕开像素、用姿态序列做跨模态时序对齐与检索的可复用范式，尤其适合第一人称/可穿戴场景下的细粒度时序同步。
原文依据 MoBind aligns IMU signals with skeletal motion sequences rather than raw pixels.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论本文旨在学习惯性测量单元（IMU）信号与从视频中提取的2D姿态序列之间的联合表征，从而实现准确的跨模态检索、时序同步、受试者与身体部位定位以及动作识别。为此，我们提出 MoBind，一个分层对比学习框架，用于应对三项挑战：(1) 滤除无关的视觉背景；(2) 建模结构化的多传感器 IMU 配置；(3) 实现细粒度的亚秒级时序对齐。为分离出与运动相关的线索，MoBind 将 IMU 信号与骨骼运动序列而非原始像素进行对齐。我们进一步将全身运动分解为局部身体部位轨迹，并将每条轨迹与其对应的 IMU 配对，从而实现具有语义依据的多传感器对齐。为捕捉细致的时序对应关系，MoBind 采用分层对比策略，先对齐 token 级的时序片段，再将局部（身体部位）对齐与全局（全身）运动聚合相融合。在 mRi、TotalCapture 和 EgoHumans 上的评估表明，MoBind 在全部四项任务上均持续优于强基线，展现出稳健的细粒度时序对齐能力，同时保持跨模态的粗粒度语义一致性。
英文原文摘要 1290 字符
We aim to learn a joint representation between inertial measurement unit (IMU) signals and 2D pose sequences extracted from video, enabling accurate cross-modal retrieval, temporal synchronization, subject and body-part localization, and action recognition. To this end, we introduce MoBind, a hierarchical contrastive learning framework designed to address three challenges: (1) filtering out irrelevant visual background, (2) modeling structured multi-sensor IMU configurations, and (3) achieving fine-grained, sub-second temporal alignment. To isolate motion-relevant cues, MoBind aligns IMU signals with skeletal motion sequences rather than raw pixels. We further decompose full-body motion into local body-part trajectories, pairing each with its corresponding IMU to enable semantically grounded multi-sensor alignment. To capture detailed temporal correspondence, MoBind employs a hierarchical contrastive strategy that first aligns token-level temporal segments, then fuses local (body-part) alignment with global (body-wide) motion aggregation. Evaluated on mRi, TotalCapture, and EgoHumans, MoBind consistently outperforms strong baselines across all four tasks, demonstrating robust fine-grained temporal alignment while preserving coarse semantic consistency across modalities.