DETACH：面向外视角视频与环境传感器的分解式时空对齐与分阶段学习
DETACH : Decomposed Spatio-Temporal Alignment for Exocentric Video and Ambient Sensors with Staged Learning
论文把外视角视频分解为空间与时序分量，并用在线聚类把环境传感器的隐式通道激活转成显式空间原型，从而建立跨模态空间视觉定位；这套「分解+原型视觉定位+加权对比」的流程可直接迁移到视频-音频、视频-IMU 等跨模态时序对齐任务，尤其是需要抑制假负样本的细粒度时序对齐场景。
原文依据 we propose DETACH, a decomposed spatio-temporal alignment framework
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论将第一视角视频与可穿戴传感器对齐已在人体动作识别中展现出潜力，但在用户舒适度、隐私顾虑与可扩展性方面存在实际局限。我们探索以外视角视频结合环境传感器作为一种非侵入、可扩展的替代方案。然而，在第一视角-可穿戴场景中普遍采用的全局对齐方法在这一新设定下失效，原因有两个问题：(P1) 无法捕捉诸如细微动作之类的局部细节；(P2) 过度依赖模态不变时序特征，从而扭曲了负样本关系。为解决这些问题，我们提出 DETACH，一个分解式时空对齐框架。通过将两种模态都分解为空间-时序分量，我们从空间特征中保留视频里的细微时序线索，并通过在线聚类将隐式的传感器通道激活转换为显式的空间原型，从而建立跨模态空间视觉定位。为避免过度依赖时序特征，一种时空加权对比损失利用该视觉定位进行细粒度时序对齐，优先关注困难负样本并抑制假负样本。在 Opportunity++ 与 HWU-USP 数据集上开展的下游任务综合实验表明，相较于经适配的第一视角-可穿戴基线，F1 最高提升 30%，平均精度均值最高提升 50%。
英文原文摘要 1350 字符
Aligning egocentric video with wearable sensors has shown promise for human action recognition, but faces practical limitations in user discomfort, privacy concerns, and scalability. We explore exocentric video with ambient sensors as a non-intrusive, scalable alternative. However, the Global Alignment approach prevalent in egocentric-wearable settings fails in this new setting due to two problems: (P1) inability to capture local details such as subtle motions, and (P2) over-reliance on modality-invariant temporal features that distort negative relationships. To resolve these problems, we propose DETACH, a decomposed spatio-temporal alignment framework. By decomposing both modalities into spatial-temporal components, we preserve subtle temporal cues in videos from spatial features, and convert implicit sensor channel activations into explicit spatial prototypes via online clustering, thereby establishing cross-modal spatial grounding. To avoid over-reliance on temporal features, a spatial-temporal weighted contrastive loss leverages this grounding for fine-grained temporal alignment, prioritizing hard negatives and suppressing false negatives. Comprehensive experiments with downstream tasks on Opportunity++ and HWU-USP datasets demonstrate improvements of up to 30% in F1 and 50% in mAP over adapted egocentric-wearable baselines.