自监督与表征学习 · CVPR2026 ·2026.06.03

读懂你的动作:通过预训练 AEMG 学习可泛化的动作表征

Reading Your Actions: Learning Generalizable Action Representations via Pre-training AEMG

Zhenghao Huang、Huilin Yao、Kaikai Wang、Lin Shu

与你有关的部分

把生理时序信号(EMG)当作「语言」来做自监督预训练的思路,可迁移到视觉-生理多模态(如可穿戴相机+肌电的手势/动作识别)中:用离散化分词器把连续信号转成结构词、再用掩码/语言建模目标预训练,从而获得跨设备、跨被试的零样本泛化表征。

原文依据 we propose Any Electromyography (AEMG), the first large-scale, self-supervised representation learning framework for EMG
可迁移到 人体动作识别手势识别多模态表征学习跨域泛化

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

肌电信号(EMG)在解码人类运动意图、实现直观人机交互方面发挥着基础性作用。然而,受数据异质性、标签稀缺以及缺乏统一表征框架的限制,其在跨被试、跨设备和跨任务上的泛化能力仍十分有限。为弥补这一空白,我们提出 Any Electromyography(AEMG),这是首个面向 EMG 的大规模自监督表征学习框架。AEMG 以语言学的方式重新概念化神经肌肉动力学,采用一种新颖的神经肌肉收缩分词器(NCT),将离散的肌肉收缩转译为结构化词语,将时序激活模式转译为连贯句子。此外,我们构建了迄今规模最大的跨设备 EMG 信号词表,从而支持在任意通道拓扑与采样率之间无缝迁移。实验表明,与六个最优基线相比,AEMG 将零样本留一被试(LOSO)准确率提升了 5.79–9.25%,并且仅使用 5% 的目标用户数据即可达到超过 90% 的少样本自适应性能。我们的工作提出了将 EMG 信号视为跨设备生理语言的概念,从海量数据中学习其语法,并为单一训练、普遍适用的 EMG 基础模型奠定了基础。我们的代码、演示及更多资源见 https://github.com/AEMG-series/AEMG。

英文原文摘要 1454 字符

A fundamental role in decoding human motor intent and enabling intuitive human-computer interaction is played by electromyography (EMG). However, its generalization capability across subjects, devices, and tasks remains substantially limited by data heterogeneity, label scarcity, and the lack of a unified representational framework. To bridge this gap, we propose Any Electromyography (AEMG), the first large-scale, self-supervised representation learning framework for EMG. AEMG reconceptualizes neuromuscular dynamics linguistically, utilizing a novel Neuromuscular Contraction Tokenizer (NCT) to translate discrete muscle contractions into structural words and temporal activation patterns into coherent sentences. Furthermore, we compile the largest cross-device EMG signal vocabulary to date, enabling seamless transfer across arbitrary channel topologies and sampling rates. Experiments demonstrate that AEMG improves the zero-shot leave-one-subject-out (LOSO) accuracy by 5.79--9.25% compared to six state-of-the-art baselines, and achieves more than 90% few-shot adaptation performance with only 5% of target user data. Our work has proposed the concept of EMG signals as a cross-device physiological language, learned their grammar from massive amounts of data, and laid the groundwork for a single-training, universally applicable EMG foundation model. Our code, demos, and more resources are available at https://github.com/AEMG-series/AEMG.

标签

会议 CVPR2026
页码 20108-20117
Bibkey
来源 CVF Open Access