自监督与表征学习 · CVPR2026 ·2026.06.03

PRISM:学习用于可迁移骨架动作表示的共享基元空间

PRISM: Learning a Shared Primitive Space for Transferable Skeleton Action Representation

Di Yang、Yaohui Wang、Shuai Shao、François Brémond、Jiangtao Wang

与你有关的部分

将动作序列表示为运动基元系数空间中的轨迹,并通过运动生成目标学习该表征,再迁移到分类与检测任务;这种以生成驱动表征学习、并支持跨任务迁移的思路,可复用于其他时序或结构化数据的表征学习。

原文依据 learns a shared motion representation from a motion generation objective and transfers it to perception tasks
可迁移到 骨架动作识别动作检测动作生成

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

真实世界的人体动作理解仍然具有挑战性,原因包括长尾标签分布、组合式运动模式以及视角变化。现有的基于骨架的方法通常缺乏结构化且可迁移的运动表示,而用于生成、分类和检测的任务特定模型通常独立训练,导致流程碎片化且跨任务泛化能力有限。我们提出 PRISM,一个以基元为中心的骨架建模框架,它从运动生成目标中学习共享运动表示,并将其迁移到感知任务。PRISM 将每个动作序列表示为基元系数空间中的轨迹,该空间捕捉了一组学习到的原子运动基元如何对观测到的运动做出贡献。一个结构化分解模块通过运动生成以物理接地且视角不变的方式学习该表示。PRISM 不强制跨任务联合或统一训练,而是提供单一的以基元为中心的表示,可通过轻量级任务头顺序迁移到下游分类和逐帧检测。该表示引入了结构、组合性,并改善了在不同监督下的泛化能力。PRISM 在长尾和多标签数据集上持续提升性能,并支持对组合式和稀有动作的可解释推理。大量实验结果表明,结构化基元空间可作为真实世界数据集中多样化动作理解任务的可迁移且稳健的基础。

英文原文摘要 1627 字符

Real-world human action understanding remains challenging due to long-tailed label distributions, compositional motion patterns, and viewpoint variations. Existing skeleton-based methods often lack a structured and transferable representation of motion, and task-specific models for generation, classification, and detection are usually trained independently, resulting in fragmented pipelines and limited cross-task generalization. We present PRISM, a PRImitive-centric Skeleton Modeling framework that learns a shared motion representation from a motion generation objective and transfers it to perception tasks. PRISM represents each action sequence as a trajectory in a primitive coefficient space, which captures how a set of learned atomic motion primitives contribute to the observed motion. A structured decomposition module learns this representation in a physically grounded and view-invariant manner via motion generation. Instead of enforcing joint or unified training across tasks, PRISM provides a single primitive-centric representation that can be sequentially transferred to downstream classification and frame-wise detection through lightweight task heads. This representation introduces structure, compositionality, and improved generalization across distinct supervisions. PRISM consistently improves performance on long-tailed and multi-label datasets and enables interpretable reasoning over compositional and rare actions. Extensive experimental results show that the structured primitive space serves as a transferable and robust foundation for diverse action understanding tasks in real-world datasets.

标签

会议 CVPR2026
页码 6085-6094
Bibkey
来源 CVF Open Access