CLEP：对比语言-姿态预训练
CLEP: Contrastive Language-Pose Pretraining
提出层次化姿态编码器与跨尺度注意力融合机制，将关节、肢体、身体多层级特征动态整合，可用于任何需要细粒度人体姿态-语言对齐的视觉-语言任务，如文本驱动的姿态生成与编辑。
原文依据 Its key innovation is a Cross-Scale Attention Fusion (CSAF) mechanism that dynamically integrates features from the joint, limb, and body levels.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论将自然语言描述与精确的3D人体姿态对齐仍然是一个巨大挑战，原因在于缺乏有效的姿态表示机制以及大规模、语义丰富的数据集。为克服这些限制，我们首先引入CLEP-2M，这是迄今为止最大的3D姿态-语言数据集，包含两百万个高质量的3D姿态-语言对。该数据集在规模上提供了20倍的提升，并且比现有基准具有更丰富的语义多样性。其次，我们提出CLEP，一种新颖的对比预训练框架。CLEP的核心是HierFormer，一个专门为语言对齐设计的层次化姿态编码器。其关键创新是跨尺度注意力融合（CSAF）机制，该机制动态整合来自关节、肢体和身体层级的特征。这使得CLEP能够将复杂的多尺度文本描述与姿态表示精确对齐。在CLEP-2M和PoseScript上的大量实验评估表明，我们的方法在一系列下游任务上始终优于现有方法。CLEP展现出卓越的零样本泛化能力，在人工标注的PoseScript-H基准上达到34.8 mRecall——相比基线提升了近6倍。此外，CLEP在姿态生成和细粒度姿态编辑上表现出优越性能。这些结果确立了CLEP作为以人为中心的理解与生成任务的强大 multimodal 基础模型。
英文原文摘要 1479 字符
Aligning natural language descriptions with precise 3D human poses remains a big challenge due to the scarcity of effective pose representation mechanisms and large-scale, semantically rich datasets. To overcome these limitations, we first introduce **CLEP-2M**, the largest 3D pose-language dataset to date, comprising two million high-quality 3D pose-language pairs. This dataset provides a **20-fold** increase in scale and far richer semantic diversity than existing benchmarks. Second, we propose **CLEP**, a novel contrastive pretraining framework. The core of CLEP is HierFormer, a hierarchical pose encoder specifically designed for language alignment. Its key innovation is a Cross-Scale Attention Fusion (CSAF) mechanism that dynamically integrates features from the joint, limb, and body levels. This enables CLEP to precisely align complex, multi-scale text descriptions with the pose representation. Extensive experimental evaluations on CLEP-2M and PoseScript demonstrate that our method consistently outperforms existing approaches across a range of downstream tasks. CLEP shows exceptional zero-shot generalization, achieving a 34.8 mRecall on the human-annotated PoseScript-H benchmark--a nearly **6-fold** improvement from the baseline. Furthermore, CLEP demonstrates superior performance on pose generation and fine-grained pose editing. These results establish CLEP as a strong multimodal foundation model for human-centric understanding and generation tasks.