Superman：统一骨架与视觉的人体运动感知与生成
Superman: Unifying Skeleton and Vision for Human Motion Perception and Generation
提出视觉引导的运动分词器，利用3D骨架与视觉数据的几何对齐构建跨模态运动词汇表，使单一MLLM统一处理时序3D姿态估计与运动生成任务，为人体运动分析提供可复用的统一建模范式。
原文依据 we propose a Vision-Guided Motion Tokenizer. Leveraging the natural geometric alignment between 3D skeletons and visual data
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论人体运动分析任务，如时序3D姿态估计、运动预测和运动中间帧生成，在计算机视觉中扮演着重要角色。然而，当前范式存在严重的碎片化问题。首先，该领域分裂为“感知”模型和“生成”模型：前者从视频中理解运动但仅输出文本，后者无法从原始视觉输入中进行感知。其次，生成式多模态大语言模型通常局限于使用稠密参数化SMPL模型的单帧静态姿态，无法处理时序运动。第三，现有运动词汇表仅从骨架数据构建，切断了与视觉域的联系。为解决这些挑战，我们提出Superman，一个统一框架，将视觉感知与基于骨架的时序运动生成相桥接。我们的解决方案包含两方面。首先，为克服模态脱节，我们提出视觉引导的运动分词器。利用3D骨架与视觉数据之间的自然几何对齐，该模块开创性地实现从两种模态中进行鲁棒的联合学习，创建统一的跨模态运动词汇表。其次，基于这种运动语言，训练单一的统一多模态大语言模型架构来处理所有任务。该模块灵活处理多样化的时序输入，将从视频进行3D骨架姿态估计（感知）与基于骨架的运动预测和中间帧生成（生成）统一起来。在包括Human3.6M在内的标准基准上的大量实验表明，我们的统一方法在所有运动任务上均达到最优或具有竞争力的性能。这展示了使用骨架进行生成式运动分析的更高效、可扩展的路径。
英文原文摘要 1689 字符
Human motion analysis tasks, such as temporal 3D pose estimation, motion prediction, and motion in-betweening, play an essential role in computer vision. However, current paradigms suffer from severe fragmentation. First, the field is split between "perception" models that understand motion from video but only output text, and "generation" models that cannot perceive from raw visual input. Second, generative MLLMs are often limited to single-frame, static poses using dense, parametric SMPL models, failing to handle temporal motion. Third, existing motion vocabularies are built from skeleton data alone, severing the link to the visual domain. To address these challenges, we introduce Superman, a unified framework that bridges visual perception with temporal, skeleton-based motion generation. Our solution is twofold. First, to overcome the modality disconnect, we propose a Vision-Guided Motion Tokenizer. Leveraging the natural geometric alignment between 3D skeletons and visual data, this module pioneers robust joint learning from both modalities, creating a unified, cross-modal motion vocabulary. Second, grounded in this motion language, a single, unified MLLM architecture is trained to handle all tasks. This module flexibly processes diverse, temporal inputs, unifying 3D skeleton pose estimation from video (perception) with skeleton-based motion prediction and in-betweening (generation). Extensive experiments on standard benchmarks, including Human3.6M, demonstrate that our unified method achieves state-of-the-art or competitive performance across all motion tasks. This showcases a more efficient and scalable path for generative motion analysis using skeletons.