MotionMaster:可泛化的文本驱动运动生成与编辑
MotionMaster: Generalizable Text-Driven Motion Generation and Editing
把预训练多模态大语言模型微调成文本到人体运动(3D 骨骼序列)的生成与编辑模型,并用 FSQ 分词器把运动离散成 token 与语言共享嵌入空间;对做 3D 人体动作/动画的研究者,这是一条可直接复用的「LLM 当运动先验 + 离散运动 token」路线。
原文依据 finetuning a pretrained MLLM with large-scale motion data yields strong zero-shot generalization
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论从自然语言合成逼真的人体运动,对动画、机器人与虚拟现实具有变革性潜力。现有方法能够处理单一动作序列与简单文本指令,但由于数据多样性有限、表示能力不足以及流程碎片化,多动作组合与精确编辑仍难以实现。关键在于,大多数现有方法从零开始训练运动生成模型,未能利用预训练多模态大语言模型(MLLM)中已编码的丰富动作语义与长时程推理能力。本文表明,用大规模运动数据微调预训练 MLLM,可在多种文本引导的运动生成与编辑任务上获得强零样本泛化能力。我们提出 MotionMaster,一个由三个组件构成的统一框架:MotionGB,一个通过时空增强从 400 小时经校验的动作捕捉数据扩展而来的 10000 小时数据集;一个基于 FSQ 的分词器,同时保持局部关节精度与全局轨迹连贯性;以及一个在共享嵌入空间中联合处理运动 token 与语言 token 的微调 MLLM。MotionMaster 在多动作语义一致性上比先前方法提升 41.6%,在身体部位组合上提升 20.8%。这些结果表明,来自 MLLM 的预训练知识能够有效迁移到运动理解,为通向通用运动智能开辟了一条可行路径。
英文原文摘要 1430 字符
Synthesizing realistic human motion from natural language holds transformative potential for animation, robotics, and virtual reality. Recent methods handle single-action sequences and simple textual instructions, yet multi-action compositions and precise editing remain elusive due to limited data diversity, inadequate representations, and fragmented pipelines. Critically, most existing methods train motion generation models from scratch, failing to exploithe rich action semantics and long-horizon reasoning already encoded in pretrained Multimodal Large Language Models (MLLMs). Here we show that finetuning a pretrained MLLM with large-scale motion data yields strong zero-shot generalization across diverse text-guided motion generation and editing tasks. We present MotionMaster, a unified framework built on three components: MotionGB, a 10,000-hour dataset expanded from 400 hours of verified motion capture via spatial-temporal augmentation; an FSQ-based tokenizer that preserves both local joint accuracy and global trajectory coherence; and a finetuned MLLM with motion and language tokens in a shared embedding space. MotionMaster outperforms prior methods by 41.6% in multi-action semantic consistency and 20.8% in body-part composition. These results demonstrate that pretraining knowledge from MLLMs transfers effectively to motion understanding, opening a viable path toward general-purpose motion intelligence.