多层次因果大语言模型驱动的文本到动作生成与人类对齐
Multi-level Causal LLM-based Text-to-Motion Generation with Human Alignment
把动作序列当作离散词元、用因果 RVQ-VAE 加时间滞后因果预测来做多层级并行生成，这套「离散词元 + 因果 LLM + 偏好对齐」的组合可直接迁移到视频/动作序列的生成式建模；同时发布的 HumanML3D-R 偏好数据集为动作生成提供了可复用的对齐评测资源。
原文依据 MoTiGA employs Causal RVQ-VAE for multi-level causal fine-grained motion representation
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论尽管基于大语言模型的文本驱动动作生成已取得进展，但仍存在生成细粒度且语义一致动作方面的局限。这些局限源于：1）细粒度动作量化误差；2）因果推理语言与非因果动作表示之间的不匹配；3）缺乏人类偏好对齐。为解决这些问题，本文提出 MoTiGA，一个具有人类对齐的多层次因果大语言模型文本到动作生成框架。首先，MoTiGA 采用因果 RVQ-VAE 进行多层次因果细粒度动作表示，随后探索迭代残差量化与因果卷积以减少细粒度动作量化误差，同时保持与语言表示一致的因果性。此外，该框架引入时间滞后的因果预测策略，在保持时序依赖的同时实现跨动作词元层级的并行预测。最后，为增强人类对齐，我们提出多层次混合加权偏好优化（MHPO），动态调整语义相似度权重与连续相似度分数。针对 MHPO，我们还发布了 HumanML3D-R 数据集，这是首个面向动作生成的大规模偏好数据集，包含 101,490 个人类偏好对。评估表明 MoTiGA 性能优越，在 HumanML3D 上弗雷歇初始距离相比其他基于大语言模型的方法提升 82.3%，在 KIT-ML 上提升 64.7%。
英文原文摘要 1462 字符
Although progress has been made in LLM-based text-driven motion generation, it still has the limitations of generating fine-grained and semantically consistent motions. These limitations stem from: 1) fine-grained motion quantization errors; 2) mismatches between causal reasoning language and non-causal motion representation; and 3) lack of human preference alignment. To solve them, this paper proposes MoTiGA, a multi-level causal LLM-based text-to-motion generation framework with human alignment. Firstly, MoTiGA employs Causal RVQ-VAE for multi-level causal fine-grained motion representation, then explores iterative residual quantization and causal convolutions to reduce fine-grained motion quantization errors, while preserving the causality as language presentation. Furthermore, the framework incorporates a time-lagged causal prediction strategy, enabling parallel prediction across motion token levels while maintaining temporal dependencies. Finally, to enhance human alignment, we propose Multi-level Hybrid-weighted Preference Optimization (MHPO), which dynamically adjusts semantic similarity weighting and continuous similarity scores. For MHPO, we also release the HumanML3D-R dataset, the first large-scale preference dataset for motion generation, with 101,490 human preference pairs. Evaluations show MoTiGA's superior performance, with an 82.3% FID improvement on HumanML3D and a 64.7% improvement on KIT-ML over other LLM-based methods.