面向文本到动作生成的下一尺度自回归模型
Next-Scale Autoregressive Models for Text-to-Motion Generation
把「下一 token 预测」换成「下一尺度预测」的由粗到细自回归框架,可直接迁移到视频/长时序视觉生成中,用跨尺度层级精修与尺度内双向重预测缓解长程结构建模与数据稀缺问题。
原文依据 We introduce MoScale, a next-scale AR framework that generates motion hierarchically from coarse to fine temporal resolutions.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论自回归(AR)模型训练稳定且高效,但标准的下一 token 预测与文本条件动作生成所需的时序结构并不契合。我们提出 MoScale,一种下一尺度自回归框架,从粗到细的时序分辨率分层生成动作。通过在最低尺度提供全局语义并逐步细化,MoScale 建立了更适合长程动作结构的因果层级。为提升在有限文本-动作数据下的鲁棒性,我们进一步引入跨尺度层级精修以改进各尺度的初始预测,以及尺度内时序精修以进行选择性的双向重预测。MoScale 在文本到动作任务上取得 SOTA 性能,同时具有高训练效率,能随模型规模有效扩展,并可零样本泛化到多样的动作生成与编辑任务。代码及补充材料见项目网页。
英文原文摘要 976 字符
Autoregressive (AR) models offer stable and efficient training, but standard next-token prediction is not well aligned with the temporal structure required for text-conditioned motion generation. We introduce MoScale, a next-scale AR framework that generates motion hierarchically from coarse to fine temporal resolutions. By providing global semantics at the coarsest scale and refining them progressively, MoScale establishes a causal hierarchy better suited for long-range motion structure. To improve robustness under limited text-motion data, we further incorporate cross-scale hierarchical refinement for improving per-scale initial predictions and in-scale temporal refinement for selective bidirectional re-prediction. MoScale achieves SOTA text-to-motion performance with high training efficiency, scales effectively with model size, and generalizes zero-shot to diverse motion generation and editing tasks. Code and additional materials are available on the webpage.