LaMoGen:通过 LLM 引导的符号推理实现语言到动作生成
LaMoGen: Language to Motion Generation Through LLM-Guided Symbolic Inference
把人体动作离散化为可解释的拉班符号序列,再用 LLM 做符号推理与重组生成动作,为视觉侧的人体动作生成/理解提供了一条可解释、可控制的中间表示思路,可迁移到动作编辑与细粒度时序对齐任务。
原文依据 we introduce LabanLite, a motion representation developed by adapting and extending the Labanotation system
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论人体动作具有高度表现力,并且与语言天然对齐,然而当前主流方法严重依赖联合的文本-动作嵌入,难以合成时序准确、细节丰富的人体动作,且往往缺乏可解释性。为解决这些局限,我们提出 LabanLite,一种通过改造与扩展拉班舞谱(Labanotation)体系而得到的动作表示。与黑箱式的文本-动作嵌入不同,LabanLite 将每个原子级的身体部位动作(例如一次左脚迈步)编码为一个离散的拉班符号,并配以文本模板。这种抽象将复杂动作分解为可解释的符号序列与身体部位指令,在高层语言与低层动作轨迹之间建立起符号化的联系。在 LabanLite 的基础上,我们提出 LaMoGen,一个「文本→LabanLite→动作」的生成框架,使大语言模型(LLM)能够通过符号推理来组合动作序列。LLM 解读动作模式,将其与文本描述相关联,并把符号重新组合为可执行的计划,从而生成既具有可解释性、又以语言为依据的动作。为支持严格的评测,我们提出一个基于拉班舞谱的基准,包含结构化的描述-动作对,以及三项从符号、时序与和谐三个维度共同衡量文本-动作对齐的指标。实验表明,LaMoGen 在可解释性与可控性两方面都建立了新的基线,在我们的基准以及两个公开数据集上优于此前方法。这些结果凸显了符号推理与智能体式设计在语言驱动动作合成中的优势。
英文原文摘要 1670 字符
Human motion is highly expressive and naturally aligned with language, yet prevailing methods relying heavily on joint text-motion embeddings struggle to synthesize temporally accurate, detailed motions and often lack explainability. To address these limitations, we introduce LabanLite, a motion representation developed by adapting and extending the Labanotation system. Unlike black-box text-motion embeddings, LabanLite encodes each atomic body-part action (e.g., a single left-foot step) as a discrete Laban symbol paired with a textual template. This abstraction decomposes complex motions into interpretable symbol sequences and body-part instructions, establishing a symbolic link between high-level language and low-level motion trajectories. Building on LabanLite, we present LaMoGen, a Text-to-LabanLite-to-Motion Generation framework that enables large language models (LLMs) to compose motion sequences through symbolic reasoning. The LLM interprets motion patterns, relates them to textual descriptions, and recombines symbols into executable plans, producing motions that are both interpretable and linguistically grounded. To support rigorous evaluation, we introduce a Labanotation-based benchmark with structured description-motion pairs and three metrics that jointly measure text-motion alignment across symbolic, temporal, and harmony dimensions. Experiments demonstrate that LaMoGen establishes a new baseline for both interpretability and controllability, outperforming prior methods on our benchmark and two public datasets. These results highlight the advantages of symbolic reasoning and agent-based design for language-driven motion synthesis.