通过语言模型连接人脸理解与动画
Bridging Facial Understanding and Animation via Language Models
将3D人脸参数序列当作语言token，用语言模型统一做运动理解与生成，为文本驱动人脸动画提供了可复用的量化运动token+LM建模范式，可迁移到人体动作或手势的文本-运动双向建模。
原文依据 the model synthesizes the corresponding sequence of 3D facial parameters via quantized motion tokens for downstream animation
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论文本引导的人体动画已取得快速进展，但人脸动画因缺乏标注良好、与文本配对的人脸语料而发展滞后。为弥补这一空白，我们利用基础生成模型合成大规模、均衡的人脸行为语料。我们设计覆盖情绪与头部运动的提示词套件，使用多种生成器生成约80小时的人脸视频，并拟合逐帧3D人脸参数，从而得到用于训练的大规模（提示词与参数）配对数据。基于该数据集，我们通过两个互补任务考察语言模型在人脸运动上的双向能力：（1）Motion2Language：给定一段3D人脸参数序列，模型生成捕捉内容、风格与动态的自然语言描述；（2）Language2Motion：给定提示词，模型通过量化运动token合成相应的3D人脸参数序列，用于下游动画。大量实验表明，在此设定下语言模型能够以较强的泛化能力既理解又合成人脸运动。据我们所知，这是首个将人脸参数建模转化为语言问题的工作，为文本条件人脸动画与运动理解建立了统一路径。
英文原文摘要 1283 字符
Text-guided human body animation has advanced rapidly, yet facial animation lags due to the scarcity of well-annotated, text-paired facial corpora. To close this gap, we leverage foundation generative models to synthesize a large, balanced corpus of facial behavior. We design prompts suite covering emotions and head motions, generate about 80 hours of facial videos with multiple generators, and fit per-frame 3D facial parameters, yielding large-scale (prompt and parameter) pairs for training. Building on this dataset, we probe language models for bidirectional competence over facial motion via two complementary tasks: (1) Motion2Language: given a sequence of 3D facial parameters, the model produces natural-language descriptions capturing content, style, and dynamics; and (2) Language2Motion: given a prompt, the model synthesizes the corresponding sequence of 3D facial parameters via quantized motion tokens for downstream animation. Extensive experiments show that in this setting language models can both interpret and synthesize facial motion with strong generalization. To best of our knowledge, this is the first work to cast facial-parameter modeling as a language problem, establishing a unified path for text-conditioned facial animation and motion understanding.