LiveGesture：可流式生成的语音协同手势生成模型
LiveGesture: Streamable Co-Speech Gesture Generation Model
把全身动作按身体区域拆成因果离散词元、再用区域专家自回归模型加时空融合来建模，这套「流式分词器 + 分层自回归」的架构可直接迁移到其他需要零前瞻、任意长度输出的动作/姿态序列生成任务（如虚拟人驱动、动作预测）。
原文依据 the first fully streamable, speech-driven full-body gesture generation framework that operates with zero look-ahead
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论我们提出 LiveGesture，这是首个完全可流式运行的、由语音驱动的全身手势生成框架，它零前瞻运行，并支持任意序列长度。现有的语音协同手势方法面向离线生成设计，要么独立处理各身体区域，要么在单一模型中纠缠所有关节；与之不同，LiveGesture 从一开始就为因果的、区域协同的动作生成而构建。LiveGesture 由两个主要模块组成：可流式向量量化动作分词器（SVQ）与分层自回归 Transformer（HAR）。SVQ 分词器将每个身体区域的动作序列转换为因果的离散动作词元，从而实现实时、可流式的词元解码。在 SVQ 之上，HAR 采用区域专家自回归（xAR）Transformer 为每个身体区域建模富有表现力、细粒度的动作动态。随后，一个因果时空融合模块（xAR-Fusion）捕获并整合跨区域相关的动作动态。xAR 与 xAR-Fusion 均以由可流式因果音频编码器编码的、实时持续到达的音频信号为条件。为增强在流式噪声与预测误差下的鲁棒性，我们引入自回归掩码训练，利用不确定性引导的词元掩码与随机区域掩码，使模型在训练中接触到不完美的、部分错误的历史。在 BEAT2 数据集上的实验表明，LiveGesture 能实时生成连贯、多样且与节拍同步的全身手势，在真正的零前瞻条件下达到或超越最优的离线方法。
英文原文摘要 1657 字符
We propose LiveGesture, the first fully streamable, speech-driven full-body gesture generation framework that operates with zero look-ahead and supports arbitrary sequence length. Unlike existing co-speech gesture methods--which are designed for offline generation and either treat body regions independently or entangle all joints within a single model--LiveGesture is built from the ground up for causal, region-coordinated motion generation. LiveGesture consists of two main modules: the Streamable Vector-Quantized Motion Tokenizer (SVQ) and the Hierarchical Autoregressive Transformer (HAR). The SVQ tokenizer converts the motion sequence of each body region into causal, discrete motion tokens, enabling real-time, streamable token decoding. On top of SVQ, HAR employs region-eXpert autoregressive (xAR) transformers to model expressive, fine-grained motion dynamics for each body region. A causal spatio-temporal fusion module (xAR-Fusion) then captures and integrates correlated motion dynamics across regions. Both xAR and xAR-Fusion are conditioned on live, continuously arriving audio signals encoded by a streamable causal audio encoder. To enhance robustness under streaming noise and prediction errors, we introduce autoregressive masking training, which leverages uncertainty-guided token masking and random region masking to expose the model to imperfect, partially erroneous histories during training. Experiments on the BEAT2 dataset demonstrate that LiveGesture produces coherent, diverse, and beat-synchronous full-body gestures in real time, matching or surpassing state-of-the-art offline methods under true zero-look-ahead conditions.