ActAvatar：面向说话头像的时序感知精确动作控制
ActAvatar: Temporally-Aware Precise Action Control for Talking Avatars
把提示词拆成全局基块与时序锚定的阶段块，并让浅层偏文本、深层偏音频，这一「分层模态权重」思路可直接迁移到任何音频驱动的视频生成或可控扩散模型中，用来缓解多条件相互干扰。
原文依据 early layers prioritize text for establishing action structure while deeper layers emphasize audio for refining lip movements
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论尽管说话头像生成已取得显著进展，现有方法仍面临关键挑战：对多样化动作的文本遵循能力不足、动作与音频内容之间缺乏时序对齐，以及依赖姿态骨架等额外控制信号。本文提出 ActAvatar，一个通过文本引导实现阶段级精确动作控制的框架，能够同时捕捉动作语义与时序上下文。该方法引入三项核心创新：(1) 阶段感知交叉注意力（PACA），将提示词分解为全局基块与按时间锚定的阶段块，使模型能够聚焦于与当前阶段相关的 token，从而实现精确的时序-语义对齐；(2) 渐进式音频-视觉对齐，使各模态的影响力与分层特征学习过程相匹配——浅层优先利用文本以建立动作结构，深层则强调音频以细化唇部运动，从而避免模态干扰；(3) 两阶段训练策略，首先在多样化数据上建立稳健的音频-视觉对应关系，随后在结构化标注上通过微调注入动作控制，从而同时保持音频-视觉对齐与模型的文本遵循能力。大量实验表明，ActAvatar 在动作控制与视觉质量两方面均显著优于最优方法。
英文原文摘要 1431 字符
Despite significant advances in talking avatar generation, existing methods face critical challenges: insufficient text-following capability for diverse actions, lack of temporal alignment between actions and audio content, and dependency on additional control signals such as pose skeletons. We present ActAvatar, a framework that achieves phase-level precision in action control through textual guidance by capturing both action semantics and temporal context. Our approach introduces three core innovations: (1) Phase-Aware Cross-Attention (PACA), which decomposes prompts into a global base block and temporally-anchored phase blocks, enabling the model to concentrate on phase-relevant tokens for precise temporal-semantic alignment; (2) Progressive Audio-Visual Alignment, which aligns modality influence with the hierarchical feature learning process--early layers prioritize text for establishing action structure while deeper layers emphasize audio for refining lip movements, preventing modality interference; (3) A two-stage training strategy that first establishes robust audio-visual correspondence on diverse data, then injects action control through fine-tuning on structured annotations, maintaining both audio-visual alignment and the model's text-following capabilities. Extensive experiments demonstrate that ActAvatar significantly outperforms state-of-the-art methods in both action control and visual quality.