AudioAvatar：个性化音频驱动的全身说话头像
AudioAvatar: Personalized Audio-driven Whole-body Talking Avatars
用3D高斯粒子变形场替代参数化姿态瓶颈，实现音频到全身高保真动画；特征级蒸馏从视频扩散模型迁移同步先验，可复用于其他音频驱动生成任务。
原文依据 The avatar is modeled as a particle-based deformation field of 3D Gaussian primitives in a canonical space
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论现有的富有表现力的全身对话头像系统将音频映射到参数化姿态再渲染，造成量化、重定向和跟踪误差累积的有损瓶颈。这降低了音频-动作同步性，并抑制了对真实感至关重要的微表情——如双唇闭合、脸颊鼓起、鼻唇运动、眨眼和精细手势——尤其在单图像个性化下。我们提出一个端到端框架，从单张图像构建全身、照片级真实的对话头像，并直接由音频驱动，绕过中间姿态预测。该头像被建模为规范空间中3D高斯基元的基于粒子的变形场，并配备音频条件动力学模块，输出面部、手部和身体的逐粒子轨迹，实现局部高频控制与全局连贯运动。基于泼溅的可微渲染器保持身份、纹理和照片真实感，同时从大型音频驱动视频扩散模型进行特征级蒸馏，以及来自合成音频条件片段的弱监督，进一步提升同步性和自然表现力。联合光度与时序目标塑造音频条件变形与渲染。跨多样说话人的实验表明，相比姿态驱动基线，唇-音频同步、精细面部细节和对话手势自然度均有提升。
英文原文摘要 1455 字符
Prior expressive whole-body conversational avatar systems map audio to parametric poses and then render, creating a lossy bottleneck where quantization, retargeting, and tracking errors accumulate. This degrades audio-motion synchronization and suppresses micro-articulations critical for realism--such as bilabial closures, cheek inflation, nasolabial motion, blinks, and fine hand gestures--especially under single-image personalization. We propose an end-to-end framework that builds a full-body, photorealistic conversational avatar from a single image and drives it directly from audio, bypassing intermediate pose prediction. The avatar is modeled as a particle-based deformation field of 3D Gaussian primitives in a canonical space, with an audio-conditioned dynamics module that outputs per-particle trajectories for face, hands, and body, enabling localized high-frequency control with globally coherent motion. A splat-based differentiable renderer preserves identity, texture, and photo realism, while feature-level distillation from a large audio-driven video diffusion model and weak supervision from synthetic audio-conditioned clips further improve synchronization and natural expressivity. Joint photometric and temporal objectives shape the audio-conditioned deformation and rendering. Experiments across diverse speakers show improved lip-audio sync, fine facial detail, and conversational gesture naturalness over pose-driven baselines.