StreamAvatar:面向实时交互式人体虚拟人的流式扩散模型
StreamAvatar: Streaming Diffusion Models for Real-Time Interactive Human Avatars
把高保真人体视频扩散模型通过自回归蒸馏+对抗精炼改造成因果流式生成器,配合 Reference Sink 与 RAPR 维持长时一致性,为实时交互式数字人提供可复用的少步流式采样方案。
原文依据 we propose a two-stage autoregressive adaptation and acceleration framework that applies autoregressive distillation and adversarial refinement
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论实时、流式的交互式虚拟人是数字人研究中一个关键但具有挑战性的目标。尽管基于扩散模型的人体虚拟人生成方法取得了显著成功,但其非因果架构与高昂计算成本使其不适合流式场景。此外,现有交互式方法通常局限于头肩区域,限制了其产生手势与身体动作的能力。为应对这些挑战,我们提出一个两阶段的自回归适配与加速框架,通过自回归蒸馏与对抗精炼,将高保真人体视频扩散模型改造为实时、交互式流式模型。为确保长期稳定性与一致性,我们引入三个关键组件:参考汇点(Reference Sink)、参考锚定的位置重编码(Reference-Anchored Positional Re-encoding, RAPR)策略,以及一致性感知判别器。基于该框架,我们构建了一个单次、交互式的人体虚拟人模型,能够生成自然的说话与倾听行为,并伴随连贯的手势。大量实验表明,我们的方法取得了最优性能,在生成质量、实时效率与交互自然度上均超越现有方法。
英文原文摘要 1282 字符
Real-time, streaming interactive avatars represent a critical yet challenging goal in digital human research. Although diffusion-based human avatar generation methods achieve remarkable success, their non-causal architecture and high computational costs make them unsuitable for streaming. Moreover, existing interactive approaches are typically restricted to the head-and-shoulder region, limiting their ability to produce gestures and body motions. To address these challenges, we propose a two-stage autoregressive adaptation and acceleration framework that applies autoregressive distillation and adversarial refinement to adapt a high-fidelity human video diffusion model for real-time, interactive streaming. To ensure long-term stability and consistency, we introduce three key components: a Reference Sink, a Reference-Anchored Positional Re-encoding (RAPR) strategy, and a Consistency-Aware Discriminator. Building on this framework, we develop a one-shot, interactive, human avatar model capable of generating both natural talking and listening behaviors with coherent gestures. Extensive experiments demonstrate that our method achieves state-of-the-art performance, surpassing existing approaches in generation quality, real-time efficiency, and interaction naturalness.