InfinityHuman：迈向长期音频驱动的人体动画
InfinityHuman: Towards Long-Term Audio-Driven Human Animation
提出粗到细框架，利用解耦的姿态序列作为稳定视觉锚点来抑制长视频生成中的误差累积，并引入手部特定奖励机制提升手势真实感，可复用于其他长时序人体动画任务。
原文依据 we propose InfinityHuman, a coarse-to-fine framework that first generates audio-synchronized representations, then progressively refines them into high-resolution, long-duration videos using a pose-guided refiner.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论音频驱动的人体动画因其实际应用而受到广泛关注。然而，在生成具有一致外观和自然手部动作的高分辨率、长时长视频方面仍存在关键挑战。现有方法使用重叠运动帧来扩展视频，但会遭受误差累积，导致身份漂移、颜色偏移和场景不稳定。此外，手部运动建模不佳，导致明显的扭曲和与音频的不对齐。在本工作中，我们提出了InfinityHuman，一个由粗到细的框架，首先生成音频同步的表示，然后使用姿态引导的细化器将其逐步细化为高分辨率、长时长视频。由于姿态序列与外观解耦且能抵抗时序退化，我们的姿态引导细化器采用稳定的姿态和初始帧作为视觉锚点，以减少漂移并改善唇形同步。此外，为了增强语义准确性和手势真实感，我们引入了一种手部特定的奖励机制，并使用高质量的手部运动数据进行训练。在EMTD和HDTF数据集上的实验表明，InfinityHuman在视频质量、身份保持、手部准确性和唇形同步方面达到了最优性能。消融研究进一步证实了每个模块的有效性。我们的项目页面可在https://infinityhuman.github.io/获取。
英文原文摘要 1426 字符
Audio-driven human animation has attracted wide attention thanks to its practical applications. However, critical challenges remain in generating high-resolution, long-duration videos with consistent appearance and natural hand motions. Existing methods extend videos using overlapping motion frames but suffer from error accumulation, leading to identity drift, color shifts, and scene instability. Additionally, hand movements are poorly modeled, resulting in noticeable distortions and misalignment with the audio. In this work, we propose InfinityHuman, a coarse-to-fine framework that first generates audio-synchronized representations, then progressively refines them into high-resolution, long-duration videos using a pose-guided refiner. Since pose sequences are decoupled from appearance and resist temporal degradation, our pose-guided refiner employs stable poses and the initial frame as a visual anchor to reduce drift and improve lip synchronization. Moreover, to enhance semantic accuracy and gesture realism, we introduce a hand-specific reward mechanism trained with high-quality hand motion data. Experiments on the EMTD and HDTF datasets show that InfinityHuman achieves state-of-the-art performance in video quality, identity preservation, hand accuracy, and lip-sync. Ablation studies further confirm the effectiveness of each module. And our project page is available at https://infinityhuman.github.io/.