DeX-Portrait：基于显式与隐式运动表征的解耦且富有表现力的肖像动画
DeX-Portrait: Disentangled and Expressive Portrait Animation via Explicit and Latent Motion Representations
把头部姿态建模为显式全局变换、表情建模为隐式潜变量，并通过双分支条件注入扩散模型，为需要解耦控制的人像/视频生成提供了一套可复用的条件注入与引导方案。
原文依据 we represent the pose as an explicit global transformation and the expression as an implicit latent code
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论基于单张源图像和驱动视频的肖像动画是一个长期存在的问题。近期方法倾向于采用基于扩散的图像/视频生成模型来实现逼真且富有表现力的动画。然而，这些扩散模型均未实现头部姿态与面部表情之间的高保真解耦控制，这阻碍了仅表情编辑或仅姿态编辑与动画等应用。为解决这一问题，我们提出 DeX-Portrait，一种能够由解耦的姿态与表情信号驱动、生成富有表现力肖像动画的新方法。具体而言，我们将姿态表示为显式的全局变换，将表情表示为隐式的潜变量。首先，我们设计了一个强大的运动训练器，用于学习姿态编码器与表情编码器，以提取精确且分解的驱动信号。随后，我们提出通过双分支条件机制将姿态变换注入扩散模型，并通过交叉注意力注入表情潜变量。最后，我们设计了渐进式混合无分类器引导，以获得更忠实的身份一致性。实验表明，我们的方法在动画质量与解耦可控性上均优于最优基线。
英文原文摘要 1253 字符
Portrait animation from a single source image and a driving video is a long-standing problem. Recent approaches tend to adopt diffusion-based image/video generation models for realistic and expressive animation. However, none of these diffusion models realizes high-fidelity disentangled control between the head pose and facial expression, hindering applications like expression-only or pose-only editing and animation. To address this, we propose DeX-Portrait, a novel approach capable of generating expressive portrait animation driven by disentangled pose and expression signals. Specifically, we represent the pose as an explicit global transformation and the expression as an implicit latent code. First, we design a powerful motion trainer to learn both pose and expression encoders for extracting precise and decomposed driving signals. Then we propose to inject the pose transformation into the diffusion model through a dual-branch conditioning mechanism, and the expression latent through cross attention. Finally, we design a progressive hybrid classifier-free guidance for more faithful identity consistency. Experiments show that our method outperforms state-of-the-art baselines on both animation quality and disentangled controllability.