基于剪枝局部 blendshape 的高保真移动端化身
High-Fidelity Mobile Avatars with Pruned Local Blendshapes
提出局部线性 blendshape 表示，用身体小区域的局部高斯相关性降低姿态相关外观的建模误差，并通过剪枝冗余 blendshape 实现极简表示，为移动端高保真人体化身重建提供了可复用的高效表示方案。
原文依据 we use local linear blendshapes in small body parts to capture global nonlinear changes of Gaussian attributes
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论我们提出一种从多视角视频重建高保真人体化身的方法，该方法可在移动设备上运行。许多工作能够从多视角视频建模高质量的基于高斯分布的全身体化身。然而，这些方法需要大量计算来获得姿态相关的外观，使得在移动设备上部署非常困难。近期的方法从预训练模型蒸馏，并通过将全局姿态特征与 blendshape 线性组合来建模姿态相关的非线性高斯属性。尽管它们可以在移动设备上运行，但会损失一些细节。我们观察到，在身体的局部区域内，邻近的高斯分布通常高度相关，并且可以用更小的误差进行线性建模。因此，我们在身体小部位中使用局部线性 blendshape 来捕捉高斯属性的全局非线性变化。为了进一步减少计算量和模型大小，我们提出移除那些属性变化很小的高斯分布的 blendshape，从而得到一种极简的 blendshape 表示。我们的方法是一种无需预训练模型的端到端训练方法。为了使其能在多种设备上运行，我们使用 WebGPU 实现我们的方法。实验表明，我们的方法能够渲染出细节更优的高质量人体化身，并且在移动设备上以 2K 分辨率达到 120 FPS。
英文原文摘要 1311 字符
We propose a method to reconstruct high-fidelity human avatars from multi-view video that can run on mobile devices. Many works can model high-quality Gaussian-based full-body avatars from multi-view video. However, these methods require heavy computation to obtain pose-dependent appearance, making deployment on mobile devices very difficult. Recent methods distill from pretrained models and model pose-dependent nonlinear Gaussian attributes by linearly combining global pose features with blendshapes. Although they can run on mobile devices, they suffer some loss of detail. We observe that nearby Gaussians are often highly correlated within a local region of the body, and can be linearly modeled with less error. Therefore, we use local linear blendshapes in small body parts to capture global nonlinear changes of Gaussian attributes. To further reduce computation and model size, we propose to remove blendshapes for Gaussians whose attributes change little, yielding a minimal blendshape representation. Our method is an end-to-end training method without a pretrained model. To make it run on multiple devices, we implement our method using WebGPU. Experiments show that our method can render high-quality human avatars with better details, and can reach 120 FPS at 2K resolution on mobile devices.