无需配准的可学习多视图人脸稠密语义对应捕获
Registration-Free Learnable Multi-View Capture of Faces in Dense Semantic Correspondence
提出无需配准训练数据的多视图3D人脸重建框架，利用伪线性逆运动学求解器保证拓扑一致性，并用点图和法向损失替代点到面距离以稳定训练；测试时优化策略可迁移到其他前馈重建任务中提升精度。
原文依据 We overcome this limitation with MOCHI (Multi-view Optimizable Correspondence of Heads from Images), a multi-view 3D face prediction framework trained without requiring registered training data.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近期如 ToFu 和 TEMPEH 等框架通过直接从标定多视图图像预测稠密语义对应的3D网格，为经典配准流程提供了自动化替代方案。然而，这些基于学习的方法依赖其试图取代的缓慢人工配准流程来提供训练监督。我们通过 MOCHI（基于图像的多视图可优化头部对应）克服了这一限制，这是一个无需配准训练数据即可训练的多视图3D人脸预测框架。MOCHI 通过伪线性逆运动学求解器强制拓扑一致性，从而消除了对配准数据的依赖。语义对齐由仅在合成数据上训练的2D关键点预测器提供的稠密关键点引导。我们的分析进一步揭示，在无配准设置下，标准点到面距离会导致训练不稳定和视觉伪影。我们提出基于点图和法向的损失函数，其提供更平滑的梯度和更优的重建保真度。最后，我们引入一种测试时优化方案，在几十次迭代内精炼网络权重。该方法弥合了前馈效率与迭代优化精度之间的差距，使 MOCHI 在重建精度和视觉质量上均超越传统劳动密集型流程。代码和模型公开于：https://filby89.github.io/mochi。
英文原文摘要 1465 字符
Recent frameworks like ToFu and TEMPEH provide an automated alternative to classical registration pipelines by predicting 3D meshes in dense semantic correspondence directly from calibrated multi-view images. However, these learning-based methods rely on the slow, manual registration pipelines they aim to replace for their training supervision. We overcome this limitation with MOCHI (Multi-view Optimizable Correspondence of Heads from Images), a multi-view 3D face prediction framework trained without requiring registered training data. MOCHI eliminates the registration data dependency by enforcing topological consistency through a pseudo-linear inverse kinematic solver. Semantic alignment is guided by dense keypoints from a 2D landmark predictor trained exclusively on synthetic data. Our analysis further reveals that standard point-to-surface distances induce training instabilities and visual artifacts in registration-free settings. We propose pointmap- and normal-based losses instead, which provide smoother gradients and superior reconstruction fidelity. Finally, we introduce a test-time optimization scheme that refines network weights over a few dozen iterations. This approach bridges the gap between feed-forward efficiency and iterative optimization precision, allowing MOCHI to outperform traditional labor-intensive pipelines in both reconstruction accuracy and visual quality. Code and model are public at: https://filby89.github.io/mochi.