Mocap-2-to-3：基于2D预训练的多视图提升单目运动恢复
Mocap-2-to-3: Multi-view Lifting for Monocular Motion Recovery with 2D Pretraining
把单目人体运动恢复重构为多视图合成问题，先在大量2D数据上预训练单视图扩散模型、再在3D数据上做多视图微调，这一「2D预训练+多视图几何约束」的范式可直接迁移到其他需要从单目输入恢复度量尺度3D结构的任务。
原文依据 we reformulate 3D motion as a multi-view synthesis process and divide the training into two stages
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论面向真实世界交互的人体运动恢复既需要精确的动作细节，也需要度量尺度的轨迹。从单目输入恢复绝对人体位姿是一种可行方案，但面临两大挑战：(1) 模型依赖受限环境下的3D训练数据，限制了其分布外泛化能力；(2) 从单目观测估计度量尺度位姿本身存在固有困难。本文提出 Mocap-2-to-3，一个与既有 HMR 方法不同的新框架：它从单目输入恢复绝对位姿，并利用丰富的2D数据增强3D运动恢复。为有效利用大规模2D数据集中的动作先验与多样性，我们将3D运动重构为多视图合成过程，并将训练分为两个阶段：首先在大量2D数据上预训练单视图扩散模型，随后在3D数据上进行多视图微调，从而兼顾强先验与几何约束。此外，为恢复绝对位姿，我们提出一种新的人体运动表示，将局部姿态与全局运动的学习解耦，同时编码地面几何先验以加速收敛，从而在物理世界中获得更精确的定位。在野外基准上的实验表明，本方法在相机空间运动真实感与世界坐标系人体定位两方面均优于最优方法，并展现出强泛化能力。
英文原文摘要 1527 字符
Human motion recovery for real-world interaction demands both precise action details and metric-scale trajectories. Recovering absolute human pose from monocular input presents a viable solution, but faces two main challenges: (1) models' reliance on 3D training data from constrained environments limits their out-of-distribution generalization; and (2) the inherent difficulty of estimating metric-scale poses from monocular observations. This paper introduces Mocap-2-to-3, a novel framework that differs from prior HMR methods by recovering absolute poses from monocular input and leveraging abundant 2D data to enhance 3D motion recovery. To effectively utilize the action priors and diversity in large-scale 2D datasets, we reformulate 3D motion as a multi-view synthesis process and divide the training into two stages: a single-view diffusion model is first pre-trained on extensive 2D data, followed by multi-view fine-tuning on 3D data, thus achieving a combination of strong priors and geometric constraints. Furthermore, to recover absolute poses, we introduce a novel human motion representation that decouples the learning of local pose and global movements, while encoding ground geometric priors to accelerate convergence, thereby yielding more precise positioning in the physical world. Experiments on in-the-wild benchmarks show that our method outperforms state-of-the-art approaches in both camera-space motion realism and world-grounded human positioning, while exhibiting strong generalization capability.