UniSH：单次前馈中统一场景与人体重建
UniSH: Unifying Scene and Human Reconstruction in a Feed-Forward Pass
提出一种利用无标注真实视频的两阶段训练范式：先在合成数据上学习粗定位，再通过优化 SMPL 网格与人体点云的几何对应关系在真实数据上微调，并蒸馏专家深度模型的高频细节来提升人体表面质量。该思路可直接迁移到其他需要弥合 sim-to-real 差距的 3D 重建任务。
原文依据 a two-stage supervision scheme, which first learns coarse localization on synthetic data, then fine-tunes on real data
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论我们提出 UniSH，一个统一的前馈框架，用于联合进行度量尺度的三维场景与人体重建。该领域的一个关键挑战是大规模带标注真实数据的稀缺，迫使研究者依赖合成数据集。这种依赖引入了显著的仿真到现实域差距，导致泛化能力差、人体几何保真度低以及在真实视频上对齐效果不佳。为解决此问题，我们提出一种创新的训练范式，有效利用无标注的真实世界数据。我们的框架桥接了来自场景重建和人体网格恢复（HMR）的强大且不同的先验，并通过两个核心组件进行训练：（1）一种鲁棒的蒸馏策略，通过从专家深度模型中蒸馏高频细节来细化人体表面细节；（2）一种两阶段监督方案，首先在合成数据上学习粗定位，然后通过直接优化 SMPL 网格与人体点云之间的几何对应关系在真实数据上微调。该方法使我们的前馈模型能够在单次前向传播中联合恢复高保真场景几何、人体点云、相机参数以及连贯的度量尺度 SMPL 人体。大量实验表明，我们的模型在以人为中心的场景重建上达到了最优性能，并在全局人体运动估计上取得了极具竞争力的结果，优于基于优化的框架和仅 HMR 的方法。项目页面：https://murphylmf.github.io/UniSH/。
英文原文摘要 1543 字符
We present UniSH, a unified, feed-forward framework for joint metric-scale 3D scene and human reconstruction. A key challenge in this domain is the scarcity of large-scale, annotated real-world data, forcing a reliance on synthetic datasets. This reliance introduces a significant sim-to-real domain gap, leading to poor generalization, low-fidelity human geometry, and poor alignment on in-the-wild videos. To address this, we propose an innovative training paradigm that effectively leverages unlabeled in-the-wild data. Our framework bridges strong, disparate priors from scene reconstruction and HMR, and is trained with two core components: (1) a robust distillation strategy to refine human surface details by distilling high-frequency details from an expert depth model, and (2) a two-stage supervision scheme, which first learns coarse localization on synthetic data, then fine-tunes on real data by directly optimizing the geometric correspondence between the SMPL mesh and the human point cloud. This approach enables our feed-forward model to jointly recover high-fidelity scene geometry, human point clouds, camera parameters, and coherent, metric-scale SMPL bodies, all in a single forward pass. Extensive experiments demonstrate that our model achieves state-of-the-art performance on human-centric scene reconstruction and delivers highly competitive results on global human motion estimation, comparing favorably against both optimization-based frameworks and HMR-only methods. Project page: https://murphylmf.github.io/UniSH/.