CrossHOI：面向单目三维人体-物体交互重建的跨视角表征学习
CrossHOI: Learning Cross-View Representations for Monocular 3D Human-Object Interaction Reconstruction
用单目输入合成新视角特征作为先验，再以双向跨视角 Transformer 融合多视角顶点特征来估计接触并精修网格——这套「无额外输入的新视角特征先验 + 顶点级跨视角融合」思路可直接迁移到遮挡严重的人体/物体/手部重建与接触建模任务。
原文依据 we for the first time, consider to introduce novel-view feature priors to enhance monocular 3D HOI reconstruction
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论从单目图像重建三维人体-物体交互（HOI）极具挑战性，尤其当人体与物体相互遮挡时。现有方法主要依赖单视角输入，这从根本上限制了其恢复被遮挡区域与准确估计接触区域的能力。为应对这些挑战，我们首次考虑引入新视角特征先验来增强单目三维 HOI 重建。我们首先设计了一个跨视角生成器，它学习从单视角输入推断新视角图像特征，在特征层面丰富空间几何信息，且推理时无需额外输入。在真实视角特征与生成视角特征的共同引导下，一个空间跨视角特征融合模块自适应地聚合互补线索，以增强人体与物体网格的初始重建。在该重建结果的基础上，我们从两个视角采样三维顶点特征，并引入双向跨视角 Transformer 来整合多视角顶点表征，从而实现准确的接触估计。最后，利用预测的接触图对人体与物体网格进行精修，得到几何一致且物理合理的重建结果。在 BEHAVE 与 InterCap 上的实验表明，我们提出的 CrossHOI 在重建精度与接触预测两方面均超越最优方法，在严重遮挡情形下尤为显著。
英文原文摘要 1422 字符
Reconstructing 3D human-object interaction (HOI) from monocular images is highly challenging especially when human and object are mutually occluded. Existing methods primarily rely on single-view inputs, which fundamentally limit their ability to recover occluded regions and accurately estimate contact areas. To address these challenges, we for the first time, consider to introduce novel-view feature priors to enhance monocular 3D HOI reconstruction. We first design a cross-view generator that learns to infer novel-view image features from a single-view input, enriching spatial geometry at the feature level without requiring extra inputs during inference. Guided by both real and generated view features, a spatial cross-view feature fusion module adaptively aggregates complementary cues to enhance the initial reconstruction of human and object meshes. Built upon this reconstruction, we sample 3D vertex features from both views and introduce a bidirectional cross-view Transformer to integrate multi-view vertex representations for accurate contact estimation. Finally, the predicted contact maps are leveraged to refine human-object meshes, yielding geometrically consistent and physically plausible reconstructions.Experiments on BEHAVE and InterCap show that our proposed CrossHOI surpasses state-of-the-art methods in both reconstruction accuracy and contact prediction, especially under severe occlusions.