ReGenHOI:统一重建与生成以理解三维人体-物体交互
ReGenHOI: Unifying Reconstruction and Generation for 3D Human-Object Interaction Understanding
把重建与生成统一到共享的语义-几何潜变量空间,并用改造的 Gravity-Field 扩散桥精修接触几何,可直接借鉴到 3D 人体-物体交互、接触区域预测与物理可信运动生成等任务。
原文依据 we modify and adapt the Gravity-Field Based Diffusion Bridge to refine fine-grained contact geometry
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论理解三维人体-物体交互(HOI)涉及两种高度相关的能力:重建,即感知观测到的几何结构;生成,即想象合理的未来交互。然而,现有方法大多将这两种能力视为彼此独立的任务,限制了其刻画人类空间推理统一本质的能力。为此,我们提出一个统一框架,通过共享的语义-几何推理空间将重建与生成连接起来。具体而言,三维接触推理机制支持直接在三维空间中进行推理,联合建模几何结构与语义关系;推理轨迹精修模块则通过融合几何与语义线索,迭代地精修接触预测。该框架通过对人体-物体接触区域的显式推理,构建统一的潜变量表示。为进一步提升重建与生成输出在真实感与物理合理性方面的表现,我们修改并适配了基于重力场的扩散桥,以精修细粒度接触几何,并确保人体-物体接触平滑且物理一致。大量实验表明,该统一框架显著提升了重建精度与生成交互质量,为三维 HOI 理解建立了一个连贯且可解释的范式。代码将发布于 https://github.com/xumiao66/ReGenHOI。
英文原文摘要 1457 字符
Understanding 3D human-object interaction (HOI) involves two highly-related abilities: reconstruction, which perceives observed geometry, and generation, which imagines plausible future interactions. However, most existing methods treat these abilities as separate tasks, limiting their capacity to capture the unified nature of human spatial reasoning. To address this, we propose a unified framework that bridges reconstruction and generation through a shared semantic-geometric reasoning space. Specifically, a 3D Contact Reasoning mechanism enables direct reasoning in 3D space, jointly modeling geometric structure and semantic relationships, while a Reasoning Trace Refinement module iteratively refines contact predictions by integrating geometric and semantic cues. The framework builds a unified latent representation via explicit reasoning on human-object contact regions. To further enhance realism and physical plausibility when generating the outputs of reconstruction and generation, we modify and adapt the Gravity-Field Based Diffusion Bridge to refine fine-grained contact geometry and ensure smooth, physically consistent human-object engagement. Extensive experiments demonstrate that our unified framework significantly improves both reconstruction accuracy and generative interaction quality, establishing a cohesive and interpretable paradigm for 3D HOI understanding. The code will be released at https://github.com/xumiao66/ReGenHOI.