TeHOR:文本引导的带纹理三维人体与物体重建
TeHOR: Text-Guided 3D Human and Object Reconstruction with Textures
用文本描述替代/补充接触先验来约束人-物联合三维重建,可复用于非接触交互场景(指向、注视)的重建与生成;其将外观线索纳入对齐过程的做法,对单图人-物重建中的视觉合理性约束有直接借鉴价值。
原文依据 leverages text descriptions of human-object interactions to enforce semantic alignment between the 3D reconstruction and its textual cues
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论从单张图像联合重建三维人体与物体是一个活跃的研究方向,在机器人与数字内容创作中具有关键应用。尽管近期取得了进展,现有方法仍存在两个根本性局限。第一,其重建严重依赖物理接触信息,而这类信息本质上无法刻画非接触式的人-物交互,例如注视或指向某个物体。第二,重建过程主要由局部几何邻近性驱动,忽略了人体与物体的外观,而外观提供了理解整体交互所必需的全局上下文。为解决这些问题,我们提出 TeHOR,一个基于两项核心设计的框架。第一,除接触信息之外,该框架利用人-物交互的文本描述,强制三维重建与其文本线索之间的语义对齐,从而能够对更广泛的交互类型(包括非接触情形)进行推理。第二,我们将三维人体与物体的外观线索纳入对齐过程,以捕获整体上下文信息,从而确保重建在视觉上合理。由此,该框架能够产生准确且语义一致的重建,并达到最优性能。
英文原文摘要 1324 字符
Joint reconstruction of 3D human and object from a single image is an active research area, with pivotal applications in robotics and digital content creation. Despite recent advances, existing approaches suffer from two fundamental limitations. First, their reconstructions rely heavily on physical contact information, which inherently cannot capture non-contact human-object interactions, such as gazing at or pointing toward an object. Second, the reconstruction process is primarily driven by local geometric proximity, neglecting the human and object appearances that provide global context crucial for understanding holistic interactions. To address these issues, we introduce TeHOR, a framework built upon two core designs. First, beyond contact information, our framework leverages text descriptions of human-object interactions to enforce semantic alignment between the 3D reconstruction and its textual cues, enabling reasoning over a wider spectrum of interactions, including non-contact cases. Second, we incorporate appearance cues of the 3D human and object into the alignment process to capture holistic contextual information, thereby ensuring visually plausible reconstructions. As a result, our framework produces accurate and semantically coherent reconstructions, achieving state-of-the-art performance.