ArtHOI：驯服基础模型实现手-关节物体交互的单目4D重建
ArtHOI: Taming Foundation Models for Monocular 4D Reconstruction of Hand-Articulated-Object Interactions
提出从单目RGB视频重建手-关节物体4D交互的优化框架，利用自适应采样精化物体尺度与位姿，并用MLLM接触推理约束手-物体网格对齐，为单目动态场景重建提供可复用思路。
原文依据 It remains an unexplored but significant challenge to reconstruct 4D human-articulated-object interactions from a single monocular RGB video.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论现有的手-物体交互（HOI）方法大多局限于刚性物体，而关节物体的4D重建方法通常需要预扫描物体甚至多视角视频。从单目RGB视频重建4D人-关节物体交互仍是一个尚未探索但意义重大的挑战。幸运的是，近期基础模型的进展为解决这一高度不适定问题提供了新机遇。为此，我们提出ArtHOI，一个基于优化的框架，集成并精化来自多个基础模型的先验。我们的关键贡献是一套新颖的方法论，旨在解决这些先验固有的不准确性和物理不真实性。具体而言，我们引入自适应采样精化（ASR）方法来优化物体的度量尺度和位姿，以将其归一化网格定位在世界空间中。此外，我们提出一种多模态大语言模型（MLLM）引导的手-物体对齐方法，利用接触推理信息作为手-物体网格组合优化的约束。为便于全面评估，我们还贡献了两个新数据集：ArtHOI-RGBD和ArtHOI-Wild。大量实验验证了ArtHOI在多样物体和交互下的鲁棒性和有效性。项目页面：https://arthoi-reconstruction.github.io。
英文原文摘要 1394 字符
Existing hand-object interactions (HOI) methods are largely limited to rigid objects, while 4D reconstruction methods of articulated objects generally require pre-scanning the object or even multi-view videos. It remains an unexplored but significant challenge to reconstruct 4D human-articulated-object interactions from a single monocular RGB video. Fortunately, recent advancements in foundation models present a new opportunity to address this highly ill-posed problem. To this end, we introduce ArtHOI, an optimization-based framework that integrates and refines priors from multiple foundation models. Our key contribution is a suite of novel methodologies designed to resolve the inherent inaccuracies and physical unreality of these priors. In particular, we introduce an Adaptive Sampling Refinement (ASR) method to optimize object's metric scale and pose for grounding its normalized mesh in world space. Furthermore, we propose a Multimodal Large Language Model (MLLM) guided hand-object alignment method, utilizing contact reasoning information as constraints of hand-object mesh composition optimization. To facilitate a comprehensive evaluation, we also contribute two new datasets, ArtHOI-RGBD and ArtHOI-Wild. Extensive experiments validate the robustness and effectiveness of our ArtHOI across diverse objects and interactions. Project: https://arthoi-reconstruction.github.io.