从黏土到岩石:面向单目关节手物操作建模的分阶段 3D 高斯泼溅
Clay-to-Stone: Phase-wise 3D Gaussian Splatting for Monocular Articulated Hand-Object Manipulation Modeling
把单目视频中的关节手物交互建模拆成「柔性形变探索 + 刚性结构固化」两阶段,用 3D 高斯泼溅表示并显式估计关节运动参数,可复用于关节物体重建与手物交互追踪中缓解形状-运动耦合歧义。
原文依据 we propose a Clay-to-Stone dual-phase framework, modeling the articulated manipulation at hierarchical granularities
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论从单目视频中理解手物交互对于 AR/VR 与机器人应用中的沉浸式灵巧交互至关重要。然而,现有的单目重建方法大多假设刚性抓取与静态物体几何。当应用于关节式操作时,连续的关节旋转与频繁的部件形变会在形状与运动之间引入强耦合,导致单目观测下的关节优化出现严重歧义与不稳定。为应对这一挑战,我们提出 Clay-to-Stone 双阶段框架,以分层粒度对关节式操作进行建模,从而实现从灵活的语义探索到结构化关节恢复的递进。在 CLAY 阶段,我们的方法在部件间语义关联学习的引导下对几何形变进行细粒度控制。随着语义与运动先验的出现,STONE 阶段施加刚性约束以固化关节结构,并显式估计运动参数。在真实操作数据集上的实验表明,我们的方法从单目视频中取得了最优的重建质量与合理的关节建模。代码见 https://github.com/ru1ven/ARGS。
英文原文摘要 1297 字符
Understanding interacting hand-object from monocular videos is crucial for immersive and dexterous interactions in AR/VR and robotic applications. However, existing monocular reconstruction methods primarily assume rigid grasping and static object geometry. When applied to articulated manipulations, the continuous joint rotations and frequent component deformations introduce a strong coupling between shape and motion, leading to severe ambiguity and instability in articulation optimization under monocular observation. To address this challenge, we propose a Clay-to-Stone dual-phase framework, modeling the articulated manipulation at hierarchical granularities, enabling a progression from flexible semantic exploration to structured articulation recovery. In the CLAY phase, our method performs fine-grained control over geometric deformation, guided by inter-part semantic correlation learning. As semantic and motion priors emerge, the STONE phase enforces rigid constraints to consolidate articulated structures and explicitly estimates motion parameters. Experiments on a real-world manipulation dataset show that our method achieves state-of-the-art reconstruction quality and plausible articulation modeling from monocular videos. Code is available at https://github.com/ru1ven/ARGS.