3D 视觉与重建 · CVPR2026 ·2026.06.03

ArtLLM:通过 3D 大语言模型生成铰接资产

ArtLLM: Generating Articulated Assets via 3D LLM

Penghao Wang、Siyuan Xie、Hongyu Yan、Xianghui Yang、Jingwei Huang、Chunchao Guo 等 7 人

与你有关的部分

将 3D 多模态大语言模型用于从点云自回归预测铰接物体的部件与关节布局,再以该布局为条件驱动 3D 生成模型合成部件几何,为铰接资产生成提供了统一布局-几何的两阶段范式。

原文依据 ArtLLM autoregressively predicts a variable number of parts and joints, inferring their kinematic structure in a unified manner from the object's point cloud.
可迁移到 铰接物体生成3D 部件分割关节预测点云条件生成

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

为游戏、机器人学和仿真创建交互式数字环境依赖于铰接 3D 物体,其功能源于部件的几何形状和运动学结构。然而,现有方法存在根本性局限:基于优化的重建方法需要缓慢的逐物体关节拟合,且通常只能处理简单的单关节物体;而基于检索的方法从固定库中组装部件,导致几何重复且泛化能力差。为解决这些挑战,我们提出 ArtLLM,一种直接从完整 3D 网格生成高质量铰接资产的新框架。其核心是一个 3D 多模态大语言模型,在从现有铰接数据集和程序化生成物体中整理的大规模铰接数据集上训练。与先前工作不同,ArtLLM 自回归地预测可变数量的部件和关节,从物体的点云中以统一方式推断其运动学结构。这种感知铰接的布局随后作为条件,驱动 3D 生成模型合成高保真部件几何。在 PartNet-Mobility 数据集上的实验表明,ArtLLM 在部件布局准确性和关节预测方面均显著优于最优方法,同时对真实世界物体具有稳健的泛化能力。最后,我们展示了其在构建数字孪生中的效用,突显了其用于可扩展机器人学习的潜力。

英文原文摘要 1469 字符

Creating interactive digital environments for gaming, robotics, and simulation relies on articulated 3D objects whose functionality emerges from their part geometry and kinematic structure. However, existing approaches remain fundamentally limited: optimization-based reconstruction methods require slow, per-object joint fitting and typically handle only simple, single-joint objects, while retrieval-based methods assemble parts from a fixed library, leading to repetitive geometry and poor generalization. To address these challenges, we introduce ArtLLM, a novel framework for generating high-quality articulated assets directly from complete 3D meshes. At its core is a 3D multimodal large language model trained on a large-scale articulation dataset curated from both existing articulation datasets and procedurally generated objects. Unlike prior work, ArtLLM autoregressively predicts a variable number of parts and joints, inferring their kinematic structure in a unified manner from the object's point cloud. This articulation-aware layout then conditions a 3D generative model to synthesize high-fidelity part geometries. Experiments on the PartNet-Mobility dataset show that ArtLLM significantly outperforms state-of-the-art methods in both part layout accuracy and joint prediction, while generalizing robustly to real-world objects. Finally, we demonstrate its utility in constructing digital twins, highlighting its potential for scalable robot learning.

标签

会议 CVPR2026
页码 34281-34291
Bibkey
来源 CVF Open Access