BrickNet:图结构驱动的生成式积木装配
BrickNet: Graph-Backed Generative Brick Assembly
用图结构(连通性关系)替代直接回归 3D 位姿来参数化积木装配序列,这一「以关系为中心的结构表示」思路可迁移到需要满足物理约束的 3D 组合式生成与场景布局任务中,避免自回归长序列快速失效。
原文依据 we design a graph-based program representation that parametrizes structure through connectivity
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论我们训练一个语言模型来生成乐高积木的搭建序列。此前的工作局限于离散的、类体素的塔状结构,而我们考虑了更广泛的零件集合,涵盖数千种具有多样连接语义的零件类型。为实现这一点,我们首先收集了一个大规模数据集,包含超过 10 万个人工设计的 LDraw 积木物体与场景。我们设定的复杂性使得自回归地装配出满足物理约束的结构颇具挑战。当直接预测积木位姿时,搭建序列在少数几步之后便迅速失效。尽管零件被放置在三维空间中,但定义整体的是零件之间的空间关系。基于这一考虑,我们设计了一种基于图的程序表示,通过连通性来参数化结构,从而提升生成序列的物理合理性。为支持未来的应用,我们将数据集与模型开放供研究使用。https://kulits.github.io/BrickNet
英文原文摘要 1013 字符
We train a language model to generate LEGO(r)-brick build sequences. While prior work has been restricted to discrete, voxel-like towers, we consider a much broader set of pieces, encompassing thousands of part types with diverse connection semantics. To enable this, we first collect a large-scale dataset of over 100,000 human-designed LDraw brick objects and scenes. The complexity of our setting makes it challenging to autoregressively assemble structures that satisfy physical constraints. When predicting block pose directly, build sequences quickly become invalid after a small number of steps. Although pieces are placed in 3D space, it is the spatial relationships of the parts which define the whole. With this in mind, we design a graph-based program representation that parametrizes structure through connectivity, improving the physical grounding of generated sequences. To enable future applications, we make our dataset and models available for research purposes. https://kulits.github.io/BrickNet