Token 化使多模态大语言模型能够理解、生成与编辑建筑平面图
Tokenization Allows Multimodal Large Language Models to Understand, Generate and Edit Architectural Floor Plans
把房间实例离散化为 token 并入语言模型词表,使 MLLM 能像生成文本一样自回归地生成与编辑结构化布局;这套「实例级离散 token + 多模态对齐」的做法可迁移到任何需要输出结构化几何(场景图、矢量图元、版面)的视觉生成任务。
原文依据 We introduce discrete room-instance tokens to construct a unified vocabulary that bridges layouts and symbolic reasoning.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论建筑平面图设计需要对几何、语义与空间层级进行联合推理,这对当前的人工智能系统而言仍是一项重大挑战。尽管近期的扩散模型与语言模型提升了视觉保真度,但它们在连贯的空间推理与可控生成方面仍存在困难。本文提出 HouseMind,一个将平面图理解、生成与编辑统一于同一框架的多模态大语言模型。我们引入离散的房间实例 token,构建起连接布局与符号推理的统一词表。借助多模态对齐与指令微调,该模型能够依据文本指令合成连贯且可控的布局。实验表明,该框架在保持高效性与本地可部署性的同时,实现了更优的几何有效性与可控性。
英文原文摘要 840 字符
Architectural floor plan design demands joint reasoning over geometry, semantics, and spatial hierarchy, which remains a major challenge for current AI systems. Although recent diffusion and language models improve visual fidelity, they still struggle with coherent spatial reasoning and controllable generation. We present HouseMind, a multimodal large language model that unifies floor plan understanding, generation, and editing in one framework. We introduce discrete room-instance tokens to construct a unified vocabulary that bridges layouts and symbolic reasoning. With multimodal alignment and instruction tuning, the model synthesizes coherent, controllable layouts from text instructions. Experiments show how the framework achieves superior geometric validity and controllability while remaining efficient and locally deployable.