MLLMSplat：一个由2D多模态大语言模型驱动的3D高斯泼溅理解、生成与编辑框架
MLLMSplat: A 2D MLLM-Powered Framework for 3D Gaussian Splatting Understanding, Generation, and Editing
将2D多模态大语言模型（MLLM）通过3DGS分词器和去分词器适配到3D高斯泼溅场景，实现无需训练的3DGS理解、高质量生成和高效编辑，为3D场景的跨模态处理提供了新范式。
原文依据 a 3DGS tokenizer that can be seamlessly integrated into MLLMs in a training-free manner
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论3D高斯泼溅（3DGS）已成为3D场景的主流表示，其理解、生成与编辑日益受到研究关注。然而，现有研究仍局限于低层次感知、低质量生成和低效率编辑，在多模态大语言模型（MLLM）时代远远落后于图像领域的对应工作。为弥合这一差距，我们提出MLLMSplat，一种新颖的框架，通过适配2D MLLM来实现对3DGS场景的高层次理解、高质量生成和高效率编辑。具体而言，我们的综合框架包含三个核心设计：（1）一个3DGS分词器，能够以无需训练的方式无缝集成到MLLM中；（2）一个3DGS去分词器，利用双旋转位置编码空间非侵入式地扩展MLLM中的2D潜变量扩散模型，同时为其增强一个联合训练和采样的3DGS解码器；（3）一个代理任务，用于增强前馈编辑能力。大量实验表明，MLLMSplat在3DGS理解、生成和编辑方面均达到了最优性能。
英文原文摘要 1164 字符
3D Gaussian Splatting (3DGS) has emerged as a mainstream representation for 3D scenes, drawing increasing research attention to its understanding, generation, and editing. However, existing studies remain limited to low-level perception, low-quality generation, and low-efficiency editing, lagging far behind their image counterparts in the era of Multimodal Large Language Models (MLLMs). To bridge this gap, we propose MLLMSplat, a novel framework that adapts 2D MLLMs to achieve high-level understanding, high-quality generation, and high-efficiency editing of 3DGS scenes. Specifically, our comprehensive framework consists of three core designs: (1) a 3DGS tokenizer that can be seamlessly integrated into MLLMs in a training-free manner; (2) a 3DGS de-tokenizer that non-intrusively extends the 2D latent diffusion model in MLLMs using a dual rotary positional encoding space, while augmenting it with a jointly trained and sampled 3DGS decoder; and (3) a surrogate task that enhances feed-forward editing capabilities. Extensive experiments demonstrate that MLLMSplat delivers state-of-the-art performance across 3DGS understanding, generation, and editing.