先思考再生成:面向一致性3D生成的结构化思维链推理
Think-Then-Generate: Structural Chain-of-Thought Reasoning for Consistent 3D Generation
把思维链式推理引入文本到3D生成:先用文本语义解析做结构规划,再用多视图渲染结果评估潜变量不一致并多轮迭代修正,可直接迁移到任何基于2D扩散先验的3D生成管线以抑制多视图幻觉。
原文依据 dynamically evaluates latent inconsistencies by analyzing multiple renderings, employing a multi-round iterative refinement mechanism
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近期,利用预训练扩散模型的视觉先验生成3D资产已取得显著成果。然而,由于2D扩散模型本身缺乏3D几何先验,合成结果常常出现空间幻觉与多视图不一致的问题。为解决这一局限,我们提出 Thoughtful3D,一种通过引入结构化思维链(CoT)推理来缓解不一致问题、抑制幻觉,从而提升3D内容生成质量的新框架。具体而言,我们设计了一种双阶段结构化思维链策略:(1)3DBlueprint-CoT 在初始化阶段通过文本语义解析与逻辑推演,显式地规划3D生成过程;(2)3DRefine-CoT 通过分析多幅渲染图像动态评估潜变量中的不一致性,采用多轮迭代精化机制抑制幻觉并增强跨视图一致性。为进一步提升各视图之间的一致性,我们提出跨视图语义外观对齐策略,通过在不同视角的相同特征之间建立动态几何关联来增强多视图一致性。大量实验表明,Thoughtful3D 显著提升了所生成3D资产的质量与一致性。
英文原文摘要 1316 字符
Recently, generating 3D assets using visual priors from pretrained diffusion models has shown remarkable results. However, due to the inherent lack of 3D geometric priors in 2D diffusion, the synthesized results often suffer from spatial hallucination and multi-view inconsistency. To address this limitation, we propose Thoughtful3D, a novel framework that enhances 3D content generation quality by introducing structural chain-of-thought (CoT) reasoning to alleviate inconsistent issues and mitigate hallucinations. Specifically, we design a dual-phase structural CoT strategy: (1) 3DBlueprint-CoT explicitly plans the 3D generation process through textual semantic parsing and logical deduction during the initialization phase. (2) 3DRefine-CoT dynamically evaluates latent inconsistencies by analyzing multiple renderings, employing a multi-round iterative refinement mechanism to suppress hallucinations and enhance cross-view consistency. To further promote consistency across views, we propose a Cross-view Semantic Appearance Alignment strategy that enhances multi-view consistency by establishing dynamic geometric associations between the same features from different viewpoints. Extensive experiments demonstrate that Thoughtful3D significantly improves the quality and consistency of generated 3D assets.