视觉语言模型 · CVPR2026 ·2026.06.03

UniVerse:以推理与知识赋能统一生成

UniVerse: Empower Unified Generation with Reasoning and Knowledge

Kaiyue Sun、Weiyang Jin、Chengqi Duan、Rongyao Fang、Xian Liu、Yuwei Niu 等 9 人

与你有关的部分

把「隐式提示→推理链→显式提示」的三元组监督引入统一多模态模型的 T2I 训练,为视觉研究者提供了一条用中间推理链提升生成模型组合性与知识性的可复用数据构造与训练范式。

原文依据 explicit training on intermediate reasoning chains is a pivotal step towards more intelligent unified generative models
可迁移到 text-to-image generationmultimodal reasoningvision-language pretraining

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

当前的文本到图像(T2I)生成模型在处理需要复杂推理或专业知识的提示时往往表现不佳,无法准确理解隐含的用户意图。为弥合这一差距,我们提出 T2I-Reason,这是一个大规模数据集,旨在为统一多模态模型(UMM)中的文本到图像生成赋予推理与知识能力。该数据集包含 12 万对文本三元组与图像。文本三元组由以下三部分构成:(1)隐式提示,需要借助推理或知识才能解读其潜在含义;(2)推理链,提供逐步分析以解析隐式提示的含义;(3)显式提示,即为 T2I 生成准备的清晰直白的视觉描述。T2I-Reason 经过精心构建:6.5 万个样本专门用于推理,具体针对算术推理、空间—属性关系推理、演绎推理(由因到果)与溯因推理(由果到因);另有 5.5 万个样本需要专业知识,涵盖多个学科、时空概念与实体知识。为验证数据集的有效性,我们在该数据集上训练了一个统一多模态模型 Bagel。在多个评估 T2I 生成推理能力的基准上的结果表明,我们的模型在组合性与推理两方面均取得显著且一致的提升,证实对中间推理链进行显式训练是迈向更智能的统一生成模型的关键一步。

英文原文摘要 1590 字符

Current text-to-image (T2I) generation models often struggle with prompts that require complex reasoning or specialized knowledge, failing to accurately interpret implicit user intent. To bridge this gap, we introduce T2I-Reason, a large-scale dataset designed to empower text-to-image generation in unified multimodal models (UMMs) with reasoning and knowledge. The dataset contains 120k pairs of text triplet and image. The text triplet consists of (1) an implicit prompt, which requires reasoning or knowledge to decipher its underlying meaning; (2) a reasoning chain, which provides a step-by-step analysis to resolve the implicit prompt's meaning; and (3) an explicit prompt, a clear and straightforward visual description prepared for T2I generation. T2I-Reason is meticulously constructed: 65k samples are dedicated to reasoning, specifically targeting arithmetic reasoning, spatial-attribute relationship reasoning, deductive reasoning (cause to effect), and abductive reasoning (effect to cause). While 55k samples necessitate specialized knowledge, which covers multiple disciplines, spatial-temporal concepts, and entity knowledge. To validate the effectiveness of our dataset, we train a unified multimodal model, Bagel, on our dataset. Results across multiple benchmarks that evaluate the reasoning capabilities of T2I generation demonstrate that our model achieves significant and consistent improvements on both composition and reasoning, confirming that explicit training on intermediate reasoning chains is a pivotal step towards more intelligent unified generative models.

标签

会议 CVPR2026
页码 21997-22006
Bibkey
来源 CVF Open Access