OmniGen2：迈向指令对齐的多模态生成
OmniGen2: Towards Instruction-Aligned Multimodal Generation
两阶段范式（先构建世界知识基础模型，再用渐进式多任务强化学习对齐）可直接迁移到任何需要细粒度指令遵循的视觉生成任务；解耦解码架构与新型位置编码方案为统一多模态生成器的效率设计提供参考。
原文依据 a two-stage design that first builds a strong, world-knowledge-grounded foundation model and then aligns it using a progressive, multi-task instruction tuning strategy
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多模态生成模型能够处理多种模态的指令，并在广泛的图像生成任务上展现出优异性能。然而，由于泛化性指令对齐不足，其在复杂真实场景中的鲁棒性仍然有限。我们提出 OmniGen2，一个旨在遵循复杂、细粒度指令的统一多模态生成器。我们的核心贡献是一个两阶段设计：首先构建一个强大的、以世界知识为基础的基础模型，然后采用渐进式多任务指令微调策略对其进行对齐。该基础模型采用精简架构，具备解耦解码能力以实现多样化的多模态生成，并提出一种新颖的位置编码方案以提升学习效率。我们通过大规模数据构建流程，使该模型建立在真实世界知识之上。在此基础之上，我们提出一种渐进式、基于强化的对齐过程。该阶段精心安排训练任务与奖励信号，以促进跨任务知识迁移，显著提升模型的指令遵循能力。我们的模型在标准基准以及我们专门构建的上下文生成基准 OmniContext 上均展现出具有竞争力的性能。我们已在 https://github.com/VectorSpaceLab/OmniGen2 发布模型、代码、基准与训练数据集。
英文原文摘要 1395 字符
Multimodal generative models can process instructions in various modalities and demonstrate outstanding performance across a wide range of image generation tasks. However, their robustness in complex real-world scenarios remains limited due to insufficient generalized instruction alignment. We introduce OmniGen2, a unified multimodal generator designed to follow complex, fine-grained instructions. Our core contribution is a two-stage design that first builds a strong, world-knowledge-grounded foundation model and then aligns it using a progressive, multi-task instruction tuning strategy. The foundation model features a streamlined architecture with decoupled decoding for versatile multimodal generation and a novel positional encoding scheme to improve learning efficiency. We ground this model in real-world knowledge using large-scale data construction pipelines. Building on this foundation, we propose a progressive, reinforcement-based alignment process. This phase carefully schedules training tasks and reward signals to foster cross-task knowledge transfer, significantly improving the model's instruction-following capabilities. Our models demonstrate competitive performance on standard benchmarks and our dedicated in-context generation benchmark, OmniContext. We have released our models, code, benchmark, and training datasets at https://github.com/VectorSpaceLab/OmniGen2.