OneCAT：面向统一理解与生成的纯解码器自回归模型
OneCAT: Decoder-Only Auto-Regressive Model for Unified Understanding and Generation
用模态专属 MoE 加统一自回归目标，在纯 decoder-only Transformer 内同时做理解、生成与编辑，推理时不再依赖 ViT 和视觉分词器，对高分辨率输入输出尤其省算力；scale-aware adapter 把多尺度视觉自回归塞进 LLM，给做统一多模态生成的人一条低延迟、可原生支持动态分辨率的替代路线。
原文依据 eliminates the need for external components such as Vision Transformers (ViT) or vision tokenizer during inference
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论我们提出 OneCAT，一个统一多模态模型，在单一 decoder-only Transformer 架构内无缝整合理解、生成与编辑。OneCAT 独特地消除了推理时对视觉 Transformer（ViT）或视觉分词器等外部组件的依赖，从而带来显著的效率提升，尤其在高分辨率图像输入与输出场景下。这一目标通过模态专属的混合专家（MoE）设计实现，并以统一的自回归（AR）目标训练，同时原生支持动态分辨率。此外，我们率先在大语言模型（LLM）内实现多尺度视觉自回归机制，提出尺度感知适配器（SAA），相比基于扩散的方法大幅降低解码延迟，同时保持最优性能。我们的发现表明，纯自回归建模作为统一多模态智能的优雅基础具有强大潜力。因此，OneCAT 在多模态理解、生成与编辑的基准上均优于现有统一模型。
英文原文摘要 1133 字符
We introduce OneCAT, a unified multimodal model that seamlessly integrates understanding, generation, and editing within a single decoder-only transformer architecture. OneCAT uniquely eliminates the need for external components such as Vision Transformers (ViT) or vision tokenizer during inference, leading to significant efficiency gains, especially for high-resolution image inputs and outputs. This is achieved through a modality-specific Mixture-of-Experts (MoE) design trained with a unified autoregressive (AR) objective, which also natively supports dynamic resolutions. Furthermore, we pioneer to achieve multi-scale visual autoregressive mechanism within the Large Language Model (LLM) with proposed scale-aware adapter (SAA) that drastically reduces decoding latency compared to diffusion-based methods while maintaining state-of-the-art performance. Our findings demonstrate the powerful potential of pure autoregressive modeling as an elegant foundation for unified multimodal intelligence. As a result, OneCAT outperforms existing unified models across benchmarks for multimodal understanding, generation, and editing.