统一多模态模型作为自编码器
Unified Multimodal Models as Auto-Encoders
将图像到文本理解与文本到图像生成统一为自编码器框架，并用重建奖励的强化学习联合优化编码器与解码器，为视觉语言模型提供了一种通过重建目标提升细粒度感知与生成保真度的后训练思路。
原文依据 we propose Unified-GRPO, a post-training method based on reinforcement learning that jointly optimizes both modules through reconstructive rewards
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论图像到文本（I2T）理解与文本到图像（T2I）生成是两项基础且重要但传统上相互隔离的多模态任务。尽管二者存在内在联系，现有方法通常独立优化它们，错失了相互增强的机会。本文认为，这两项任务可以在共享的自编码器视角下联系起来，其中文本作为中间潜变量表示，桥接两个方向——将图像编码为文本语义（I2T）以及将文本解码回图像（T2I）。我们的关键洞察是：如果编码器真正“理解”图像，它应捕获所有必要结构；如果解码器真正“理解”文本，它应忠实地恢复该结构。基于此原则，我们提出 Unified-GRPO，一种基于强化学习的后训练方法，通过重建奖励联合优化两个模块，最大化输入图像与生成图像之间的语义一致性。在该重建目标下，编码器被鼓励从输入图像中提取尽可能多准确且全面的语义信息以最大化重建质量，同时解码器被优化以基于编码器的先验进行生成，从而实现自我进化的改进。实验上，我们发现使用文本作为中间表示并在重建强化学习范式下训练，能有效同时受益于 I2T 和 T2I。I2T 模块获得更强的细粒度视觉感知能力，如小物体识别、定位等，而其稠密嵌入与语言先验反过来提供更丰富的语义信号，提升 T2I 的保真度与复杂指令遵循能力。这些结果表明，重建强化学习在自编码框架内建立了相互增强的跨模态协同。
英文原文摘要 1915 字符
Image-to-text (I2T) understanding and text-to-image (T2I) generation are two fundamental, important yet traditionally isolated multimodal tasks. Despite their intrinsic connection, existing approaches typically optimize them independently, missing the opportunity for mutual enhancement. In this paper, we argue that the both tasks can be connected under a shared Auto-Encoder perspective, where text serves as the intermediate latent representation bridging the two directions - encoding images into textual semantics (I2T) and decoding text back into images (T2I). Our key insight is that if the encoder truly "understands" the image, it should capture all essential structure, and if the decoder truly "understands" the text, it should recover that structure faithfully. Building upon this principle, we propose Unified-GRPO, a post-training method based on reinforcement learning that jointly optimizes both modules through reconstructive rewards, maximizing the semantic consistency between the input and the generated images. Under this reconstruction objective, the encoder is encouraged to extract as much accurate and comprehensive semantic information from the input image to maximize reconstruction quality, while the decoder is simultaneously optimized to generate conditioned on the encoder's prior, enabling a self-evolving improvement. Empirically, we find that using text as the intermediate representation and training under a reconstructive RL paradigm effectively benefits both I2T and T2I. The I2T module gains stronger fine-grained visual perception, such as small-object recognition, grounding, etc, while its dense embeddings and language priors, in turn, provide richer semantic signals that improve T2I fidelity and complex instruction following. These results demonstrate that the reconstructive RL establishes a mutually reinforcing cross-modal synergy within the auto-encoding framework.