机器心理意象：以潜视觉 token 增强多模态推理
Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens
把 VLM 的隐藏状态直接当作「潜视觉 token」续写到解码轨迹里，替代显式生成图像来做视觉想象，为多模态推理提供了一条不依赖图像生成器的中间表示路径；其「先蒸馏图像嵌入、再纯文本监督、最后 RL 微调」的三段式训练流程可直接迁移到其他需要视觉中间步骤的 VLM 任务。
原文依据 augments VLM decoding with latent visual tokens alongside ordinary text
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉语言模型（VLM）在多模态理解方面表现出色，但其纯文本解码迫使模型将视觉推理诉诸语言表述，从而在需要视觉想象的任务上限制了性能。近期的尝试通过训练 VLM 渲染显式图像来弥补这一不足，但沉重的图像生成预训练往往会损害推理能力。受人类借助心理意象进行推理——即在内部构建与操作视觉线索——的启发，我们探究 VLM 能否在不产生显式图像的情况下，通过交错的多模态轨迹进行推理。为此，我们提出机器心理意象框架，称为“Mirage”，它在 VLM 解码过程中于普通文本之外引入潜视觉 token。具体而言，当模型选择“以视觉方式思考”时，它将自己的隐藏状态重新作为下一批 token，从而延续一条多模态轨迹，而不生成像素级图像。我们首先通过从真实图像嵌入中蒸馏来监督潜 token，随后切换为纯文本监督，使潜轨迹与任务目标紧密对齐。此后的强化学习阶段进一步增强多模态推理能力。在多个基准上的实验表明，Mirage 无需显式图像生成即可释放更强的多模态推理能力。
英文原文摘要 1343 字符
Vision-language models (VLMs) excel at multimodal understanding, yet their text-only decoding forces them to verbalize visual reasoning, limiting performance on tasks that demand visual imagination. Recent attempts train VLMs to render explicit images, but the heavy image-generation pre-training often hinders the reasoning ability. Inspired by the way humans reason with mental imagery--the internal construction and manipulation of visual cues--we investigate whether VLMs can reason through interleaved multimodal trajectories without producing explicit images. To this end, we present a Machine Mental Imagery framework, dubbed as "Mirage", which augments VLM decoding with latent visual tokens alongside ordinary text. Concretely, whenever the model chooses to "think visually", it recasts its hidden states as next tokens, thereby continuing a multimodal trajectory without generating pixel-level images. Begin by supervising the latent tokens through distillation from ground-truth image embeddings, we then switch to text-only supervision to make the latent trajectory align tightly with the task objective. A subsequent reinforcement learning stage further enhances the multimodal reasoning capability. Experiments on diverse benchmarks demonstrate that \Model unlocks stronger multimodal reasoning without explicit image generation.