VisPlay：自演化视觉语言模型
VisPlay: Self-Evolving Vision-Language Models
把无标注图像变成自生成视觉问答的监督信号，用 GRPO 的多样性与难度奖励联合训练提问器和推理器，可作为 VLM 视觉推理与幻觉抑制的可复用自训练流程。
原文依据 VisPlay assigns the model into two interacting roles: an Image-Conditioned Questioner that formulates challenging yet answerable visual questions, and a Multimodal Reasoner that generates silver responses.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论强化学习（RL）为提升视觉语言模型（VLM）在复杂推理任务上的表现提供了系统性框架。然而，现有 RL 方法往往依赖人工标注标签或任务特定的启发式规则来定义可验证奖励，二者成本高昂且可扩展性有限。我们提出 VisPlay，一个自演化 RL 框架，使 VLM 能够从海量无标注图像数据中自主提升推理能力。VisPlay 从单一基座 VLM 出发，将模型分配为两个相互作用的角色：一个图像条件提问者，负责构造具有挑战性但可回答的视觉问题；以及一个多模态推理者，负责生成银标准回答。这两个角色使用组相对策略优化（GRPO）联合训练，该方法利用多样性与难度奖励，在生成问题的难度与银标准回答的质量之间取得平衡。VisPlay 在两个模型系列上均能高效扩展。在 Qwen2.5-VL 与 MiMo-VL 上训练后，VisPlay 在包括 MM-Vet 与 MMMU 在内的八个基准上，在视觉推理、组合泛化与幻觉减少方面均取得一致提升，并为自演化多模态智能建立了一条可扩展的路径。
英文原文摘要 1244 字符
Reinforcement learning (RL) provides a principled framework for improving vision-language models (VLMs) on complex reasoning tasks. However, existing RL approaches often depend on human-annotated labels or task-specific heuristics to define verifiable rewards--both costly and limited in scalability. We introduce VisPlay, a self-evolving RL framework that enables VLMs to autonomously improve their reasoning capabilities from massive unlabeled image data. Starting from a single base VLM, VisPlay assigns the model into two interacting roles: an Image-Conditioned Questioner that formulates challenging yet answerable visual questions, and a Multimodal Reasoner that generates silver responses. These roles are jointly trained using Group Relative Policy Optimization (GRPO), which uses diversity and difficulty rewards to balance the difficulty of generated questions with the quality of silver answers. VisPlay scales efficiently across two model families. Trained on Qwen2.5-VL and MiMo-VL, VisPlay achieves consistent improvements in visual reasoning, compositional generalization, and hallucination reduction across eight benchmarks including MM-Vet and MMMU, and establishes a scalable path toward self-evolving multimodal intelligence.