解耦以泛化：面向数据稀缺视觉语言推理的上下文优先自进化学习
Decouple to Generalize: Context-First Self-Evolving Learning for Data-Scarce Vision-Language Reasoning
把 RL 后训练中的奖励信号按「理解上下文」与「解题」两个角色解耦，并配合迭代演化的种子题池做课程学习，可直接迁移到视觉推理类 VLM 的数据稀缺微调场景，缓解奖励黑客与策略熵坍塌。
原文依据 By decoupling learning process into dual components (Thinker and Solver), we reasonably quantify the reward signals
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近期，视觉语言模型（VLM）通过强化学习（RL）实现了显著的推理能力，这为在经验时代实现可持续自进化的大型视觉语言模型（LVLM）提供了可行方案。然而，面向 VLM 的强化学习需要大量高质量多模态数据——在化学、地球科学和多模态数学等专业领域尤为困难。现有的合成数据与自奖励机制等策略受限于数据分布有限和对齐困难，最终导致奖励黑客：模型利用高奖励模式，使策略熵坍塌并破坏训练稳定性。我们提出 DoGe（Decouple to Generalize），一个双重解耦框架，通过重新关注合成数据方法所忽视的问题上下文场景，引导模型首先从上下文学习，而非直接解题。通过将学习过程解耦为两个组件（思考者与求解者），我们合理地量化该过程的奖励信号，并提出一种从自由探索上下文到实际求解任务的两阶段强化学习后训练方法。其次，为提升训练数据的多样性，DoGe 构建了一条演化式课程学习流水线：一个扩展的原生领域知识语料库和一个迭代演化的种子问题池。实验表明，我们的方法在多个基准上持续优于基线，为实现自进化 LVLM 提供了一条可扩展的路径。
英文原文摘要 1510 字符
Recent vision-language models (VLMs) achieve remarkable reasoning through reinforcement learning (RL), which provides a feasible solution for realizing continuous self-evolving large vision-language models (LVLMs) in the era of experience. However, RL for VLMs requires abundant high-quality multimodal data--especially challenging in specialized domains like chemistry, earth sciences, and multimodal mathematics. Existing strategies such as synthetic data and self-rewarding mechanisms suffer from limited distributions and alignment difficulties, ultimately causing reward hacking: models exploit high-reward patterns, collapsing policy entropy and destabilizing training. We propose DoGe (Decouple to Generalize), a dual-decoupling framework that guides models to first learn from context rather than problem solving by refocusing on the problem context scenarios overlooked by synthetic data methods. By decoupling learning process into dual components (Thinker and Solver), we reasonably quantify the reward signals of this process and propose a two-stage RL post-training approach from freely exploring context to practically solving tasks. Second, to increase the diversity of training data, DoGe constructs an evolving curriculum learning pipeline: an expanded native domain knowledge corpus and an iteratively evolving seed problems pool. Experiments show that our method consistently outperforms the baseline across various benchmarks, providing a scalable pathway for realizing self-evolving LVLMs.