视觉语言模型 · CVPR2026 ·2026.06.03

OneThinker:面向图像与视频的一体化推理模型

OneThinker: All-in-one Reasoning Model for Image and Video

Kaituo Feng、Manyuan Zhang、Hongyu Li、Kaixuan Fan、Shuang Chen、Yilei Jiang 等 14 人

与你有关的部分

把图像与视频的问答、描述、空间/时序定位、跟踪、分割统一进一个 MLLM,并用 EMA-GRPO 处理多任务 RL 的奖励异质性,为做统一视觉推理模型的人提供了可复用的多任务 RL 平衡策略与跨任务知识迁移证据。

原文依据 we propose EMA-GRPO to handle reward heterogeneity in multi-task RL by tracking task-wise moving averages of reward standard deviations
可迁移到 视觉问答图像描述空间定位时序定位

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

强化学习(RL)近期在多模态大语言模型(MLLM)中激发视觉推理方面取得了显著成功。然而,现有方法通常为不同任务分别训练独立模型,并将图像推理与视频推理视为互不相干的领域。这导致其向多模态推理通用模型扩展的能力受限,既削弱了实际使用的通用性,也阻碍了跨任务、跨模态的潜在知识共享。为此,我们提出 OneThinker,一个一体化推理模型,在多种基础视觉任务上统一图像与视频理解,涵盖问答、描述、空间与时序定位、跟踪以及分割。为实现这一目标,我们构建了覆盖上述全部任务的 OneThinker-600k 训练语料,并借助商用模型进行思维链(CoT)标注,得到用于 SFT 冷启动的 OneThinker-SFT-340k。此外,我们提出 EMA-GRPO,通过跟踪各任务奖励标准差的滑动平均来应对多任务 RL 中的奖励异质性,从而实现均衡优化。在多种视觉基准上的大量实验表明,OneThinker 在 10 项基础视觉理解任务的 31 个基准上均取得强劲性能。此外,它在某些任务之间展现出有效的知识迁移,并具备初步的零样本泛化能力,标志着向统一多模态推理通用模型迈出了一步。所有代码、模型与数据均已开源。

英文原文摘要 1481 字符

Reinforcement learning (RL) has recently achieved remarkable success in eliciting visual reasoning within Multimodal Large Language Models (MLLMs). However, existing approaches typically train separate models for different tasks and treat image and video reasoning as disjoint domains. This results in limited scalability toward a multimodal reasoning generalist, which restricts practical versatility and hinders potential knowledge sharing across tasks and modalities.To this end, we propose OneThinker, an all-in-one reasoning model that unifies image and video understanding across diverse fundamental visual tasks, including question answering, captioning, spatial and temporal grounding, tracking, and segmentation. To achieve this, we construct the OneThinker-600k training corpus covering all these tasks and employ commercial models for CoT annotation, resulting in OneThinker-SFT-340k for SFT cold start.Moreover, we propose EMA-GRPO to handle reward heterogeneity in multi-task RL by tracking task-wise moving averages of reward standard deviations for balanced optimization. Extensive experiments on diverse visual benchmarks show that OneThinker delivers strong performance on 31 benchmarks, across 10 fundamental visual understanding tasks.Moreover, it exhibits effective knowledge transfer between certain tasks and preliminary zero-shot generalization ability, marking a step toward a unified multimodal reasoning generalist. All code, model, and data are released.

标签

会议 CVPR2026
页码 5432-5443
Bibkey
来源 CVF Open Access