隐式视觉推理
Latent Implicit Visual Reasoning
提出一种任务无关的隐式视觉推理 token 机制：让 LMM 在无显式中间监督下自行发现并重编码图像，可迁移到任何需要视觉中间步骤的多模态推理任务，替代依赖深度图、裁剪图等人工先验的做法。
原文依据 we propose a task-agnostic mechanism that trains LMMs to discover and use visual reasoning tokens without explicit supervision
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论尽管大型多模态模型（LMM）已取得显著进展，但它们仍以文本为中心，依赖语言作为核心推理模态。因此，它们在处理以视觉为主的推理任务时能力有限。近期方法试图通过辅助图像、深度图或图像裁剪来监督中间视觉步骤，以缓解这一问题。然而，这些策略对“有用的”视觉抽象应是什么样施加了限制性先验，带来沉重的标注成本，且难以跨任务泛化。为应对这一关键局限，我们提出一种任务无关的机制，在无显式监督的情况下训练 LMM 发现并使用视觉推理 token。这些 token 进行全局注意力并以任务自适应的方式重新编码图像，使模型无需手工设计的监督即可提取相关视觉信息。我们的方法优于直接微调，并在多种以视觉为中心的任务上取得最优结果——包括那些中间抽象难以明确指定的任务——同时还能泛化到多任务指令微调。
英文原文摘要 1155 字符
While Large Multimodal Models (LMMs) have made significant progress, they remain largely text-centric, relying on language as their core reasoning modality. As a result, they are limited in their ability to handle reasoning tasks that are predominantly visual. Recent approaches have sought to address this by supervising intermediate visual steps with helper images, depth maps, or image crops. However, these strategies impose restrictive priors on what "useful" visual abstractions look like, add heavy annotation costs, and struggle to generalize across tasks. To address this critical limitation, we propose a task-agnostic mechanism that trains LMMs to discover and use visual reasoning tokens without explicit supervision. These tokens attend globally and re-encode the image in a task-adaptive way, enabling the model to extract relevant visual information without hand-crafted supervision. Our approach outperforms direct fine-tuning and achieves state-of-the-art results on a diverse range of vision-centric tasks -- including those where intermediate abstractions are hard to specify -- while also generalizing to multi-task instruction tuning.