DiG：差分视觉定位——增强多模态大语言模型的细粒度感知
DiG: Differential Grounding for Enhancing Fine-Grained Perception in Multimodal Large Language Models
把「找出两张相似图的全部差异并定位」做成无需标注数量的代理任务，配合3D渲染自动造配对数据，可直接用于增强MLLM的细粒度定位与空间推理，并可迁移到RefCOCO系列指代定位。
原文依据 identifying and localizing all differences between similar image pairs without prior knowledge of their number
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多模态大语言模型在多种视觉-语言任务上已取得令人瞩目的表现，但其细粒度视觉感知与精确空间推理仍然有限。本文提出 DiG（差分视觉定位），一种新的代理任务框架：多模态大语言模型通过识别并定位相似图像对之间的全部差异来学习细粒度感知，且无需事先知道差异的数量。为支持可扩展的训练，我们构建了一条基于三维渲染的自动化数据生成流程，能够产出差异完全可控的高质量配对图像。针对差异信号的稀疏性，我们进一步采用课程学习，使复杂度从单一差异逐步提升到多重差异，从而实现稳定优化。大量实验表明，DiG 在多种视觉感知基准上显著提升模型性能，且所学到的细粒度感知能力可有效迁移到标准下游任务，包括 RefCOCO、RefCOCO+、RefCOCOg 以及通用多模态感知基准。结果凸显了差分视觉定位作为一种可扩展且稳健的方法，在推进多模态大语言模型细粒度视觉推理方面的价值。
英文原文摘要 1262 字符
Multimodal Large Language Models have achieved impressive performance on a variety of vision-language tasks, yet their fine-grained visual perception and precise spatial reasoning remain limited. In this work, we introduce DiG (Differential Grounding), a novel proxy task framework where MLLMs learn fine-grained perception by identifying and localizing all differences between similar image pairs without prior knowledge of their number. To support scalable training, we develop an automated 3D rendering-based data generation pipeline that produces high-quality paired images with fully controllable discrepancies. To address the sparsity of difference signals, we further employ curriculum learning that progressively increases complexity from single to multiple differences, enabling stable optimization. Extensive experiments demonstrate that DiG significantly improves model performance across a variety of visual perception benchmarks and that the learned fine-grained perception skills transfer effectively to standard downstream tasks, including RefCOCO, RefCOCO+, RefCOCOg, and general multimodal perception benchmarks. Our results highlight differential grounding as a scalable and robust approach for advancing fine-grained visual reasoning in MLLMs.