相同还是不同？增强视觉语言模型的视觉感知
Same or Not? Enhancing Visual Perception in Vision-Language Models
TWIN 提供了一种可直接加入 VLM 训练语料的图像对「同/不同」判别任务，用来强化模型对细微视觉差异的注意力；做 VLM 微调或细粒度识别时，可把它当作低成本的数据侧增强手段，并用 FGVQA 这类改造自细粒度识别与检索数据集的基准来量化感知精度提升。
原文依据 TWIN is a large-scale dataset of 561,000 image-pair queries that task models to determine whether two visually similar images depict the same object
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉语言模型（VLM）在广泛的视觉理解上表现出色，但仍停留在粗粒度层面，存在视觉偏差，并会遗漏细微的视觉细节。现有训练语料强化了这一局限：它们更强调一般性识别（“这是猫还是狗？”），而非细粒度感知。为解决这一问题，我们引入一个新的训练语料与任务，旨在增强 VLM 的感知能力。TWIN 是一个大规模数据集，包含 561,000 条图像对查询，要求模型判断两张视觉上相似的图像是否描绘同一物体，从而促使模型关注细微的视觉线索。该数据集涵盖跨场景、视角与外观的多种日常物体。在 TWIN 上微调 VLM 可在细粒度识别上取得显著提升，即使在艺术、动物、植物和地标等未见领域也是如此。为量化这些提升，我们提出 FGVQA，这是一个包含 12,000 条查询的基准套件，改造自多个领域的细粒度识别与检索数据集。现有 VLM 在 FGVQA 上表现吃力，而在 TWIN 上微调后提升最高达 19.3%，且不损害其在通用 VQA 基准上的性能。最后，我们的 TWIN 数据集随物体标注数量的增加而良好扩展，分析表明规模是性能的关键。我们期望 TWIN 能作为开源 VLM 训练语料的即插即用补充，推动未来模型的感知精度。
英文原文摘要 1414 字符
Vision-language models (VLMs) excel at broad visual understanding but remain coarse-grained, exhibit visual biases, and miss subtle visual details. Existing training corpora reinforce this limitation by emphasizing general recognition ("Is it a cat or a dog?") over fine-grained perception. To address this, we introduce a new training corpus and task designed to enhance the perceptual abilities of VLMs. TWIN is a large-scale dataset of 561,000 image-pair queries that task models to determine whether two visually similar images depict the same object, encouraging attention to nuanced visual cues. The dataset spans a diverse range of everyday objects across contexts, viewpoints, and appearances. Fine-tuning VLMs on TWIN yields notable gains in fine-grained recognition, even on unseen domains such as art, animals, plants, and landmarks. To quantify these gains, we introduce FGVQA, a benchmark suite of 12,000 queries that repurposes fine-grained recognition and retrieval datasets from multiple domains. While existing VLMs struggle on FGVQA, when fine-tuned on TWIN they improve by up to 19.3%, without compromising performance on general VQA benchmarks. Finally, our TWIN dataset scales favorably with object annotations, and our analysis shows that scale is key to performance. We envision TWIN as a drop-in addition to open-source VLM training corpora, advancing perceptual precision of future models.