InstAP：面向时空理解的实例感知视觉-语言预训练
InstAP: Instance-Aware Vision-Language Pre-Train for Spatial-Temporal Understanding
提出实例级对比对齐目标，将文本提及定位到时空区域，可复用于需要细粒度视觉-语言对齐的任务（如指代分割、视频实例检索），并证明实例级预训练能提升全局零样本视频理解。
原文依据 jointly optimizes global vision-text alignment and fine-grained, instance-level contrastive alignment by grounding textual mentions to specific spatial-temporal regions
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论当前视觉-语言预训练（VLP）范式在全局场景理解上表现优异，但由于仅依赖全局监督，在实例级推理上仍存在困难。我们提出 InstAP，一种实例感知预训练框架，通过将文本提及定位到特定时空区域，联合优化全局视觉-文本对齐与细粒度、实例级对比对齐。为支持该框架，我们构建了 InstVL，一个大规模数据集（200 万张图像、5 万个视频），具有双粒度标注：整体场景描述和密集的、定位的实例描述。在 InstVL 基准上，InstAP 在实例级检索任务上大幅优于现有 VLP 模型，并且也超越了在完全相同数据语料上训练的强 VLP 基线，从而分离出我们实例感知目标带来的收益。此外，以实例为中心的预训练还提升了全局理解：InstAP 在多个视频基准（包括 MSR-VTT 和 DiDeMo）上取得了有竞争力的零样本性能。定性可视化进一步表明，InstAP 能将文本提及定位到正确的实例，而仅全局模型则表现出更弥散、场景级的注意力。
英文原文摘要 1192 字符
Current vision-language pre-training (VLP) paradigms excel at global scene understanding but struggle with instance-level reasoning due to global-only supervision. We introduce InstAP, an Instance-Aware Pre-training framework that jointly optimizes global vision-text alignment and fine-grained, instance-level contrastive alignment by grounding textual mentions to specific spatial-temporal regions. To support this, we present InstVL, a large-scale dataset (2 million images, 50,000 videos) with dual-granularity annotations: holistic scene captions and dense, grounded instance descriptions. On the InstVL benchmark, InstAP substantially outperforms existing VLP models on instance-level retrieval, and also surpasses a strong VLP baseline trained on the exact same data corpus, isolating the benefit of our instance-aware objective. Moreover, instance-centric pre-training improves global understanding: InstAP achieves competitive zero-shot performance on multiple video benchmarks, including MSR-VTT and DiDeMo. Qualitative visualizations further show that InstAP localizes textual mentions to the correct instances, while global-only models exhibit more diffuse, scene-level attention.