BabyVLM-V2：迈向具有发展依据的视觉基础模型预训练与基准评测
BabyVLM-V2: Toward Developmentally Grounded Pretraining and Benchmarking of Vision Foundation Models
DevCV Toolbox 把 NIH Baby Toolbox 的视觉相关测查改造成十个多模态任务的基准，为「样本高效/发展式预训练」提供了可量化的评测协议；做数据高效预训练或小模型 VLM 的工作可以直接用它作为认知对齐的评测集，也可借鉴其纵向婴儿视听语料的构建方式来设计低资源预训练数据。
原文依据 DevCV Toolbox adapts all vision-related measures of the recently released NIH Baby Toolbox into a benchmark suite of ten multimodal tasks
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论儿童早期的发展轨迹为视觉基础模型的样本高效预训练设定了一个自然目标。我们提出 BabyVLM-V2，一个具有发展依据的婴儿启发式视觉-语言建模框架，它在 BabyVLM-V1 的基础上做了大幅改进：构建了纵向、多方面的预训练数据集，提出了一个通用模型，并且最重要的是给出了用于认知评测的 DevCV Toolbox。该预训练集在最大化覆盖范围的同时尽量减少人工筛选，构建了一个纵向的、以婴儿为中心的音视频语料，产出视频-话语、图像-话语以及多轮对话数据，以模拟婴儿的体验。DevCV Toolbox 将最近发布的 NIH Baby Toolbox 中所有与视觉相关的测查项改造为一个包含十项多模态任务的基准套件，涵盖与幼儿能力相匹配的空间推理、记忆和词汇理解。实验结果表明，一个从零开始预训练的紧凑模型可以在 DevCV Toolbox 上取得有竞争力的性能，在部分任务上超过 GPT-4o。我们希望这一有原则、统一的 BabyVLM-V2 框架能够加速视觉基础模型在发展上合理的预训练研究。
英文原文摘要 1198 字符
Early children's developmental trajectories set up a natural goal for sample-efficient pretraining of vision foundation models. We introduce BabyVLM-V2, a developmentally grounded framework for infant-inspired vision-language modeling that extensively improves upon BabyVLM-V1 through a longitudinal, multifaceted pretraining set, a versatile model, and, most importantly, DevCV Toolbox for cognitive evaluation. The pretraining set maximizes coverage while minimizing curation of a longitudinal, infant-centric audiovisual corpus, yielding video-utterance, image-utterance, and multi-turn conversational data that mirror infant experiences. DevCV Toolbox adapts all vision-related measures of the recently released NIH Baby Toolbox into a benchmark suite of ten multimodal tasks, covering spatial reasoning, memory, and vocabulary understanding aligned with early children's capabilities. Experimental results show that a compact model pretrained from scratch can achieve competitive performance on DevCV Toolbox, outperforming GPT-4o on some tasks. We hope the principled, unified BabyVLM-V2 framework will accelerate research in developmentally plausible pretraining of vision foundation models.