以婴儿之眼学习看世界:早期视觉食谱在人类与机器中催生鲁棒视觉智能
Learning to See Through a Baby's Eyes: Early Visual Diets Enable Robust Visual Intelligence in Humans and Machines
把婴儿视觉发育的「视觉食谱」(灰度到彩色、模糊到清晰、保持时序连续)当作自监督预训练的课程设计,可直接迁移到视频自监督预训练的数据调度与增强策略上,并提供了纹理-形状冲突、剪影识别、深度序分类、视觉悬崖等一整套评测协议来检验表征的鲁棒性。
原文依据 we train self-supervised learning (SSL) models on object-centric videos under constraints that simulate infant vision
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论新生儿感知世界时视觉敏锐度低、色彩退化且时序连续,随着婴儿发育逐渐变得清晰。为探究这种分阶段「视觉食谱」的生态优势,我们在模拟婴儿视觉的约束下,于以物体为中心的视频上训练自监督学习(SSL)模型:灰度到彩色(C)、模糊到清晰(A)以及保持时序连续性(T),合称 CATDiet。在评测方面,我们构建了覆盖十个数据集的综合基准,涵盖干净与受损图像识别、纹理-形状线索冲突测试、剪影识别、深度序分类以及视觉悬崖范式。所有 CATDiet 变体在物体识别上均表现出更强的鲁棒性,尽管仅在以物体为中心的视频上训练。值得注意的是,模型还呈现出与生物学一致的发展模式,包括镜像猕猴 V1 区突触密度的神经可塑性变化,以及与婴儿视觉悬崖反应相似的行为。基于这些发现,CombDiet 先用 CATDiet 初始化 SSL,再进行标准训练,同时保持时序连续性。在以物体为中心的视频或头戴式婴儿视频上训练后,CombDiet 在域内与域外物体识别及深度感知上均优于标准 SSL。综上,这些结果表明,早期婴儿视觉经验的发展进程为理解机器中鲁棒视觉智能的涌现提供了一个强大的逆向工程框架。所有代码、数据与模型已在 Github 公开。
英文原文摘要 1593 字符
Newborns perceive the world with low-acuity, color-degraded, and temporally continuous vision, which gradually sharpens as infants develop. To explore the ecological advantages of such staged "visual diets", we train self-supervised learning (SSL) models on object-centric videos under constraints that simulate infant vision: grayscale-to-color (C), blur-to-sharp (A), and preserved temporal continuity (T)--collectively termed CATDiet. For evaluation, we establish a comprehensive benchmark across ten datasets, covering clean and corrupted image recognition, texture-shape cue conflict tests, silhouette recognition, depth-order classification, and the visual cliff paradigm.All CATDiet variants demonstrate enhanced robustness in object recognition, despite being trained solely on object-centric videos. Remarkably, models also exhibit biologically aligned developmental patterns, including neural plasticity changes mirroring synaptic density in macaque V1 and behaviors resembling infants' visual cliff responses. Building on these insights, CombDiet initializes SSL with CATDiet before standard training while preserving temporal continuity. Trained on object-centric or head-mounted infant videos, CombDiet outperforms standard SSL on both in-domain and out-of-domain object recognition and depth perception. Together, these results suggest that the developmental progression of early infant visual experience offers a powerful reverse-engineering framework for understanding the emergence of robust visual intelligence in machines. All code, data, and models are available at Github.