LOREAL：以属性驱动提示自蒸馏缓解视觉语言模型的低分辨率挑战
LOREAL: Mitigating Low-Resolution Challenges in Vision-Language Models with Attribute-driven Prompt Self-Distillation
针对边缘设备低分辨率输入（<224×224、token 数减少）下 CLIP 类 VLM 的性能退化，提出双学生提示自蒸馏框架，用 LLM 查询的分辨率不变属性语义与跨模态元网络在低层/高层做跨分辨率蒸馏，可直接迁移到任何需要分辨率鲁棒性的提示学习或参数高效微调流程中。
原文依据 we propose LOREAL, an efficient prompt self-distillation framework that learns resolution-invariant representations by excavating attribute semantics
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论提示学习（PL）已成为一种将视觉语言模型（VLM）适配到下游任务的参数高效技术。然而，几乎所有现有 PL 方法都主要在精心整理的数据集上设计与评估，忽视了部署后出现的一个关键现象，即输入分辨率与存储内存消耗之间的内在联系。具体而言，为满足边缘设备上严格的存储内存约束，模型常被限制为低分辨率输入（例如 CLIP-ViT/B-16 的输入小于 224×224）并生成更少的 token（位置嵌入相应调整大小），这在性能鲁棒性上构成了独特挑战。为解决该问题，我们提出 LOREAL，一个高效的提示自蒸馏框架，通过挖掘属性语义来学习分辨率不变的表征。LOREAL 的核心是双学生架构，即两个分别接收不同分辨率输入的学生模型协同地相互学习。在此基础上，我们用从 LLM 查询得到的、分辨率不变的属性来构造学生的提示词上下文，然后利用跨模态元网络生成属性语义。这些元网络在两个学生的不同编码器之间建立桥接，其中我们引入低层蒸馏（LLD）和高层蒸馏（HLD），以促进学习更具跨分辨率能力的表征。大量实验表明，LOREAL 在不同分辨率设置下显著提升了 VLM 的性能与鲁棒性，凸显出重要的实用价值。
英文原文摘要 1590 字符
Prompt Learning (PL) has emerged as a parameter-efficient technique for adapting Vision-Language Models (VLMs) to downstream tasks. However, almost all existing PL methods are primarily designed and evaluated on well-curated datasets, overlooking a critical post-deployment phenomenon, i.e., the intrinsic connection between input resolution and storage-memory consumption. Specifically, to satisfy the stringent storage-memory constraints on edge devices, models are often limited to low-resolution inputs (e.g., <224x224 for CLIP-ViT/B-16) and generate fewer tokens (with the position embedding resized), which poses a unique challenge in performance robustness. To tackle this issue, we propose LOREAL, an efficient prompt self-distillation framework that learns resolution-invariant representations by excavating attribute semantics. At the heart of LOREAL is a dual-student architecture, i.e., two student models fed with inputs at different resolutions synergistically learn from each other. Building upon this, we contextualize the students' prompt with resolution-invariant attributes queried from the LLM, then leverage cross-modality meta-nets to generate attribute semantics. These meta-nets are bridged between the different encoders of two students, wherein we introduce Low-Level Distillation (LLD) and High-Level Distillation (HLD) to facilitate the learning of more cross-resolution representations. Extensive experiments show that LOREAL significantly improves VLMs' performance and robustness under varied resolution settings, underscoring significant practical utilities.