仅训练期异构图像块-文本图监督用于提升少样本学习适配器
Training-Only Heterogeneous Image-Patch-Text Graph Supervision for Advancing Few-Shot Learning Adapters
把训练期的异构图教师（多尺度图像块+文本提示的图 Transformer）蒸馏进 Tip-Adapter 的键值缓存，测试时零额外延迟，为视觉研究者提供了一种「训练加料、推理不变」的少样本 CLIP 适配增强范式。
原文依据 we employ a cache-aware dual-objective strategy to supervise this relational knowledge directly into the Tip-Adapter's key-value cache
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近期基于适配器的 CLIP 微调方法（如 Tip-Adapter）是强力的少样本学习器，通过缓存支持集特征实现快速原型匹配，从而获得高效性。然而，这些方法依赖全局单模态特征向量，忽略了细粒度的图像块关系及其与类文本的结构对齐。为在不引入推理开销的前提下弥合这一差距，我们提出一种新颖的非对称仅训练期框架。我们不改动轻量适配器，而是构建一个仅在训练期运行的高容量辅助异构图教师。该教师（i）将多尺度视觉图像块与文本提示整合进统一图，（ii）通过模态感知图 Transformer（MGT）执行深度跨模态推理，（iii）施加判别性节点过滤以提取高保真类特征。关键在于，我们采用缓存感知的双目标策略，将这种关系知识直接监督进 Tip-Adapter 的键值缓存，从而有效升级原型，而图教师在测试时被丢弃。因此，推理过程与 Tip-Adapter 完全一致，零额外延迟或内存。在标准 1-16 样本基准上，我们的方法持续刷新最优结果。消融实验证实，辅助图监督、文本引导推理与节点过滤是实现鲁棒少样本适配的关键要素。
英文原文摘要 1422 字符
Recent adapter-based CLIP tuning (e.g., Tip-Adapter) is a strong few-shot learner, achieving efficiency by caching support features for fast prototype matching. However, these methods rely on global uni-modal feature vectors, overlooking fine-grained patch relations and their structural alignment with class text. To bridge this gap without incurring inference costs, we introduce a novel asymmetric training-only framework. Instead of altering the lightweight adapter, we construct a high-capacity auxiliary Heterogeneous Graph Teacher that operates solely during training. This teacher (i) integrates multi-scale visual patches and text prompts into a unified graph, (ii) performs deep cross-modal reasoning via a Modality-aware Graph Transformer (MGT), and (iii) applies discriminative node filtering to extract high-fidelity class features. Crucially, we employ a cache-aware dual-objective strategy to supervise this relational knowledge directly into the Tip-Adapter's key-value cache, effectively upgrading the prototypes while the graph teacher is discarded at test time. Thus, inference remains identical to Tip-Adapter with zero extra latency or memory. Across standard 1-16-shot benchmarks, our method consistently establishes a new state-of-the-art. Ablations confirm that the auxiliary graph supervision, text-guided reasoning, and node filtering are the essential ingredients for robust few-shot adaptation.