FlowComposer:面向组合零样本学习的可组合流
FlowComposer: Composable Flows for Compositional Zero-Shot Learning
把流匹配的速度场组合作为显式的嵌入空间融合算子,可替代 token 拼接/提示词调优来实现组合式概念构造,对做 VLM 下游组合推理与概念解耦的研究者是一个可直接插入基线的即插即用模块。
原文依据 we are the first to systematically study flow matching for CZSL and introduce FlowComposer, a model-agnostic framework
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论组合零样本学习(CZSL)旨在通过重组从已见组合中学习到的基元,识别未见的属性-物体组合。近期基于视觉-语言模型(VLM)的 CZSL 方法通常采用参数高效微调(PEFT),它们使用视觉解耦器进行分解,并操纵词元级提示词或前缀来编码组合。然而,这类基于 PEFT 的设计存在两个根本性局限:(1)隐式组合构造,即组合仅通过词元拼接或分支级提示词调优实现,而非在嵌入空间中执行显式操作;(2)残余特征纠缠,即不完美的解耦使属性、物体与组合特征相互污染。二者共同限制了当前 CZSL 模型的泛化能力。本文首次系统研究将流匹配用于 CZSL,并提出 FlowComposer——一个模型无关的框架,它学习两个基元流,将视觉特征分别输运至属性与物体文本嵌入,并引入一个可学习的 Composer,将二者的速度场显式融合为组合流。为利用不可避免的残余纠缠,我们进一步设计了泄漏引导的增强方案,将泄漏特征复用为辅助信号。我们通过将 FlowComposer 作为即插即用组件集成到多种基线中,在三个公开 CZSL 基准上进行了充分评测,一致取得显著提升。
英文原文摘要 1533 字符
Compositional zero-shot learning (CZSL) aims to recognize unseen attribute-object compositions by recombining primitives learned from seen pairs. Recent CZSL methods built on vision-language models (VLMs) typically adopt parameter-efficient fine-tuning (PEFT).They apply visual disentanglers for decomposition and manipulate token-level prompts or prefixes to encode compositions.However, such PEFT-based designs suffer from two fundamental limitations: (1) Implicit Composition Construction, where composition is realized only via token concatenation or branch-wise prompt tuning rather than an explicit operation in the embedding space; (2) Remained Feature Entanglement, where imperfect disentanglement leaves attribute, object, and composition features mutually contaminated. Together, these issues limit the generalization ability of current CZSL models.In this paper, we are the first to systematically study flow matching for CZSL and introduce FlowComposer, a model-agnostic framework that learns two primitive flows to transport visual features toward attribute and object text embeddings, and a learnable Composer that explicitly fuses their velocity fields into a composition flow. To exploit the inevitable residual entanglement, we further devise a leakage-guided augmentation scheme that reuses leaked features as auxiliary signals.We thoroughly evaluate FlowComposer on three public CZSL benchmarks by integrating it as a plug-and-play component into various baselines, consistently achieving significant improvements.