检索反事实样本改进视觉上下文学习
Retrieving Counterfactuals Improves Visual In-Context Learning
把「示例选择」从被动相似度检索改成属性引导的组合图像检索，主动构造反事实示例对，可直接用于任何基于 ICL 的 VLM 少样本适配流程，尤其是小模型与信息稀缺场景。
原文依据 actively constructs demonstration sets by retrieving counterfactual-style examples through targeted, attribute-guided composed image retrieval
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉语言模型（VLM）在多模态推理任务上已取得令人瞩目的表现，但往往难以区分细粒度视觉属性，也难以对潜在的因果关系进行推理。上下文学习（ICL）为 VLM 适配新任务提供了有前景的途径，但其效果关键取决于示例样本的选择。现有的检索增强方法通常依赖被动的基于相似度的检索，容易选出相关但非因果的示例，从而放大虚假关联并限制模型的鲁棒性。我们提出 CIRCLES（Composed Image Retrieval for Causal Learning Example Selection，面向因果学习示例选择的组合图像检索），这是一个新框架，通过有针对性的、属性引导的组合图像检索来获取反事实风格的示例，从而主动构造示例集。通过引入反事实风格的示例，CIRCLES 使 VLM 能够隐式地推理属性与结果之间的因果关系，超越表层相关性，促成更鲁棒、更有依据的推理。在四个多样化数据集上的全面实验表明，CIRCLES 在多种架构上一致优于现有方法，在小规模模型上尤为明显，并在信息稀缺条件下取得显著增益。此外，CIRCLES 检索到的示例更加多样、更具因果信息量，为模型如何利用上下文示例改进推理提供了定性洞见。代码已发布于 https://github.com/gzxiong/CIRCLES。
英文原文摘要 1594 字符
Vision-language models (VLMs) have achieved impressive performance across a wide range of multimodal reasoning tasks, but they often struggle to disentangle fine-grained visual attributes and reason about underlying causal relationships. In-context learning (ICL) offers a promising avenue for VLMs to adapt to new tasks, but its effectiveness critically depends on the selection of demonstration examples. Existing retrieval-augmented approaches typically rely on passive similarity-based retrieval, which tends to select correlated but non-causal examples, amplifying spurious associations and limiting model robustness. We introduce CIRCLES (Composed Image Retrieval for Causal Learning Example Selection), a novel framework that actively constructs demonstration sets by retrieving counterfactual-style examples through targeted, attribute-guided composed image retrieval. By incorporating counterfactual-style examples, CIRCLES enables VLMs to implicitly reason about the causal relations between attributes and outcomes, moving beyond superficial correlations and fostering more robust and grounded reasoning. Comprehensive experiments on four diverse datasets demonstrate that CIRCLES consistently outperforms existing methods across multiple architectures, especially on small-scale models, with pronounced gains under information scarcity. Furthermore, CIRCLES retrieves more diverse and causally informative examples, providing qualitative insights into how models leverage in-context demonstrations for improved reasoning. Our code is available at https://github.com/gzxiong/CIRCLES.