自监督与表征学习 · CVPR2026 ·2026.06.03

SATTC:面向跨被试脑电到图像检索的结构感知无标签测试时校准

SATTC: Structure-Aware Label-Free Test-Time Calibration for Cross-Subject EEG-to-Image Retrieval

Qunjie Huang、Weina Zhu

与你有关的部分

提供了一种无标签的测试时校准头,直接作用于冻结编码器的相似度矩阵,通过白化与CSLS变体缓解跨域检索中的hubness问题,可复用于视觉-语言跨模态检索或跨域检索的排序稳定化。

原文依据 a label-free calibration head that operates directly on the similarity matrix of frozen EEG and image encoders
可迁移到 跨模态检索测试时自适应表征对齐

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

跨被试脑电到图像的视觉解码检索面临被试偏移和嵌入空间中的hubness问题,这些问题扭曲了相似度几何结构并破坏了top-k排序的稳定性,使得小k候选列表不可靠。我们提出SATTC(结构感知测试时校准),一种无标签校准头,直接作用于冻结的脑电和图像编码器的相似度矩阵。SATTC将脑电嵌入的几何专家被试自适应白化与跨域相似度局部缩放(CSLS)的自适应变体相结合,并构建了一个由互最近邻、双向top-k排名和类别流行度组成的结构专家,通过简单的专家乘积规则进行融合。在THINGS-EEG数据集上,采用严格的留一被试协议,使用余弦相似度、l2归一化嵌入和候选白化的标准化推理已经比原始ATM检索设置产生了强大的跨被试基线。在此基线基础上,SATTC进一步提高了Top-1和Top-5准确率,减少了hubness和每类不平衡,并产生了更可靠的小k候选列表。这些增益在多个脑电编码器上可迁移,支持SATTC作为跨被试神经解码的编码器无关、无标签测试时校准层。代码见https://github.com/QunjieHuang/SATTC-CVPR2026。

英文原文摘要 1349 字符

Cross-subject EEG-to-image retrieval for visual decoding is challenged by subject shift and hubness in the embedding space, which distort similarity geometry and destabilize top-k rankings, making small-k shortlists unreliable. We introduce SATTC (Structure-Aware Test-Time Calibration), a label-free calibration head that operates directly on the similarity matrix of frozen EEG and image encoders. SATTC combines a geometric expert-subject-adaptive whitening of EEG embeddings with an adaptive variant of Cross-domain Similarity Local Scaling (CSLS)--and a structural expert built from mutual nearest neighbors, bidirectional top-k ranks, and class popularity, fused via a simple Product-of-Experts rule. On THINGS-EEG under a strict leave-one-subject-out protocol, standardized inference with cosine similarities, l2-normalized embeddings, and candidate whitening already yields a strong cross-subject baseline over the original ATM retrieval setup. Building on this baseline, SATTC further improves Top-1 and Top-5 accuracy, reduces hubness and per-class imbalance, and produces more reliable small-k shortlists. These gains transfer across multiple EEG encoders, supporting SATTC as an encoder-agnostic, label-free test-time calibration layer for cross-subject neural decoding. Code is available at https://github.com/QunjieHuang/SATTC-CVPR2026

标签

会议 CVPR2026
页码 16887-16896
Bibkey
来源 CVF Open Access