TAMER：面向零样本心电图诊断的三模态对比对齐与多尺度嵌入细化框架
TAMER: A Tri-Modal Contrastive Alignment and Multi-Scale Embedding Refinement Framework for Zero-Shot ECG Diagnosis
论文提出的三模态对比对齐与多尺度嵌入细化框架，可迁移到视觉-语言-信号的多模态融合任务中，特别是利用诊断报告进行细粒度语义对齐的思路，对医学图像的多模态预训练有直接参考价值。
原文依据 TAMER achieves state-of-the-art zero-shot classification performance (AUC: 81.2%) and strong cross-domain generalization (AUC: 83.1%)
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论心血管疾病（CVD）的诊断高度依赖心电图（ECG）。然而，现有大多数自监督单模态方法受限于表征能力不足，而多模态框架则受困于跨模态的粗粒度语义对齐，从而限制了其在临床环境中的泛化能力。为解决这些局限，我们提出TAMER，一个三模态对比对齐与多尺度嵌入细化框架，联合建模心电图记录、频谱图和诊断报告。TAMER由三个关键组件构成：首先，三模态特征编码与投影（TFEP）模块采用模态特定的编码器从心电图记录、频谱图和诊断报告中提取全局与局部特征，并将其投影到潜变量空间。接着，全局-局部时序-频谱对齐（GLTSA）模块通过对比对齐和时序与频谱模态间的注意力交互，捕获互补的节律级和波形级特征。最后，报告感知对齐与细化（RAAR）模块利用临床报告进行诊断级对齐和波形级细化，实现心电图表征的语义增强。在三个公开心电图数据集上的大量实验表明，TAMER实现了最优的零样本分类性能（AUC：81.2%）和强大的跨域泛化能力（AUC：83.1%），优于现有的单模态和多模态基线方法。源代码已发布于https://github.com/zhouxw12345/TAMER。
英文原文摘要 1593 字符
Cardiovascular disease (CVD) diagnosis relies heavily on electrocardiograms (ECGs). However, most existing self-supervised uni-modal methods suffer from limited representational capacity, while multi-modal frameworks are hindered by coarse-grained semantic alignment across modalities, thus restricting their generalizability in clinical settings. To address these limitations, we propose TAMER, a Tri-modal contrastive Alignment and Multi-scale Embedding Refinement framework that jointly models ECG recordings, spectrograms, and diagnostic reports. TAMER is composed of three key components: First, the tri-modal feature encoding and projection (TFEP) module employs modality-specific encoders to extract global and local features from ECG recordings, spectrograms, and diagnostic reports, and projects them into latent spaces. Then, the global-local temporal-spectral alignment (GLTSA) module captures complementary rhythm- and wave-level characteristics via contrastive alignment and attentive interaction between temporal and spectral modalities. Finally, the report-aware alignment and refinement (RAAR) module performs diagnostic-level alignment and wave-level refinement with clinical reports, enabling semantic enrichment of ECG representations.Extensive experiments on three public ECG datasets demonstrate that TAMER achieves state-of-the-art zero-shot classification performance (AUC: 81.2%) and strong cross-domain generalization (AUC: 83.1%), outperforming existing uni-modal and multi-modal baselines methods.The source code is available at https://github.com/zhouxw12345/TAMER.