CLCR：面向多模态学习的跨层级语义协同表征
CLCR: Cross-Level Semantic Collaborative Representation for Multimodal Learning
对做视觉-语言或多模态融合的研究者，可复用其「层级化共享/私有子空间 + 可学习词元预算约束跨模态注意力」的设计，避免私有信息在融合时泄漏，并可用其层级锚点聚合思路处理视觉与文本语义尺度不同步的问题。
原文依据 restricts cross-modal attention to the shared subspace via a learnable token budget
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多模态学习旨在从多种模态中同时捕获共享信息与私有信息。然而，现有方法将所有模态投影到单一潜空间中进行融合，往往忽视了多模态数据异步的、多层次的语义结构。这一疏漏会引发语义错位与误差传播，从而降低表征质量。为此，我们提出跨层级协同表征（CLCR），它显式地将每个模态的特征组织为三层语义层级，并为跨模态交互规定逐层约束。首先，语义层级编码器对齐各模态的浅层、中层与深层特征，为交互建立共同基础。随后，在每一层级上，层内协同交换域（IntraCED）将特征分解为共享子空间与私有子空间，并通过可学习的词元预算将跨模态注意力限制在共享子空间内。该设计确保只有共享语义被交换，并防止私有通道的信息泄漏。为跨层级整合信息，层间协同聚合域（InterCAD）利用学习到的锚点同步语义尺度，选择性地融合共享表征，并对私有线索进行门控，从而形成紧凑的任务表征。我们进一步引入正则化项，以强化共享特征与私有特征的分离，并最小化跨层级干扰。在涵盖情感识别、事件定位、情感分析与动作识别的六个基准上的实验表明，CLCR 取得了强劲的性能，并在不同任务间具有良好的泛化能力。
英文原文摘要 1651 字符
Multimodal learning aims to capture both shared and private information from multiple modalities. However, existing methods that project all modalities into a single latent space for fusion often overlook the asynchronous, multi-level semantic structure of multimodal data. This oversight induces semantic misalignment and error propagation, thereby degrading representation quality. Hence, we propose Cross-Level Collaborative Representation (CLCR), which explicitly organizes each modality's features into a three-level semantic hierarchy and specifies level-wise constraints for cross-modal interactions. First, a semantic hierarchy encoder aligns shallow, mid, and deep features across modalities, establishing a common basis for interaction. And then, at each level, an Intra-Level Co-Exchange Domain (IntraCED) factorizes features into shared and private subspaces and restricts cross-modal attention to the shared subspace via a learnable token budget. This design ensures that only shared semantics are exchanged and prevents leakage from private channels. To integrate information across levels, the Inter-Level Co-Aggregation Domain (InterCAD) synchronizes semantic scales using learned anchors, selectively fuses the shared representations, and gates private cues to form a compact task representation. We further introduce regularization terms to enforce separation of shared and private features and to minimize cross-level interference. Experiments on six benchmarks spanning emotion recognition, event localization, sentiment analysis, and action recognition show that CLCR achieves strong performance and generalizes well across tasks.