跨模态引导的视觉合成用于数据高效的多模态抑郁识别
Cross-Modal Guided Visual Synthesis for Data-Efficient Multimodal Depression Recognition
对视觉研究者而言，可复用其「用音频/文本条件生成视觉特征来扩充训练分布，并以识别器性能闭环反馈驱动生成」的思路，用于视觉样本稀缺时的多模态表征增强。
原文依据 leverages the rich context from audio and text modalities to guide a conditional generative model
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多模态学习系统的性能，尤其是在自动抑郁识别这类高风险领域，从根本上受制于如何从有限且复杂的临床数据中学习鲁棒视觉表征这一难题。为克服该问题，我们提出跨模态引导的视觉合成（CMG-VS），一种在训练过程内部通过合成新的、任务相关的视觉特征来增强学习过程的新型训练框架。CMG-VS 的核心在于利用音频与文本模态所提供的丰富上下文来引导一个条件生成模型。该模型学习从语音与语言到视觉表达的复杂映射，生成多样化的合理视觉行为流形，从而丰富训练分布。关键在于，该合成并非独立的预处理步骤。通过任务引导的联合优化方案，生成过程由下游多模态识别器的性能动态驱动。这一闭环反馈机制确保所合成的视觉特征被优化为对识别任务具有最大判别性，而不仅仅是逼真。在广泛使用的 DAIC-WOZ 与 E-DAIC 基准数据集上的综合实验表明，CMG-VS 在所有标准回归与分类指标上均显著优于现有最优方法。消融研究进一步验证，任务引导的合成是这一性能提升的关键驱动因素，证明其作为鲁棒多模态表征学习新范式的有效性。
英文原文摘要 1574 字符
The performance of multimodal learning systems, particularly in high-stakes domains like automated depression recognition, is fundamentally constrained by the challenge of learning robust visual representations from limited and complex clinical data. To overcome this, we introduce Cross-Modal Guided Visual Synthesis (CMG-VS), a novel training framework that internally enhances the learning process by synthesizing new, task-relevant visual features. At its core, CMG-VS leverages the rich context from audio and text modalities to guide a conditional generative model. This model learns the intricate mapping from speech and language to visual expression, generating a diverse manifold of plausible visual behaviors to enrich the training distribution. Crucially, this synthesis is not a separate pre-processing step. Through a task-guided joint optimization scheme, the generative process is dynamically steered by the downstream multimodal recognizer's performance. This closed-loop feedback mechanism ensures the synthesized visual features are optimized to be maximally discriminative for the recognition task, rather than merely realistic. Comprehensive experiments on the widely-used DAIC-WOZ and E-DAIC benchmark datasets demonstrate that CMG-VS significantly outperforms existing state-of-the-art methods across all standard regression and classification metrics. Ablation studies further validate that our task-guided synthesis is the key driver of this performance gain, proving its effectiveness as a new paradigm for robust multimodal representation learning.