文化何处褪色:揭示文本到图像生成中的文化鸿沟
Where Culture Fades: Revealing the Cultural Gap in Text-to-Image Generation
提出一种通过定位文化敏感神经元并在推理时放大或选择性微调相关层来提升多语言T2I模型文化一致性的方法,为可控生成提供神经元级干预思路。
原文依据 We propose a probing method that localizes culture-sensitive signals to a small set of neurons in a few fixed layers.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多语言文本到图像(T2I)模型在视觉真实感和语义对齐方面已快速发展,并得到广泛应用。然而,输出结果因文化语境而异:由于语言承载文化内涵,从多语言提示合成的图像应保持跨语言的文化一致性。我们进行了一项全面分析,表明当前T2I模型在多语言提示下常产生文化中性或英语偏置的结果。对两个代表性模型的分析表明,问题并非源于缺失文化知识,而是文化相关表征激活不足。我们提出一种探测方法,将文化敏感信号定位到少数固定层中的一小部分神经元。在此发现指导下,我们引入两种互补的对齐策略:(1)推理时文化激活,在不微调骨干网络的情况下放大所识别的神经元;(2)层定向文化增强,仅更新与文化相关的层。在我们的CultureBench上的实验表明,在保持保真度和多样性的同时,文化一致性相比强基线取得了一致提升。
英文原文摘要 1202 字符
Multilingual text-to-image (T2I) models have advanced rapidly in terms of visual realism and semantic alignment, and are now widely utilised. Yet outputs vary across cultural contexts: because language carries cultural connotations, images synthesized from multilingual prompts should preserve cross-lingual cultural consistency. We conduct a comprehensive analysis showing that current T2I models often produce culturally neutral or English-biased results under multilingual prompts.Analyses of two representative models indicate that the issue stems not from missing cultural knowledge but from insufficient activation of culture-related representations. We propose a probing method that localizes culture-sensitive signals to a small set of neurons in a few fixed layers. Guided by this finding, we introduce two complementary alignment strategies: (1) inference-time cultural activation that amplifies the identified neurons without backbone fine-tuned; and (2) layer-targeted cultural enhancement that updates only culturally relevant layers. Experiments on our CultureBench demonstrate consistent improvements over strong baselines in cultural consistency while preserving fidelity and diversity.