混合风格专家用于多样图像风格化
Mixture of Style Experts for Diverse Image Stylization
将混合专家(MoE)架构引入扩散风格化,通过相似度感知门控动态路由风格到专门专家,可复用于其他需要多风格控制的生成任务,提升语义和材质细节保持能力。
原文依据 We introduce StyleExpert, a semantic-aware framework based on Mixture of Experts (MoE).
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论基于扩散的风格化已取得显著进展,但现有方法局限于颜色驱动的变换,忽略了复杂的语义和材质细节。我们提出StyleExpert,一个基于混合专家(MoE)的语义感知框架。我们的框架采用统一的风格编码器,在我们大规模的内容-风格-风格化三元组数据集上训练,将多样风格嵌入到一致的潜变量空间中。该嵌入随后用于条件化一个相似度感知的门控机制,该机制动态地将风格路由到MoE架构内的专门专家。利用这种MoE架构,我们的方法能够熟练处理跨越多个语义层次(从浅层纹理到深层语义)的多样风格。大量实验表明,StyleExpert在保持语义和材质细节方面优于现有方法,同时能泛化到未见过的风格。
英文原文摘要 906 字符
Diffusion-based stylization has advanced significantly, yet existing methods are limited to color-driven transformations, neglecting complex semantics and material details. We introduce StyleExpert, a semantic-aware framework based on Mixture of Experts (MoE).Our framework employs a unified style encoder, trained on our large-scale dataset of content-style-stylized triplets, to embed diverse styles into a consistent latent space. This embedding is then used to condition a similarity-aware gating mechanism, which dynamically routes styles to specialized experts within the MoE architecture. Leveraging this MoE architecture, our method adeptly handles diverse styles spanning multiple semantic levels, from shallow textures to deep semantics. Extensive experiments show that StyleExpert outperforms existing approaches in preserving semantics and material details, while generalizing to unseen styles.