一个风格只值一个编码：用离散风格空间解锁编码到风格的图像生成
A Style is Worth One Code: Unlocking Code-to-Style Image Generation with Discrete Style Space
把风格压缩成离散码本中的嵌入，再用自回归模型建模其分布，从而用单个数字码控制 T2I 扩散模型生成风格——这套「离散风格码本 + 自回归采样」的范式可直接迁移到个性化生成、风格 LoRA 替代方案等需要可复现、可枚举风格控制的任务。
原文依据 we first train a discrete style codebook from a collection of images to extract style embeddings
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论创新的视觉风格化是艺术创作的基石，然而生成新颖且一致的视觉风格仍是一项重大挑战。现有的生成方法通常依赖冗长的文本提示、参考图像或参数高效微调来引导风格感知的图像生成，但往往在风格一致性、创造力受限以及风格表示复杂等方面存在不足。本文考虑编码到风格的图像生成任务，其目标是在仅由一个数字编码指定的情况下，生成具有新颖且一致视觉风格的图像。迄今为止，该领域主要仅由工业界（如 Midjourney）探索，学术界尚无开源研究。为填补这一空白，我们提出 CoTyle，这是该任务的首个开源方法。具体而言，我们首先从一组图像中训练一个离散风格码本，以提取风格嵌入。这些嵌入作为文本到图像扩散模型（T2I-DM）的条件，用于生成风格化图像。随后，我们在离散风格嵌入上训练一个自回归风格生成器，以对其分布建模，从而能够合成新颖的风格嵌入。在推理时，一个数字风格编码由风格生成器映射为唯一的风格嵌入，该嵌入引导 T2I-DM 生成相应风格的图像。大量实验验证了 CoTyle 能有效地将数字编码转换为风格控制器，表明一个风格只值一个编码。与现有方法相比，我们的方法生成的风格化图像更加多样且一致，从极简输入中解锁了广阔的可复现风格空间。
英文原文摘要 1716 字符
Innovative visual stylization is a cornerstone of artistic creation, yet generating novel and consistent visual styles remains a significant challenge. Existing generative approaches typically rely on lengthy textual prompts, reference images, or parameter-efficient fine-tuning to guide style-aware image generation, but often struggle with style consistency, limited creativity, and complex style representations. In this paper, we consider the code-to-style image generation task, which aims to produce images with novel and consistent visual styles specified by only a numerical code. To date, this field has only been primarily explored by the industry (e.g., Midjourney), with no open-source research from the academic community. To fill this gap, we propose CoTyle, the first open-source method for this task. Specifically, we first train a discrete style codebook from a collection of images to extract style embeddings. These embeddings serve as conditions for a text-to-image diffusion model (T2I-DM) to generate stylistic images. Subsequently, we train an autoregressive style generator on the discrete style embeddings to model their distribution, allowing the synthesis of novel style embeddings. During inference, a numerical style code is mapped to a unique style embedding by the style generator, and this embedding guides the T2I-DM to generate images in the corresponding style. Extensive experiments validate that CoTyle effectively converts a numerical code into a style controller, demonstrating a style is worth one code. Compared to existing methods, the stylized images generated by our method are more diverse and consistent, unlocking a vast space of reproducible styles from minimal input.