LacTokGen：用 256 个词元生成 1024 像素图像的潜一致性分词器
LacTokGen: Latent Consistency Tokenizer for 1024-pixel Image Generation by 256 Tokens
LacTok 将离散视觉词元与预训练潜扩散模型的紧凑潜空间桥接，用 256 个词元表示 1024×1024 图像，为高分辨率自回归图像生成提供了一种高效的分词方案；其潜一致性解码器将多步采样压缩至 1-2 步，可直接用于像素级监督，这一思路可迁移到其他需要高效解码的生成任务中。
原文依据 we propose Latent Consistency Tokenizer (LacTok) that bridges discrete visual tokens with the compact latent space of pretrained latent diffusion models (LDMs)
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论图像分词显著推动了视觉生成与多模态建模的发展，尤其是与自回归模型结合时。然而，现有方法在效率与质量之间难以平衡：高分辨率图像生成要么需要过量的词元，要么通过词元缩减而牺牲关键细节。为解决这一问题，我们提出潜一致性分词器（LacTok），它将离散视觉词元与预训练潜扩散模型（LDM）的紧凑潜空间桥接起来，从而仅用 256 个词元即可高效表示 1024×1024 图像——相比 VQGAN 实现了 16 倍压缩。LacTok 集成了 Transformer 编码器、量化码本和潜一致性解码器。直接将 LDM 用作解码器会导致颜色和亮度差异；因此，我们将其转换为潜一致性解码器，将多步采样缩减为 1-2 步，以进行直接的像素级监督。为使 LacTok 具备文本到图像生成能力，我们将其与自回归 Transformer 无缝集成，形成 LacTokGen。该 Transformer 通过以文本指令为条件预测紧凑词元序列进行训练。实验表明，LacTok 在高保真重建方面具有优越性，在 MSCOCO-2017 5K 基准上对 1024×1024 图像重建取得了 10.8 的重建 Frechet Inception 距离。LacTokGen 在 GenEval 基准上取得 0.73 的分数，在 MSCOCO-2017 数据集上取得 0.304 的 HPSv2 分数。
英文原文摘要 1480 字符
Image tokenization has significantly advanced visual generation and multimodal modeling, particularly when paired with autoregressive models. However, current methods face challenges in balancing efficiency and quality: high-resolution image generation either requires an excessive number of tokens or compromises critical details through token reduction. To resolve this, we propose Latent Consistency Tokenizer (LacTok) that bridges discrete visual tokens with the compact latent space of pretrained latent diffusion models (LDMs), enabling efficient representation of 1024x1024 images using only 256 tokens--a 16xcompression over VQGAN. LacTok integrates a transformer encoder, a quantized codebook, and a latent consistency decoder. Direct application of LDM as the decoder results in color and brightness discrepancies; thus, we convert it to latent consistency decoder, reducing multi-step sampling to 1-2 steps for direct pixel-level supervision. To endow LacTok with text-to-image generation capabilities, we seamlessly integrate it with an autoregressive transformer, forming LacTokGen. This transformer is trained by predicting compact token sequences conditioned on text instructions. Experiments demonstrate LacTok's superiority in high-fidelity reconstruction, with 10.8 reconstruction Frechet Inception Distance on MSCOCO-2017 5K benchmark for 1024x1024 image reconstruction. LacTokGen achieves 0.73 score on GenEval benchmark and 0.304 HPSv2 on MSCOCO-2017 dataset.