残差解码器适配器：面向自回归文本渲染的ID保持分词器适配
Residual Decoder Adapter: ID-Preserving Tokenizer Adaption for Autoregressive Text Rendering
提出一种后置适配器，在不改变分词器词元空间和自回归模型的前提下，通过残差分支增强解码器输出，从而提升文本渲染质量；该思路可复用于其他需要提升分词器重建细节的生成任务。
原文依据 we propose the Residual Decoder Adapter that upgrades an existing tokenizer post-hoc without changing its token space
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉自回归（AR）模型通过预测离散词元来生成图像，这些词元由视觉分词器解码。尽管在整体图像生成能力上表现强劲，但它们在文本渲染方面仍存在笔画模糊和字母形状扭曲的问题。本工作将这一局限归因于视觉分词器难以重建细粒度细节。改进分词器虽然直接但代价高昂，因为需要重新训练分词器和AR模型。我们能否在不重新训练现有分词器和AR模型的情况下提升AR模型的文本渲染性能？为此，我们提出了残差解码器适配器（Residual Decoder Adapter），它可以在不改变分词器词元空间的情况下事后升级现有分词器。具体而言，它通过引入两个新颖组件来精炼视觉分词器的解码器输出：（i）一个与原始码本共享词元分布的配对码本；（ii）一个并行分支，用于在像素空间中学习重建图像与真实图像之间的微小差异（残差）。这种残差设计使我们能够以非侵入式方式增强分词器，同时保持与先前AR模型的兼容性。残差解码器适配器大幅显著提升了文本渲染效果。例如，在具有竞争力的TextAtlas基准上，微调后的Janus-Pro的OCR准确率从24.52%提升至58.26%（TextVisionBlend），从12.75%提升至36.81%（StyledTextSynth）。代码将公开。
英文原文摘要 1498 字符
Visual Autoregressive (AR) models generate images by predicting discrete tokens that are decoded by a visual tokenizer.Despite demonstrating strong overall image generation ability, they still underperform on text rendering with blur strokes and disrupt letter shapes. In this work, we trace this limitation to the visual tokenizer, which struggles to reconstruct fine-grained detail.Improving the tokenizer is straightforward but expensive, as it necessitates retraining both the tokenizer and the AR model.Can we improve text rendering performance of AR models without retraining the existing tokenizer and AR model? To achieve this, we propose the Residual Decoder Adapter(\method) that upgrades an existing tokenizer post-hoc without changing its token space. Specifically, it refines the decoder output of the visual tokenizer by introducing two novel components: (i) a paired codebook that shares the token distribution with the original one; (ii) a parallel branch to learn the tiny differences (residual) between the reconstructed image and the ground-truth images in the pixel space. This residual design allows us to enhance the tokenizer non-invasively while preserving compatibility with prior AR models. \method substantially improves text rendering significantly by a large margin. For instance, we boost finetuned Janus-Pro OCR accuracy rises from 24.52% to 58.26% (TextVisionBlend), from 12.75% to 36.81% (StyledTextSynth) on competitive TextAtlas benchmark. Codes will be released.