Sparse-LaViDa：稀疏多模态离散扩散语言模型
Sparse-LaViDa: Sparse Multimodal Discrete Diffusion Language Models
对做视觉生成/编辑的人，这篇给出了一个可直接套用的加速手段：在掩码离散扩散的每一步采样中动态截断冗余的掩码 token，并用 register token 保留其信息、用匹配的注意力掩码对齐训练与推理，从而在文本到图像生成与图像编辑上获得约 2 倍加速而不掉质量。
原文依据 dynamically truncates unnecessary masked tokens at each inference step to accelerate MDM sampling
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论掩码离散扩散模型（MDM）在图像理解、生成与编辑等广泛的多模态任务上已取得强劲性能。然而，由于在每个采样步骤都需要重复处理冗余的掩码 token，其推理速度仍不理想。本文提出 Sparse-LaViDa，一种新的建模框架，在每次推理步骤动态截断不必要的掩码 token，以加速 MDM 采样。为保持生成质量，我们引入专门的 register token 作为被截断 token 的稀疏表示。此外，为保证训练与推理之间的一致性，我们设计了一种专门的注意力掩码，使其在训练时忠实匹配截断式采样过程。在最优的统一 MDM 模型 LaViDa-O 的基础上，Sparse-LaViDa 在文本到图像生成、图像编辑与数学推理等多种任务上实现了最高 2 倍加速，同时保持生成质量。
英文原文摘要 1025 字符
Masked Discrete Diffusion Models (MDMs) have achieved strong performance across a wide range of multimodal tasks, including image understanding, generation, and editing. However, their inference speed remains suboptimal due to the need to repeatedly process redundant masked tokens at every sampling step. In this work, we propose Sparse-LaViDa, a novel modeling framework that dynamically truncates unnecessary masked tokens at each inference step to accelerate MDM sampling. To preserve generation quality, we introduce specialized register tokens that serve as sparse representations for the truncated tokens. Furthermore, to ensure consistency between training and inference, we design a specialized attention mask that faithfully matches the truncated sampling procedure during training. Built upon the state-of-the-art unified MDM LaViDa-O, Sparse-LaViDa achieves up to a 2xspeedup across diverse tasks including text-to-image generation, image editing, and mathematical reasoning, while maintaining generation quality.