DiT-IC：面向高效图像压缩的对齐扩散 Transformer
DiT-IC: Aligned Diffusion Transformer for Efficient Image Compression
把多步文本到图像 DiT 通过自蒸馏对齐与方差引导重建流改造成单步、32 倍下采样潜空间中的重建器，为视觉研究者提供了一个可复用的思路：用预训练扩散先验做极低分辨率潜空间的高效解码，可迁移到超分、修复等图像复原任务。
原文依据 replaces the UNet with a Diffusion Transformer capable of performing diffusion in latent space entirely at 32xdownscaled resolution
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论基于扩散的图像压缩近来展现出卓越的感知保真度，但其实际可用性受制于高昂的采样开销与内存占用。现有扩散编解码器大多采用 UNet 架构，其层级下采样迫使扩散在较浅的潜空间（通常仅 8 倍空间下采样）中运行，导致计算量过大。相比之下，传统基于 VAE 的编解码器工作在深得多的潜变量域（16 倍至 64 倍下采样），这引出一个关键问题：扩散能否在不损害重建质量的前提下，在如此紧凑的潜空间中有效运行？为此，我们提出 DiT-IC——一种用于图像压缩的对齐扩散 Transformer——它以扩散 Transformer 取代 UNet，从而能够在完全 32 倍下采样分辨率的潜空间中执行扩散。DiT-IC 通过三种关键对齐机制，将预训练的文本到图像多步 DiT 改造为单步重建模型：（1）方差引导的重建流，根据潜变量不确定性调整去噪强度以实现高效重建；（2）自蒸馏对齐，强制与编码器所定义的潜变量几何保持一致，从而实现单步扩散；（3）潜变量条件引导，以语义对齐的潜变量条件替代文本提示，实现无文本推理。借助这些设计，DiT-IC 在达到最优感知质量的同时，相比现有基于扩散的编解码器实现最高 30 倍的解码加速，并大幅降低内存占用。值得注意的是，它能在 16 GB 的笔记本 GPU 上重建 2048×2048 的图像。
英文原文摘要 1642 字符
Diffusion-based image compression has recently shown outstanding perceptual fidelity, yet its practicality is hindered by prohibitive sampling overhead and high memory usage.Most existing diffusion codecs employ UNet architectures, where hierarchical downsampling forces diffusion to operate in shallow latent spaces (typically with only 8xspatial downscaling), resulting in excessive computation.In contrast, conventional VAE-based codecs work in much deeper latent domains (16x-64xdownscaled), motivating a key question:Can diffusion operate effectively in such compact latent spaces without compromising reconstruction quality?To address this, we introduce DiT-IC--an Aligned Diffusion Transformer for Image Compression--which replaces the UNet with a Diffusion Transformer capable of performing diffusion in latent space entirely at 32xdownscaled resolution.DiT-IC adapts a pretrained text-to-image multi-step DiT into a single-step reconstruction model through three key alignment mechanisms:(1) a variance-guided reconstruction flow that adapts denoising strength to latent uncertainty for efficient reconstruction;(2) a self-distillation alignment that enforces consistency with encoder-defined latent geometry to enable one-step diffusion; and(3) a latent-conditioned guidance that replaces text prompts with semantically aligned latent conditions, enabling text-free inference.With these designs, DiT-IC achieves state-of-the-art perceptual quality while offering up to 30x faster decoding and drastically lower memory usage than existing diffusion-based codecs. Remarkably, it can reconstruct 2048x2048 images on a 16 GB laptop GPU.