OmniZip：学习一个统一且轻量的多模态数据无损压缩器
OmniZip: Learning a Unified and Lightweight Lossless Compressor for Multi-Modal Data
其模态统一分词器把图像可逆地映射为 token 序列，配合模态路由的上下文建模，为视觉研究者提供了一种把图像压缩/编码与轻量 Transformer 骨干结合的思路，可用于端侧图像存储与传输、以及作为生成模型分词器设计的参考。
原文依据 a modality-unified tokenizer that reversibly transforms diverse data into tokens
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论无损压缩对于高效的数据存储与传输至关重要。尽管基于学习的无损压缩器已取得优异结果，但它们大多针对单一模态设计，导致在多模态场景下需要冗余地部署多个压缩器。设计统一的压缩器至关重要却颇具挑战，因为不同数据类型在格式、维度和统计特性上差异巨大。多模态大语言模型提供了一种有前景的解决途径，但其复杂度仍过高，难以实用。为此，我们提出 OmniZip，一个面向多模态数据（如图像、文本、语音、触觉、数据库和基因序列）的统一且轻量的无损压缩器。OmniZip 构建于轻量骨干网络之上，并引入三个关键组件以实现高效的多模态无损压缩：一个模态统一分词器，可将多样数据可逆地转换为 token；一种模态路由上下文学习机制，可实现灵活的多模态上下文建模；以及一种模态路由前馈设计，进一步增强模型的非线性表示灵活性。此外采用重参数化训练策略以增强模型容量。它在多个模态上优于或匹配其他最优压缩器，在 CLIC-M、TouchandGo、enwik9、LibriSpeech 和 WikiSQL 数据集上，相比 gzip 分别取得 42%、57%、62% 以及 42%、53% 的更高压缩效率。它还支持在资源受限的边缘设备上进行近实时推理，在 MacBook CPU 和 iPhone NPU 上可达 1MB/s。我们的代码将在论文录用后发布。
英文原文摘要 1592 字符
Lossless compression is essential for efficient data storage and transmission. Although learning-based lossless compressors achieve strong results, most of them are designed for a single modality, leading to redundant compressor deployments in multi-modal settings. Designing a unified multi-modal compressor is critical yet challenging, as different data types vary largely in format, dimension, and statistics. Multi-modal large language models offer a promising resolution but remain too complex for practical use. Thus, we propose OmniZip, a unified and lightweight lossless compressor for multi-modal data (like image, text, speech, tactile, database, and gene sequence). Built on a lightweight backbone, OmniZip incorporates three key components to enable efficient multi-modal lossless compression: a modality-unified tokenizer that reversibly transforms diverse data into tokens, a modality-routing context learning mechanism that enables flexible multi-modal context modeling, and a modality-routing feedforward design that further enhances the model's nonlinear representation flexibility. A reparameterization training strategy is used to enhance model capacity. It outperforms or matches other state-of-the-art compressors on multiple modalities, achieving 42%, 57%, 62% and 42%, 53% higher compression efficiency than gzip on CLIC-M, TouchandGo, enwik9, LibriSpeech, and WikiSQL datasets, respectively. It also supports near real-time inference on resource-constrained edge devices, reaching up to 1MB/s on MacBook CPUs and iPhone NPUs. Our code will be released upon acceptance.