AToken：面向视觉的统一分词器
AToken: A Unified Tokenizer for Vision
AToken 提供了一个统一的视觉分词器，将图像、视频和3D资产编码到共享的4D潜变量空间，其纯Transformer架构和4D旋转位置编码可处理任意分辨率和时长，为多模态生成与理解任务提供了可复用的表征基础。
原文依据 AToken encodes these diverse visual inputs into a shared 4D latent space, unifying both tasks and modalities in a single framework.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论我们提出 AToken，这是首个在图像、视频和3D资产上同时实现高保真重建与语义理解的统一视觉分词器。现有分词器通常针对单一模态的重建或理解进行专门化，与之不同，AToken 将这些多样化的视觉输入编码到一个共享的4D潜变量空间中，在单一框架内统一了任务与模态。具体而言，我们引入了一种纯 Transformer 架构，并采用4D旋转位置编码来处理任意分辨率和时序长度的视觉输入。为确保训练稳定，我们引入了一种无对抗的训练目标，结合感知损失和 Gram 矩阵损失，实现了最优的重建质量。通过采用渐进式训练课程，AToken 从单张图像、视频和3D逐步扩展，并同时支持连续和离散潜变量 token。AToken 在图像上达到 0.21 rFID 和 82.2% 的 ImageNet 准确率，在视频上达到 3.01 rFVD 和 40.2% 的 MSRVTT 检索性能，在3D上达到 28.28 峰值信噪比和 90.9% 的分类准确率。在下游应用中，AToken 支持视觉生成任务（例如使用连续和离散 token 的图像生成、文本到视频生成、图像到3D合成）和理解任务（例如多模态大语言模型），在所有基准测试上均取得有竞争力的性能。这些结果为基于统一视觉分词构建的下一代多模态 AI 系统提供了启示。
英文原文摘要 1493 字符
We present AToken, the first unified visual tokenizer that achieves both high-fidelity reconstruction and semantic understanding across images, videos, and 3D assets. Unlike existing tokenizers that specialize in either reconstruction or understanding for single modalities, AToken encodes these diverse visual inputs into a shared 4D latent space, unifying both tasks and modalities in a single framework. Specifically, we introduce a pure transformer architecture with 4D rotary position embeddings to process visual inputs of arbitrary resolutions and temporal durations. To ensure stable training, we introduce an adversarial-free training objective that combines perceptual and Gram matrix losses, achieving state-of-the-art reconstruction quality. By employing a progressive training curriculum, AToken gradually expands from single images, videos, and 3D, and supports both continuous and discrete latent tokens. AToken achieves 0.21 rFID with 82.2% ImageNet accuracy for images, 3.01 rFVD with 40.2% MSRVTT retrieval for videos, and 28.28 PSNR with 90.9% classification accuracy for 3D.. In downstream applications, AToken enables both visual generation tasks (e.g., image generation with continuous and discrete tokens, text-to-video generation, image-to-3D synthesis) and understanding tasks (e.g., multimodal LLMs), achieving competitive performance across all benchmarks. These results shed light on the next-generation multimodal AI systems built upon unified visual tokenization.