Archon：面向整体数字人生成的统一多模态模型
Archon: A Unified Multimodal Model for Holistic Digital Human Generation
语义视频重参数化把高保真说话视频的词元量压缩 4 倍，再配合语义驱动的视频扩散解码器还原细节，这套「先语义压缩、后扩散解码」的流程可直接迁移到任何需要长视频自回归生成的视觉任务；「Thinking in Modality」的交替模态逐步推理思路也可用于提升跨模态条件生成的保真度与可控性。
原文依据 we introduce a memory-efficient semantic video reparameterization, achieving 4x token reduction while preserving fine-grained dynamics
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论数字人是沉浸式交互的基础，然而构建一个涵盖文本、音频、动作与视觉内容等整体模态的统一模型仍是一个开放挑战。本文提出 Archon，一个完全预训练、以人为中心的统一多模态模型，用于整体虚拟形象生成。Archon 通过模态专属分词器统一七种模态，并采用一个原生自回归统一多模态模型，在同步模态与 72 个多样化任务上预训练，以建模整体联合分布。为应对高保真说话视频中的词元爆炸问题，我们引入一种内存高效的语义视频重参数化方法，在保持细粒度动态的同时实现 4 倍词元缩减，并配合一个语义驱动的视频扩散解码器。我们进一步提出「模态内思考」，将歧义的跨模态任务分解为在交替模态链中的逐步思考，渐进式提升保真度与可控性。大量实验表明，Archon 在多种数字人生成任务上取得更优或相当的性能，验证了统一框架的有效性。
英文原文摘要 1191 字符
Digital humans are fundamental to immersive interaction, yet creating a unified model for holistic modalities, including text, audio, motion, and visual content, remains an open challenge. In this paper, we present Archon, a fully pretrained, human-centric unified multimodal model for holistic avatar generation. Archon unifies seven modalities with modality-specific tokenizers, and a native autoregressive unified multimodal model pretrained on synchronized modalities and 72 diverse tasks to model holistic joint distributions. To address the token explosion challenge in high-fidelity talking videos, we introduce a memory-efficient semantic video reparameterization, achieving 4x token reduction while preserving fine-grained dynamics, coupled with a semantic-driven video diffusion decoder. We further propose a "Thinking in Modality" that decomposes ambiguous cross-modal tasks into stepwise thinking in an alternative chain of modality, progressively enhancing fidelity and controllability. Extensive experiments demonstrate that Archon achieves superior or comparable performance across diverse digital human generation tasks, validating the effectiveness of our unified framework.