迈向统一的人类感知与机器理解：词元流引导的压缩框架
Towards Unified Human Perception and Machine Understanding: Token Flow Guided Compression Framework
TFGC 提出基于词元的灵活压缩框架，让 LVLM 直接处理压缩后的词元而无需图像重建，并支持超低码率下的可变码率控制；其词元流传播和语义引导对齐策略可被视觉研究者复用于设计面向机器理解的压缩-理解联合优化方案。
原文依据 TFGC supports variable bitrate control in ultra-low bitrate regimes and enables LVLMs to directly process compressed token without image reconstruction.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论随着用于图像理解的大型视觉语言模型（LVLMs）迅速崛起，图像压缩的目标正逐渐从人类视觉感知转向面向机器的语义理解。然而，传统的学习型压缩技术针对像素级保真度进行优化，并且通常在固定或刚性的码率点运行，与语义一致性和灵活的码率控制不匹配。这一差距在超低码率场景下变得尤为关键，因为随着码率变化，潜变量表示往往忽略语义相关性，难以从冗余的视觉细节中分离出有意义的内容。为了解决这些挑战，我们开发了一种基于词元的灵活压缩框架——词元流引导压缩（TFGC），它统一了面向人类和面向机器的目标。TFGC 在超低码率场景下支持可变码率控制，并使 LVLMs 能够直接处理压缩后的词元而无需图像重建。具体而言，我们探索了一维词元序列中的词元流现象，并利用它设计词元流传播，通过从未掩码词元传播上下文信息来预测缺失词元。此外，词元语义引导将压缩表示与 LVLM 语义空间对齐，而渐进式语义对齐训练策略进一步弥合了感知重建与语义推理之间的差距。实验表明，我们的框架在可比码率下实现了最优的 LVLMs 理解性能，同时保持了令人满意的感知质量。
英文原文摘要 1578 字符
With the rapid rise of Large Vision Language Models (LVLMs) for image understanding, the objective of image compression is gradually shifting from human visual perception to machine-oriented semantic understanding. However, conventional learned compression techniques are optimized for pixel-level fidelity and typically operate at fixed or rigid bitrate points, misaligned with the semantic consistency and flexible bitrate control. This gap becomes critical in ultra-low-bitrate regimes, where latent representations often ignore semantic relevance and struggle to disentangle meaningful content from redundant visual details as the bitrate varies. To address these challenges, we develop a token-based flexible compression framework, Token Flow Guided Compression (TFGC), which unifies human- and machine-oriented objectives. TFGC supports variable bitrate control in ultra-low bitrate regimes and enables LVLMs to directly process compressed token without image reconstruction. Specifically, we explore token flow phenomenon in 1D token sequences and exploit it to design token flow propagation, which predicts missing tokens by propagating contextual information from unmasked tokens. Moreover, token semantic guidance aligns compressed representations with LVLM semantic space, while a progressive semantic alignment training strategy further bridges the gap between perceptual reconstruction and semantic reasoning. Experiments show that our framework achieves state-of-the-art LVLMs understanding at comparable bitrates while maintaining satisfactory perceptual quality.