基于一维潜变量表示的生成式视频压缩
Generative Video Compression with One-Dimensional Latent Representation
把视频编码成极紧凑的一维潜变量词元、并用长短期记忆上下文做条件，为视觉研究者提供了一条绕开二维网格刚性对应、按语义自适应聚合帧间信息的表示思路，可直接迁移到视频生成与视频表征压缩的潜空间设计上。
原文依据 GVC1D encodes the video data into extreme compact 1D latent tokens conditioned on both short- and long-term contexts.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近期生成式视频编解码器（GVC）的进展通常将视频编码为二维潜变量网格，并采用高容量生成式解码器进行重建。然而，这一范式在充分利用时空冗余方面仍存在两个关键挑战：在空间上，二维潜变量网格由于其刚性结构不可避免地保留了帧内冗余，相邻图像块之间仍高度相似，因而需要更高的码率；在时间上，二维潜变量网格难以以紧凑且语义连贯的方式建模长程相关性，因为它阻碍了跨帧共同内容的聚合。为解决上述局限，我们提出了基于一维潜变量表示的生成式视频压缩方法（GVC1D）。GVC1D 在短期与长期上下文的共同条件下，将视频数据编码为极度紧凑的一维潜变量词元。由于不再受刚性二维空间对应的约束，这些一维潜变量词元能够自适应地关注语义区域，并自然地便于词元缩减，从而降低空间冗余。此外，所提出的一维记忆在保持低计算成本的同时提供了语义丰富的长期上下文，从而进一步降低时间冗余。实验结果表明，GVC1D 取得了更优的压缩效率：在 HEVC Class B 数据集上，其在感知相似度指标下实现了 60.4% 的码率降低，在 DISTS 指标下实现了 68.8% 的码率降低，超越了以往的视频压缩方法。
英文原文摘要 1496 字符
Recent advancements in generative video codec (GVC) typically encode video into a 2D latent grid and employ high-capacity generative decoders for reconstruction. However, this paradigm still leaves two key challenges in fully exploiting spatial-temporal redundancy: Spatially, the 2D latent grid inevitably preserves intra-frame redundancy due to its rigid structure, where adjacent patches remain highly similar, thereby necessitating a higher bitrate. Temporally, the 2D latent grid is less effective for modeling long-term correlations in a compact and semantically coherent manner, as it hinders the aggregation of common contents across frames. To address these limitations, we introduce Generative Video Compression with One-Dimensional (1D) Latent Representation (GVC1D). GVC1D encodes the video data into extreme compact 1D latent tokens conditioned on both short- and long-term contexts. Without the rigid 2D spatial correspondence, these 1D latent tokens can adaptively attend to semantic regions and naturally facilitate token reduction, thereby reducing spatial redundancy. Furthermore, the proposed 1D memory provides semantically rich long-term context while maintaining low computational cost, thereby further reducing temporal redundancy. Experimental results indicate that GVC1D attains superior compression efficiency, where it achieves bitrate reductions of 60.4% under LPIPS and 68.8% under DISTS on the HEVC Class B dataset, surpassing the previous video compression methods.