InternVideo-Next：迈向世界理解的视频模型
InternVideo-Next: Towards World-Understanding Video Models
提出 Encoder-Predictor-Decoder 框架，用条件扩散解码器注入图像语义先验，解决像素级 MVM 收敛难与语义冲突问题，为视频自监督预训练提供可复用的潜空间世界模型构建方案。
原文依据 we disentangle the traditional encoder-decoder design into an Encoder-Predictor-Decoder (EPD) framework, where the predictor acts as a latent world model
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论大规模视频-文本预训练取得了强劲性能，但依赖于噪声大、语义覆盖有限的合成字幕，往往忽略了物体运动、3D几何和物理线索等隐式世界知识。相比之下，掩码视频建模（MVM）直接利用时空结构，但在通用任务上落后于文本监督方法。我们发现这一差距源于被忽视的架构问题：像素级重建难以收敛，其低层要求常与语义冲突，而潜变量预测往往鼓励捷径学习。为解决这些问题，我们将传统的编码器-解码器设计解耦为编码器-预测器-解码器（EPD）框架，其中预测器充当潜空间世界模型，并提出InternVideo-Next，一种两阶段预训练方案，为该世界模型构建语义一致且保留细节的潜空间。首先，像素MVM中传统的线性解码器强制预测器的输出潜变量被线性投影，从而在像素空间中可分，导致与语义抽象的冲突。我们的第一阶段提出条件扩散解码器，并注入干净的图像级语义先验以增强语义和收敛，从而弥合像素级保真度与高层语义抽象。第二阶段通过在该空间内预测冻结的第一阶段目标来学习世界知识，缓解捷径学习。在公开无标签视频上训练，InternVideo-Next在各项基准上达到最优结果，并为通用视频表征学习提供了可扩展的路径。
英文原文摘要 1643 字符
Large-scale video-text pretraining achieves strong performance but depends on noisy, synthetic captions with limited semantic coverage, often overlooking implicit world knowledge such as object motion, 3D geometry, and physical cues. In contrast, masked video modeling (MVM) directly exploits spatiotemporal structures but trails text-supervised methods on general tasks.We find this gap arises from overlooked architectural issues: pixel-level reconstruction struggles with convergence and its low-level requirement often conflicts with semantics, while latent prediction often encourages shortcut learning.To address these, we disentangle the traditional encoder-decoder design into an Encoder-Predictor-Decoder (EPD) framework, where the predictor acts as a latent world model, and propose InternVideo-Next, a two-stage pretraining scheme that builds a semantically consistent yet detail-preserving latent space for this world model.First, conventional linear decoder in pixel MVM enforces the predictor's output latent to be linearly projected to, thus separable in pixel space, causing the conflict with semantic abstraction.Our Stage 1 proposes a conditional diffusion decoder and injects clean image-level semantic priors to enhance semantics and convergence, thus bridging pixel-level fidelity with high-level semantic abstraction.Stage 2 further learns world knowledge by predicting frozen Stage 1 targets within this space, mitigating shortcut learning.Trained on public, unlabeled videos, InternVideo-Next achieves state-of-the-art results across benchmarks and provides a scalable path toward general video representation learning.