生成模型与扩散 · CVPR2026 ·2026.06.03

CubeComposer:从透视视频时空自回归生成 4K 360 度视频

CubeComposer: Spatio-Temporal Autoregressive 4K 360deg Video Generation from Perspective Video

Lingen Li、Guangzhi Wang、Xiaoyu Li、Zhaoyang Zhang、Qi Dou、Jinwei Gu 等 8 人

与你有关的部分

把高分辨率全景视频生成拆成立方体贴图六面 + 时间窗的自回归序列,配合稀疏上下文注意力与立方体感知位置编码,为在有限显存下原生生成超高分辨率视频提供了可复用的时空自回归分解范式。

原文依据 a novel spatio-temporal autoregressive diffusion model that natively generates 4K-resolution 360deg videos
可迁移到 全景视频生成视频超分辨率自回归视频生成虚拟现实内容生成

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

从透视输入生成高质量 360 度全景视频是虚拟现实(VR)的关键应用之一,其中高分辨率视频对沉浸式体验尤为重要。现有方法受限于原始扩散模型的计算约束,仅支持不超过 1K 分辨率的原生生成,并依赖次优的后处理超分辨率。本文提出 CubeComposer,一种原生生成 4K 分辨率 360 度视频的新型时空自回归扩散模型。通过将视频分解为包含六个面的立方体贴图表示,CubeComposer 以精心规划的时空顺序自回归地合成内容,在降低峰值内存需求的同时实现高分辨率输出。具体而言,为应对多维自回归中的挑战,本文提出:(1)时空自回归策略,在立方体各面与时间窗之间统筹 360 度视频生成以实现连贯合成;(2)立方体面上下文管理机制,配备稀疏上下文注意力设计以提升效率;(3)连续性感知技术,包括立方体感知位置编码、填充与混合,以消除边界接缝。在基准数据集上的大量实验表明,CubeComposer 在原生分辨率与视觉质量上优于最优方法,可支撑实际 VR 应用场景。

英文原文摘要 1447 字符

Generating high-quality 360deg panoramic videos from perspective input is one of the crucial applications for virtual reality (VR), whereby high-resolution videos are especially important for immersive experience. Existing methods are constrained by computational limitations of vanilla diffusion models, only supporting <= 1K resolution native generation and relying on suboptimal post-hoc super-resolution. We introduce CubeComposer, a novel spatio-temporal autoregressive diffusion model that natively generates 4K-resolution 360deg videos. By decomposing videos into cubemap representations with six faces, CubeComposer autoregressively synthesizes content in a well-planned spatio-temporal order, reducing peak memory demands while enabling high-resolution output. Specifically, to address challenges in multi-dimensional autoregression, we propose: (1) a spatio-temporal autoregressive strategy that orchestrates 360deg video generation across cube faces and time windows for coherent synthesis; (2) a cube face context management mechanism, equipped with a sparse context attention design to improve efficiency; and (3) continuity-aware techniques, including cube-aware positional encoding, padding, and blending to eliminate boundary seams. Extensive experiments on benchmark datasets demonstrate that CubeComposer outperforms state-of-the-art methods in native resolution and visual quality, supporting practical VR application scenarios.

标签

会议 CVPR2026
页码 32625-32635
Bibkey
来源 CVF Open Access