PackUV:用于 4D 体积视频的打包高斯 UV 图集
PackUV: Packed Gaussian UV Maps for 4D Volumetric Video
把全部高斯属性烘焙进多尺度 UV 图集,使 4D 高斯表示可直接走 HEVC/FFV1 等标准视频编解码器,为体积视频的存储与流式传输提供了图像原生的表示思路;其流引导高斯标注与关键帧机制也可复用于大运动与遮挡下的动态高斯拟合。
原文依据 maps all Gaussian attributes into a sequence of structured, multi-scale UV atlas, enabling compact, image-native storage
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论体积视频能带来沉浸式 4D 体验,但在大规模场景下仍难以重建、存储与流式传输。现有的基于高斯泼溅的方法虽能实现高质量重建,却在长序列上失效,存在时序不一致问题,并在大运动与遮挡消失情形下失败。此外,它们的输出通常与常规视频编码流水线不兼容,阻碍了实际应用。我们提出 PackUV,一种新颖的 4D 高斯表示,将所有高斯属性映射为一系列结构化的多尺度 UV 图集,从而实现紧凑的、图像原生的存储。为从多视角视频拟合该表示,我们提出 PackUV-GS,一种时序一致的拟合方法,直接在 UV 域中优化高斯参数。一个流引导的高斯标注与视频关键帧模块可识别动态高斯、稳定静态区域,并在大运动与遮挡消失下仍保持时序一致性。所得的 UV 图集格式是首个与标准视频编解码器(如 HEVC、FFV1)兼容且不损失质量的统一体积视频表示,使得在现有多媒体基础设施内进行高效流式传输成为可能。为评估长时体积捕捉,我们提出 PackUV-2B,这是迄今最大的多视角 4D 数据集,包含 50 多台同步的 360 度相机、显著的运动会以及 100 个序列、20 亿帧中频繁出现的遮挡消失。大量实验表明,我们的方法在渲染保真度上超越现有基线,同时可扩展到长达 30 分钟的序列并保持一致的画质。
英文原文摘要 1636 字符
Volumetric videos offer immersive 4D experiences, but remain difficult to reconstruct, store, and stream at scale. Existing Gaussian Splatting based methods achieve high-quality reconstruction but break down on long sequences, temporal inconsistency, and fail under large motions and disocclusions. Moreover, their outputs are typically incompatible with conventional video coding pipelines, preventing practical applications. We introduce PackUV, a novel 4D Gaussian representation that maps all Gaussian attributes into a sequence of structured, multi-scale UV atlas, enabling compact, image-native storage. To fit this representation from multi-view videos, we propose PackUV-GS, a temporally consistent fitting method that directly optimizes Gaussian parameters in the UV domain. A flow-guided Gaussian labeling and video keyframing module identifies dynamic Gaussians, stabilizes static regions, and preserves temporal coherence even under large motions and disocclusions. The resulting UV atlas format is the first unified volumetric video representation compatible with standard video codecs (e.g., HEVC, FFV1) without losing quality, enabling efficient streaming within existing multimedia infrastructure. To evaluate long-duration volumetric capture, we present PackUV-2B, the largest multi-view 4D dataset to date, featuring more than 50 synchronized 360 cameras, substantial motion, and frequent disocclusions across 100 sequences and 2B (billion) frames. Extensive experiments demonstrate that our method surpasses existing baselines in rendering fidelity while scaling to sequences up to 30 minutes with consistent quality.