DreamStereo：面向高清视频的实时立体修复
DreamStereo: Towards Real-Time Stereo Inpainting for HD Videos
Sparsity-Aware Stereo Inpainting 通过稀疏 token 选择把扩散推理的冗余计算削减 70% 以上，这一思路可直接迁移到任何只需局部修改的扩散式图像/视频编辑任务；Parallax-Based Dual Projection 则提供了一种无需真实立体视频即可合成几何一致训练对的构造方式。
原文依据 reduces over 70% of redundant tokens, achieving a 10.7xspeedup during diffusion inference
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论立体视频修复旨在为经视差变换后的视频中的遮挡区域填充视觉上连贯的内容，同时保持时序一致性，目前仍是一个具有挑战性的开放问题。待填充区域散布在物体边界附近，且仅占每帧的一小部分，这带来两个关键挑战。第一，由于高质量立体修复数据集稀缺，现有方法难以学习有效的修复先验，因而在此类任务上表现不佳。第二，这些方法对帧内所有区域施加同等处理，尽管大多数像素并不需要修改，从而造成大量冗余计算。为解决上述问题，我们引入三个相互关联的组件。首先提出梯度感知视差变换（Gradient-Aware Parallax Warping, GAPW），利用反向变换与坐标映射函数的梯度获得连续的边缘和平滑的遮挡区域。随后引入基于视差的双投影（Parallax-Based Dual Projection, PBDP）策略，该策略结合 GAPW，在无需立体视频的情况下生成几何一致的立体修复图像对与准确的遮挡掩码。最后提出稀疏感知立体修复（Sparsity-Aware Stereo Inpainting, SASI），减少超过 70% 的冗余 token，在扩散推理阶段实现 10.7 倍加速，并取得与全计算版本相当的结果，从而在单张 A100 GPU 上以 25 FPS 实时处理高清（768x1280）视频。
英文原文摘要 1487 字符
Stereo video inpainting, which aims to fill the occluded regions of warped videos with visually coherent content while maintaining temporal consistency, remains a challenging open problem. The regions to be filled are scattered along object boundaries and occupy only a small fraction of each frame, leading to two key challenges. First, existing approaches perform poorly on such tasks due to the scarcity of high-quality stereo inpainting datasets, which limits their ability to learn effective inpainting priors. Second, these methods apply equal processing to all regions of the frame, even though most pixels require no modification, resulting in substantial redundant computation. To address these issues, we introduce three interconnected components. We first propose Gradient-Aware Parallax Warping (GAPW), which leverages backward warping and the gradient of the coordinate mapping function to obtain continuous edges and smooth occlusion regions. Then, a Parallax-Based Dual Projection (PBDP) strategy is introduced, which incorporates GAPW to produce geometrically consistent stereo inpainting pairs and accurate occlusion masks without requiring stereo videos. Finally, we present Sparsity-Aware Stereo Inpainting (SASI), which reduces over 70% of redundant tokens, achieving a 10.7xspeedup during diffusion inference and delivering results comparable to its full-computation counterpart, enabling real-time processing of HD (768x1280) videos at 25\,FPS on a single A100 GPU.