压缩域感知的在线视频超分辨率
Compressed-Domain-Aware Online Video Super-Resolution
把压缩码流里的运动矢量、残差图、帧类型当作免费先验来替代昂贵的运动估计：运动矢量做粗对齐、网络只学局部残差偏移，可显著降低在线视频超分辨率的对齐开销，这套「压缩域引导对齐+按帧类型分配算力」的思路可直接迁移到其他视频复原/增强任务。
原文依据 a motion-vector-guided deformable alignment module that uses motion vectors for coarse warping and learns only local residual offsets
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论在带宽受限的在线视频流场景中，视频通常会被下采样并压缩。尽管近期的在线视频超分辨率（online VSR）方法取得了有前景的结果，但它们仍然计算密集，且在高分辨率下难以达到实时处理，原因在于用于对齐的复杂运动估计以及对连续帧的冗余处理。为解决这些问题，我们提出了一种用于在线视频超分辨率的压缩域感知网络（CDA-VSR），它利用压缩域信息（包括运动矢量、残差图和帧类型）来平衡质量与效率。具体而言，我们提出了一种运动矢量引导的可变形对齐模块，该模块利用运动矢量进行粗变形，并仅学习局部残差偏移以进行微调，从而在保持精度的同时降低计算量。随后，我们利用残差图门控融合模块从残差图中导出空间权重，抑制不匹配区域并强调可靠细节。此外，我们设计了一种帧类型感知的重建模块，用于在不同帧类型之间自适应分配计算量，从而平衡精度与效率。在 REDS4 数据集上，我们的 CDA-VSR 超越了最优方法 TMP，峰值信噪比最高提升 0.13 dB，同时推理速度提升一倍以上。代码将发布于 https://github.com/sspBIT/CDA-VSR。
英文原文摘要 1405 字符
In bandwidth-limited online video streaming, videos are usually downsampled and compressed. Although recent online video super-resolution (online VSR) approaches achieve promising results, they are still compute-intensive and fall short of real-time processing at higher resolutions, due to complex motion estimation for alignment and redundant processing of consecutive frames. To address these issues, we propose a compressed-domain-aware network (CDA-VSR) for online VSR, which utilizes compressed-domain information, including motion vectors, residual maps, and frame types to balance quality and efficiency. Specifically, we propose a motion-vector-guided deformable alignment module that uses motion vectors for coarse warping and learns only local residual offsets for fine-tuned adjustments, thereby maintaining accuracy while reducing computation. Then, we utilize a residual map gated fusion module to derive spatial weights from residual maps, suppressing mismatched regions and emphasizing reliable details. Further, we design a frame-type-aware reconstruction module for adaptive compute allocation across frame types, balancing accuracy and efficiency. On the REDS4 dataset, our CDA-VSR surpasses the state-of-the-art method TMP, with a maximum PSNR improvement of 0.13 dB while delivering more than double the inference speed. The code will be released at https://github.com/sspBIT/CDA-VSR.