TextOVSR:文本引导的真实世界戏曲视频超分辨率
TextOVSR: Text-Guided Real-World Opera Video Super-Resolution
用退化描述文本作为负分支约束、内容描述文本作为正分支与判别器语义引导,为真实世界视频超分提供了一种可复用的文本条件化退化建模与跨模态融合思路,可直接迁移到其他低质档案视频复原任务。
原文依据 we propose a Text-guided Dual-Branch Opera Video Super-Resolution (TextOVSR) network
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论许多经典戏曲视频由于早期拍摄设备的限制以及存储过程中的长期退化,呈现出较差的视觉质量。尽管真实世界视频超分辨率(RWVSR)近年来取得了显著进展,但直接将现有方法应用于退化戏曲视频仍然面临挑战。困难主要体现在两方面。首先,准确建模真实世界退化十分复杂:经典退化核的简单组合无法刻画真实的噪声分布,而从外部数据集提取真实噪声块的方法又容易因风格不匹配而引入视觉伪影。其次,当前仅依赖退化图像特征的 RWVSR 方法由于缺乏高层语义引导,难以重建真实且细节丰富的纹理。为解决上述问题,本文提出文本引导的双分支戏曲视频超分辨率网络(TextOVSR),引入两类文本提示来引导超分辨率过程。具体而言,由退化过程得到的退化描述文本被引入负分支以约束解空间;同时,内容描述文本被引入正分支以及本文提出的文本增强判别器(TED),为增强纹理重建提供语义引导。此外,本文设计了退化鲁棒特征融合(DRF)模块,在抑制退化干扰的同时促进跨模态特征融合。在 OperaLQ 基准上的实验表明,TextOVSR 在定性和定量两方面均优于最优方法。代码已开源,地址为 https://github.com/ChangHua0/TextOVSR。
英文原文摘要 1742 字符
Many classic opera videos exhibit poor visual quality due to the limitations of early filming equipment and long-term degradation during storage. Although real-world video super-resolution (RWVSR) has achieved significant advances in recent years, directly applying existing methods to degraded opera videos remains challenging. The difficulties are twofold. First, accurately modeling real-world degradations is complex: simplistic combinations of classical degradation kernels fail to capture the authentic noise distribution, while methods that extract real noise patches from external datasets are prone to style mismatches that introduce visual artifacts. Second, current RWVSR methods, which rely solely on degraded image features, struggle to reconstruct realistic and detailed textures due to a lack of high-level semantic guidance. To address these issues, we propose a Text-guided Dual-Branch Opera Video Super-Resolution (TextOVSR) network, which introduces two types of textual prompts to guide the super-resolution process. Specifically, degradation-descriptive text, derived from the degradation process, is incorporated into the negative branch to constrain the solution space. Simultaneously, content-descriptive text is incorporated into a positive branch and our proposed Text-Enhanced Discriminator (TED) to provide semantic guidance for enhanced texture reconstruction. Furthermore, we design a Degradation-Robust Feature Fusion (DRF) module to facilitate cross-modal feature fusion while suppressing degradation interference. Experiments on our OperaLQ benchmark show that TextOVSR outperforms state-of-the-art methods both qualitatively and quantitatively. The code is available at https://github.com/ChangHua0/TextOVSR.