VOSR：一种仅视觉的生成式图像超分辨率模型
VOSR: A Vision-Only Generative Model for Image Super-Resolution
提出仅用视觉编码器注入语义、无需文本先验的扩散超分框架，并采用改进的 shortcut 模型实现单步推理，为低层视觉任务摆脱多模态预训练依赖提供了可复用的训练与加速方案。
原文依据 we leverage a pretrained vision encoder to inject semantic cues, and introduce a relaxed unconditional objective
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论大规模预训练的文本到图像（T2I）扩散模型（如 Stable Diffusion）可通过微调实现具有高度逼真细节的图像超分辨率（SR）。尽管效果显著，但预训练此类多模态模型需要数十亿高质量文本-图像对和大量计算资源，而超分辨率本质上是一个图像到图像（I2I）任务。这引出一个关键问题：我们是否真的需要多模态先验和十亿级文本-图像数据来解决一个纯视觉任务？本文提出 VOSR，一种仅视觉的超分辨率框架，无需文本先验和多模态预训练。我们指出先前基于图像的单模态扩散模型存在两个关键局限：视觉语义引导有限和无条件训练不稳定。为此，我们利用预训练视觉编码器注入语义线索，并引入一种宽松的无条件目标，部分利用低质量条件来稳定训练。为加速推理，我们采用改进的 shortcut 模型实现单步超分辨率，且质量下降极小。与基于 T2I 的扩散模型相比，VOSR 从零开始训练，所需数据和计算成本显著降低。然而，VOSR 在合成和真实世界基准上达到与最优 T2I 微调超分辨率方法相当甚至更好的性能，展示了其作为可扩展且有竞争力的生成式超分辨率替代方案的潜力。代码和模型将公开。
英文原文摘要 1553 字符
Large-scale pre-trained text-to-image (T2I) diffusion models, such as Stable Diffusion, can be finetuned for image super-resolution (SR) with highly realistic details. While impressive, pre-training such multi-modal models demands billions of high-quality text-image pairs and substantial computational resources, despite that SR is fundamentally an image-to-image (I2I) task. This raises a critical question: do we truly need multi-modal priors and billion-scale text-image data to solve a purely vision task? In this paper, we propose **VOSR**, a **V**ision-**O**nly **S**uper-**R**esolution framework that eliminates the need for textual priors and multi-modal pretraining. We identify two key limitations in previous image-based, uni-modal diffusion models: limited visual semantic guidance and unstable unconditional training. To this end, we leverage a pretrained vision encoder to inject semantic cues, and introduce a relaxed unconditional objective that partially uses the low-quality condition to stabilize training. To accelerate inference, we adopt a modified shortcut model for one-step SR with minimal quality degradation. VOSR is trained from scratch with significantly less data and a lower computational cost compared to T2I-based diffusion models. However, VOSR achieves comparable or even better performance than state-of-the-art T2I-tuned SR methods on both synthetic and real-world benchmarks, demonstrating its potential as a scalable and competitive alternative for generative SR. Codes and models will be made publicly available.