Face2Scene:以人脸退化为先验的扩散式场景复原
Face2Scene: Using Facial Degradation as an Oracle for Diffusion-Based Scene Restoration
把已复原的人脸与退化人脸构成配对,从中提取退化编码,再转成多尺度 token 作为扩散模型的条件,从而用单步扩散复原整幅场景;这一「以局部高质量复原结果反推退化属性、再引导全局复原」的思路,可迁移到其他需要显式退化建模的复原任务。
原文依据 we extract a face-derived degradation code that captures degradation attributes (e.g., noise, blur, compression)
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近年来图像复原的进展使得基于参考的人脸复原模型(Ref-FR)能够从退化输入中高保真地恢复人脸。然而,这类方法只关注人脸区域,忽略了包括人体与背景在内的整个场景的退化,限制了实际可用性。与此同时,全场景复原器往往完全忽略退化线索,导致预测欠定并产生视觉伪影。在本工作中,我们提出 Face2Scene,一个两阶段复原框架,将人脸作为感知先验来估计退化并引导整幅图像的复原。给定一张退化图像和一个或多个身份参考,我们首先应用 Ref-FR 模型重建高质量的人脸细节。由复原后的人脸与退化人脸构成的配对中,我们提取出人脸导出的退化编码,用以刻画噪声、模糊、压缩等退化属性,随后将其转换为多尺度的退化感知 token。这些 token 作为扩散模型的条件,以单步方式复原包括人体与背景在内的整个场景。大量实验表明,所提方法相比最优方法具有更优的有效性。
英文原文摘要 1246 字符
Recent advances in image restoration have enabled high-fidelity recovery of faces from degraded inputs using reference-based face restoration models (Ref-FR). However, such methods focus solely on facial regions, neglecting degradation across the full scene, including body and background, which limits practical usability. Meanwhile, full-scene restorers often ignore degradation cues entirely, leading to underdetermined predictions and visual artifacts. In this work, we propose **Face2Scene**, a two-stage restoration framework that leverages the face as a perceptual oracle to estimate degradation and guide the restoration of the entire image. Given a degraded image and one or more identity references, we first apply a Ref-FR model to reconstruct high-quality facial details. From the restored-degraded face pair, we extract a face-derived degradation code that captures degradation attributes (e.g., noise, blur, compression), which is then transformed into multi-scale degradation-aware tokens. These tokens condition a diffusion model to restore the full scene in a single step, including the body and background. Extensive experiments demonstrate the superior effectiveness of the proposed method compared to state-of-the-art methods.