Diff-SemiER：面向半透明眼镜去除的透明度感知自适应融合扩散模型与生成先验
Diff-SemiER: Transparency-Aware Adaptive Fusion Diffusion Model with Generative Prior for Semi-Transparent Eyeglasses Removal
针对半透明遮挡这一特殊编辑场景，提出软掩码感知的多尺度自适应融合机制，在生成先验与细节复原之间动态权衡；其透射率驱动的数据合成思路可迁移到其他半透明/部分遮挡的复原任务（如墨镜、面纱、玻璃反光去除）。
原文依据 a Soft Mask-Aware Adaptive Fusion (SMAF) mechanism to adaptively merge generative and restorative features across multiple scales
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论现有的眼镜去除方法主要针对不透明或完全透明的镜片。然而，在处理半透明太阳镜时，这些方法往往会破坏镜片下方可见的面部细节，从而降低下游视觉任务的性能。为解决该问题，本文提出 Diff-SemiER，一种新颖的基于扩散模型的半透明眼镜去除框架，利用生成先验与透明度感知自适应融合。该框架充分利用镜片下方可见的眼部区域信息，同时保留足够的生成灵活性，从而在半透明区域内实现生成与复原之间的平衡。具体而言，Diff-SemiER 包含两个扩散分支：生成先验扩散模型（GPDM）通过图像修复生成高质量的无眼镜人脸图像，为高度遮挡场景提供全局语义引导；透明度感知自适应融合扩散模型（TAFDM）采用软掩码感知自适应融合（SMAF）机制，在多个尺度上自适应地融合生成特征与复原特征，从而在不同遮挡程度下实现生成能力与细节保真之间的动态权衡。此外，本文设计了一种基于透射率的数据合成方法，构建了大规模、高质量的半透明眼镜人脸数据集，用于模型训练与评测。大量实验结果表明，Diff-SemiER 在合成场景与真实场景下均显著优于最优方法。
英文原文摘要 1632 字符
Existing eyeglasses removal methods primarily focus on opaque or fully transparent lenses. However, when dealing with semi-transparent sunglasses, these methods often corrupt the visible facial details beneath the lenses, thereby degrading the performance of downstream vision tasks. To address this issue, we propose Diff-SemiER, a novel diffusion-based framework for semi-transparent eyeglasses removal that leverages generative priors and transparency-aware adaptive fusion. The proposed framework fully utilizes the visible eye-region information beneath the lenses while retaining sufficient generative flexibility, thereby striking a balance between generation and restoration within semi-transparent regions. Specifically, Diff-SemiER comprises two diffusion branches: the Generative Prior Diffusion Model (GPDM) generates high-quality eyeglass-free facial images via image inpainting, which provides global semantic guidance for highly occluded scenarios. The Transparency-Aware Adaptive Fusion Diffusion Model (TAFDM) employs a Soft Mask-Aware Adaptive Fusion (SMAF) mechanism to adaptively merge generative and restorative features across multiple scales, enabling dynamic trade-offs between generative capability and fine-detail preservation under varying occlusion levels. Furthermore, we design a transmittance-based data synthesis method to construct a large-scale, high-quality dataset of faces with semi-transparent eyeglasses for model training and evaluation. Extensive experimental results demonstrate that Diff-SemiER significantly outperforms state-of-the-art methods in both synthetic and real-world scenarios.