DMAligner：通过基于扩散模型的视图合成增强图像对齐
DMAligner: Enhancing Image Alignment via Diffusion Model Based View Synthesis
用扩散模型做面向对齐的新视图合成，替代传统光流 warping，能处理遮挡和光照变化；DMP 模块动态区分前景背景，思路可迁移到其他需要条件生成的视觉任务。
原文依据 we present DMAligner, a diffusion-based framework for image alignment through alignment-oriented view synthesis
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论图像对齐是计算机视觉中的一项基础任务，具有广泛的应用。现有方法主要采用基于光流的图像扭曲。然而，该技术容易受到遮挡和光照变化等常见挑战的影响，导致对齐视觉质量下降，并损害下游任务的准确性。本文提出 DMAligner，一个通过面向对齐的视图合成实现图像对齐的扩散框架。DMAligner 旨在从新视角应对图像对齐中的挑战，采用基于生成的解决方案，展现出强大能力，并避免了基于流的图像扭曲相关问题。具体而言，我们提出一种动态感知扩散训练方法，用于学习条件图像生成，合成用于图像对齐的新视图。该方法包含一个动态感知掩码生成（DMP）模块，自适应地区分动态前景区域与静态背景，使扩散模型能够更有效地处理经典方法难以解决的挑战。此外，我们使用 Blender 开发了动态场景图像对齐（DSIA）数据集，包含 1,033 个室内外场景和超过 30K 个专为图像对齐设计的图像对。大量实验结果表明，所提方法在 DSIA 基准以及一系列广泛使用的视频数据集上在定性比较中具有优越性。我们的代码可在 https://github.com/boomluo02/DMAligner 获取。
英文原文摘要 1560 字符
Image alignment is a fundamental task in computer vision with broad applications. Existing methods predominantly employ optical flow-based image warping. However, this technique is susceptible to common challenges such as occlusions and illumination variations, leading to degraded alignment visual quality and compromised accuracy in downstream tasks. In this paper, we present DMAligner, a diffusion-based framework for image alignment through alignment-oriented view synthesis. DMAligner is crafted to tackle the challenges in image alignment from a new perspective, employing a generation-based solution that showcases strong capabilities and avoids the problems associated with flow-based image warping. Specifically, we propose a Dynamics-aware Diffusion Training approach for learning conditional image generation, synthesizing a novel view for image alignment. This incorporates a Dynamics-aware Mask Producing (DMP) module to adaptively distinguish dynamic foreground regions from static backgrounds, enabling the diffusion model to more effectively handle challenges that classical methods struggle to solve. Furthermore, we develop the Dynamic Scene Image Alignment (DSIA) dataset using Blender, which includes 1,033 indoor and outdoor scenes with over 30K image pairs tailored for image alignment. Extensive experimental results demonstrate the superiority of the proposed approach on DSIA benchmarks, as well as on a series of widely-used video datasets for qualitative comparisons. Our code is available at https://github.com/boomluo02/DMAligner.