面向多视角扩散模型的对应注意力对齐
Correspondence-Attention Alignment for Multi-View Diffusion Models
把跨视角几何对应作为注意力图的显式监督信号，只需监督单层注意力即可显著加速多视角扩散模型收敛并提升新视角合成质量，这一思路可直接迁移到任何以参考图为条件的扩散生成/编辑模型中，用于增强结构保持与视角一致性。
原文依据 supervising a single attention layer is sufficient to guide the model toward learning precise correspondences
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多视角扩散模型近来已成为新视角合成的一种强大范式，然而其实现视角一致性的内在机制仍不清楚。在本工作中，我们首先验证了这些模型的注意力图在训练过程中会获得几何对应关系，即在参考视角与目标视角之间关注几何上对应的区域，从而实现视角一致的生成。然而，这种对应信号并不完整，其精度在视角变化较大时会下降。基于这些发现，我们提出了 CAMEO，一种简单而有效的训练技术，它利用几何对应直接监督注意力图，以同时提升多视角扩散模型的训练效率与生成质量。值得注意的是，仅监督单个注意力层就足以引导模型学习精确的对应关系，从而保持参考图像的几何与结构、加速收敛并提升新视角合成性能。CAMEO 将收敛所需的训练迭代次数减半，同时在相同迭代次数下取得更优性能。我们进一步证明 CAMEO 与模型无关，可应用于任意多视角扩散模型。项目主页见 https://cvlab-kaist.github.io/CAMEO/。
英文原文摘要 1376 字符
Multi-view diffusion models have recently emerged as a powerful paradigm for novel view synthesis, yet the underlying mechanism that enables their view consistency remains unclear. In this work, we first verify that the attention maps of these models acquire geometric correspondence throughout training, attending to the geometrically corresponding regions across reference and target views for view-consistent generation. However, this correspondence signal remains incomplete, with its accuracy degrading under large viewpoint changes. Building on these findings, we introduce CAMEO, a simple yet effective training technique that directly supervises attention maps using geometric correspondence to enhance both the training efficiency and generation quality of multi-view diffusion models. Notably, supervising a single attention layer is sufficient to guide the model toward learning precise correspondences, thereby preserving the geometry and structure of reference images, accelerating convergence, and improving novel view synthesis performance. CAMEO reduces the number of training iterations required for convergence by half while achieving superior performance at the same iteration counts. We further demonstrate that CAMEO is model-agnostic and can be applied to any multi-view diffusion model. Project page is available at https://cvlab-kaist.github.io/CAMEO/.