当线条遇上纹理：面向跨稀疏度对应的空间-频率对齐扩散特征
When Lines Meet Textures: Spatial-Frequency Aligned Diffusion Features for Cross-Sparsity Correspondence
把扩散模型中间层特征当作免训练的描述子来做稀疏线条图与纹理照片之间的匹配，并指出仅做空间对齐不够、需在频域聚合低频分量以抹平纹理密度差异——这套双域对齐思路可直接迁移到草图检索、跨模态配准、线稿上色等需要跨越抽象层级差异的匹配任务。
原文依据 our key innovation performs dual-domain alignment by learning unified clean diffusion features while strategically aggregating low-frequency components in the frequency domain
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论在稀疏线条表示与丰富纹理图像之间建立准确对应关系仍是一项艰巨挑战。尽管扩散特征在语义对应上表现优异，却难以弥合抽象草图与纹理丰富照片之间的根本鸿沟。我们识别出两处关键差异：结构抽象差异导致的空间域错位，以及纹理密度变化导致的频域不一致。基于这一分析，我们提出 SFA-DIFT，一种学习空间-频率对齐扩散特征以实现鲁棒跨模态对应的新方法。与以往仅关注空间对齐的方法不同，我们的核心创新在于执行双域对齐：在学习统一的干净扩散特征的同时，于频域中策略性地聚合低频分量。这种全面的空间-频率对齐使稀疏抽象与丰富纹理之间能够获得对等的理解。为验证该方法，我们扩展了现有的草图-照片对应数据集（PSC6K），通过生成多风格纹理图像构建了 MS-PSC6K，一个全面的对应基准。大量实验表明，SFA-DIFT 取得了最优性能，相较此前最佳方法在 PCK@1、PCK@5 和 PCK@10 上分别平均提升 0.87%、2.20% 和 0.95%，验证了双域对齐方法的有效性与鲁棒性。代码已发布于 https://github.com/Mofr77/SFA-DIFT。
英文原文摘要 1551 字符
Establishing accurate correspondence between sparse line representations and rich textured imagery remains a formidable challenge. While diffusion features excel in semantic correspondence, they struggle to bridge the fundamental gap between abstract sketches and texture-rich photographs. We identify two critical disparities: spatial domain misalignment from structural abstraction differences, and frequency domain inconsistencies from texture density variations. Based on this analysis, we propose SFA-DIFT, a novel approach that learns spatial-frequency aligned diffusion features for robust cross-modal correspondence. Unlike previous methods focusing solely on spatial alignment, our key innovation performs dual-domain alignment by learning unified clean diffusion features while strategically aggregating low-frequency components in the frequency domain. This comprehensive spatial-frequency alignment enables equitable understanding between sparse abstractions and rich textures. To validate our approach, we extend the existing sketch-photo correspondence dataset (PSC6K) by generating multi-style textured imagery, creating MS-PSC6K, a comprehensive correspondence benchmark. Extensive experiments demonstrate that SFA-DIFT achieves state-of-the-art performance, delivering substantial improvements with an average of 0.87% on PCK@1, 2.20% on PCK@5, and 0.95% on PCK@10 over previous best methods, validating the effectiveness and robustness of our dual-domain alignment approach. Codes are available on https://github.com/Mofr77/SFA-DIFT.