通用正态嵌入
The Universal Normal Embedding
把 DDIM 反演得到的扩散噪声当作可线性探测的语义潜变量,用线性探针与正交化方向实现无需改架构的可控属性编辑;这为视觉研究者提供了一条把生成噪声与 CLIP/DINO 嵌入对齐、复用于编辑与表征分析的具体路径。
原文依据 linear probes in both spaces yield strong, aligned attribute predictions
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论生成模型与视觉编码器基本沿着各自独立的路线发展,优化目标不同,所依据的数学原理也不同。然而它们共享一个基本性质:潜空间高斯性。生成模型将高斯噪声映射为图像,而编码器将图像映射为语义嵌入,其坐标在经验上表现为高斯分布。我们假设二者是同一潜在来源的两个视图,即通用正态嵌入(Universal Normal Embedding, UNE):一个近似高斯的潜空间,编码器嵌入与 DDIM 反演噪声都是它经过带噪线性投影后的产物。为检验该假设,我们提出 NoiseZoo,一个逐图像潜变量数据集,包含 DDIM 反演的扩散噪声以及与之匹配的编码器表示(CLIP、DINO)。在 CelebA 上,两个空间中的线性探针均给出强且一致的属性预测,表明生成噪声沿线性方向编码了有意义的语义。这些方向进一步支持忠实的可控编辑(如微笑、性别、年龄)而无需改动架构,其中简单的正交化可缓解虚假的纠缠。综上,我们的结果为 UNE 假设提供了经验支持,并揭示出一种共享的类高斯潜几何,将编码与生成具体地联系起来。
英文原文摘要 1301 字符
Generative models and vision encoders have largely advanced on separate tracks, optimized for different goals and grounded in different mathematical principles. Yet, they share a fundamental property: latent space Gaussianity. Generative models map Gaussian noise to images, while encoders map images to semantic embeddings whose coordinates empirically behave as Gaussian. We hypothesize that both are views of a shared latent source, the Universal Normal Embedding (UNE): an approximately Gaussian latent space from which encoder embeddings and DDIM-inverted noise arise as noisy linear projections. To test our hypothesis, we introduce NoiseZoo, a dataset of per-image latents comprising DDIM-inverted diffusion noise and matching encoder representations (CLIP, DINO). On CelebA, linear probes in both spaces yield strong, aligned attribute predictions, indicating that generative noise encodes meaningful semantics along linear directions. These directions further enable faithful, controllable edits (e.g., smile, gender, age) without architectural changes, where simple orthogonalization mitigates spurious entanglements. Taken together, our results provide empirical support for the UNE hypothesis and reveal a shared Gaussian-like latent geometry that concretely links encoding and generation.