MatLat：面向 PBR 纹理生成的材质潜空间
MatLat: Material Latent Space for PBR Texture Generation
把预训练潜空间图像扩散模型迁移到多通道 PBR 材质生成时，关键是在 VAE 微调阶段保持潜变量分布与像素-潜变量空间对应关系；其「裁剪潜块解码并对齐图像区域」的正则化思路可直接复用于任何需要扩展通道数或保持潜空间局部性的生成任务。
原文依据 we introduce a regularization in the VAE fine-tuning that crops latent patches, decodes them, and aligns the corresponding image regions
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论我们提出了一个在给定三维网格上生成高质量 PBR 纹理的生成框架。由于大规模 PBR 纹理数据集稀缺，我们的方法侧重于有效利用预训练潜空间图像生成模型的嵌入空间与扩散先验，同时通过有针对性的微调学习一个材质潜空间 MatLat。已有方法冻结嵌入网络，这会在编码额外 PBR 通道时导致分布偏移，并阻碍后续的扩散训练；与之不同，我们微调预训练的变分自编码器，使新的材质通道能够在潜变量分布偏差极小的情况下被纳入。我们进一步表明，仅靠对应关系感知注意力不足以实现跨视角一致性，除非潜变量到图像的映射保持局部性。为强制实现这种局部性，我们在变分自编码器微调中引入一种正则化：裁剪潜变量块、将其解码，并对相应的图像区域进行对齐，以维持强的像素-潜变量空间对应关系。消融实验以及与先前基线的比较表明，我们的框架提升了 PBR 纹理的保真度，且每个组件对于达到最优性能都至关重要。项目主页见 https://matlat-proj.github.io。
英文原文摘要 1286 字符
We propose a generative framework for producing high-quality PBR textures on a given 3D mesh. As large-scale PBR texture datasets are scarce, our approach focuses on effectively leveraging the embedding space and diffusion priors of pretrained latent image generative models while learning a material latent space, MatLat, through targeted fine-tuning. Unlike prior methods that freeze the embedding network, which leads to distribution shifts when encoding additional PBR channels and hinders subsequent diffusion training, we fine-tune the pretrained VAE so that new material channels can be incorporated with minimal latent distribution deviation. We further show that correspondence-aware attention alone is insufficient for cross-view consistency unless the latent-to-image mapping preserves locality. To enforce this locality, we introduce a regularization in the VAE fine-tuning that crops latent patches, decodes them, and aligns the corresponding image regions to maintain strong pixel-latent spatial correspondence. Ablation studies and comparison with previous baselines demonstrate that our framework improves PBR texture fidelity and that each component is critical for achieving state-of-the-art performance. Our project page is available at https://matlat-proj.github.io.