NaTex：作为潜变量颜色扩散的无缝纹理生成
NaTex: Seamless Texture Generation as Latent Color Diffusion
把纹理表示为稠密颜色点云，并用几何感知的点云 VAE 加多控制 DiT 直接在 3D 空间扩散生成颜色，绕开了多视图扩散烘焙带来的遮挡修补、边界对齐和跨视图一致性问题；做 3D 资产纹理、材质生成或网格贴图的人可以直接复用这套潜空间颜色扩散与几何潜变量对齐机制。
原文依据 NaTex features a novel paradigm that addresses the aforementioned issues by viewing texture as a dense color point cloud.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论本文提出 NaTex，一种直接在三维空间中预测纹理颜色的原生纹理生成框架。与以往依赖烘焙由几何条件多视图扩散模型（MVD）合成的二维多视图图像的方法不同，NaTex 规避了 MVD 流程的若干固有局限，包括难以处理需要修补的遮挡区域、难以沿边界实现精确的网格与纹理对齐，以及难以在内容和颜色强度上保持跨视图的一致性与连贯性。NaTex 提出一种新范式，将纹理视为稠密的颜色点云，从而解决上述问题。在这一思路驱动下，本文提出潜变量颜色扩散，它由一个几何感知的颜色点云变分自编码器和一个多控制扩散 Transformer（DiT）组成，完全使用三维数据从零训练，用于纹理重建与生成。为实现精确对齐，本文引入原生几何控制，通过位置嵌入和几何潜变量使 DiT 以直接的三维空间信息为条件。本文对 VAE-DiT 架构进行协同设计，其中几何潜变量由与颜色 VAE 紧密耦合的专用几何分支提取，提供细粒度的表面引导，并与纹理保持强对应关系。凭借这些设计，NaTex 展现出强劲性能，在纹理连贯性和对齐方面显著优于以往方法。此外，NaTex 还表现出很强的泛化能力，无论免训练还是经过简单微调，均可用于多种下游应用，例如材质生成、纹理细化和部件分割与纹理化。
英文原文摘要 1702 字符
We present NaTex, a native texture generation framework that predicts texture color directly in 3D space. In contrast to previous approaches that rely on baking 2D multi-view images synthesized by geometry-conditioned Multi-View Diffusion models (MVDs), NaTex avoids several inherent limitations of the MVD pipeline. These include difficulties in handling occluded regions that require inpainting, achieving precise mesh-texture alignment along boundaries, and maintaining cross-view consistency and coherence in both content and color intensity. NaTex features a novel paradigm that addresses the aforementioned issues by viewing texture as a dense color point cloud. Driven by this idea, we propose latent color diffusion, which comprises a geometry-awared color point cloud VAE and a multi-control diffusion transformer (DiT), entirely trained from scratch using 3D data, for texture reconstruction and generation. To enable precise alignment, we introduce native geometry control that conditions the DiT on direct 3D spatial information via positional embeddings and geometry latents. We co-design the VAE-DiT architecture, where the geometry latents are extracted via a dedicated geometry branch tightly coupled with the color VAE, providing fine-grained surface guidance that maintains strong correspondence with the texture. With these designs, NaTex demonstrates strong performance, significantly outperforming previous methods in texture coherence and alignment. Moreover, NaTex also exhibits strong generalization capabilities, either training-free or with simple tuning, for various downstream applications, e.g., material generation, texture refinement, and part segmentation and texturing.