CustomTex:基于多参考定制的高保真室内场景纹理生成
CustomTex: High-fidelity Indoor Scene Texturing via Multi-Reference Customization
提出双蒸馏框架(语义级+像素级)结合变分分数蒸馏,实现基于参考图像的实例级3D场景纹理生成,可复用于可控3D外观编辑与高保真纹理合成。
原文依据 The core of our method is a dual-distillation approach that separates semantic control from pixel-level enhancement.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论创建高保真、可定制的3D室内场景纹理仍然是一项重大挑战。虽然文本驱动的方法提供了灵活性,但它们缺乏对细粒度、实例级控制的精确性,并且通常生成的纹理质量不足、存在伪影且带有烘焙阴影。为了克服这些限制,我们引入了CustomTex,一种由参考图像驱动的实例级高保真场景纹理生成新框架。CustomTex接收一个未纹理化的3D场景和一组指定每个物体实例所需外观的参考图像,并生成统一的高分辨率纹理贴图。我们方法的核心是一种双蒸馏方法,将语义控制与像素级增强分离。我们采用配备实例交叉注意力的语义级蒸馏来确保语义合理性和“参考-实例”对齐,并采用像素级蒸馏来强制高视觉保真度。两者统一在变分分数蒸馏优化框架内。实验表明,与最优方法相比,CustomTex实现了与参考图像的精确实例级一致性,并生成了具有更优锐度、更少伪影和最少烘焙阴影的纹理。我们的工作为高质量、可定制的3D场景外观编辑建立了一条更直接、更用户友好的路径。
英文原文摘要 1399 字符
The creation of high-fidelity, customizable 3D indoor scene textures remains a significant challenge. While text-driven methods offer flexibility, they lack the precision for fine-grained, instance-level control, and often produce textures with insufficient quality, artifacts, and baked-in shading. To overcome these limitations, we introduce CustomTex, a novel framework for instance-level, high-fidelity scene texturing driven by reference images. CustomTex takes an untextured 3D scene and a set of reference images specifying the desired appearance for each object instance, and generates a unified, high-resolution texture map. The core of our method is a dual-distillation approach that separates semantic control from pixel-level enhancement. We employ semantic-level distillation, equipped with an instances cross attention, to ensure semantic plausibility and "reference-instance" alignment, and pixel-level distillation to enforce high visual fidelity. Both are unified within a Variational Score Distillation optimization framework. Experiments demonstrate that CustomTex achieves precise instance-level consistency with reference images and produces textures with superior sharpness, reduced artifacts, and minimal baked-in shading compared to state-of-the-art methods. Our work establishes a more direct and user-friendly path to high-quality, customizable 3D scene appearance editing.