Photo3D：通过结构对齐的细节增强推进照片级真实感三维生成
Photo3D: Advancing Photorealistic 3D Generation through Structure-Aligned Detail Enhancement
用 GPT-4o-Image 生成的多视图图像作为真实细节先验，通过结构对齐的多视图合成管线与感知特征适配、语义结构匹配，把细节注入 3D-native 生成器的几何之上；对做 3D 生成的人，这是一套可直接挂到现有几何生成器后面的纹理/外观增强方案，并给出了耦合与解耦两种范式的训练策略。
原文依据 we present a realistic detail enhancement scheme that leverages perceptual feature adaptation and semantic structure matching
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论尽管近期的三维原生生成器在合成可靠几何方面取得了很大进展，它们在实现逼真外观上仍显不足。一个关键障碍在于缺乏具有丰富表面细节的多样、高质量真实世界三维资产，因为受场景尺度多样、物体非刚性运动以及扫描仪精度有限的影响，采集此类数据本身就很困难。我们提出 Photo3D，一个由 GPT-4o-Image 模型生成的图像数据驱动的、旨在推进照片级真实感三维生成的框架。考虑到生成的图像可能因缺乏多视图一致性而扭曲三维结构，我们设计了一条结构对齐的多视图合成管线，并构建了与三维几何配对、细节增强的多视图数据集。在此基础上，我们提出一种真实细节增强方案，利用感知特征适配与语义结构匹配，在保持与三维原生几何的结构一致性的同时，强制外观与真实细节先验保持一致。虽然我们的方案对不同三维原生生成器具有通用性，我们仍提出了专门的训练策略，以促进几何-纹理耦合与解耦两类三维原生生成范式的优化。实验表明，Photo3D 在不同三维原生生成范式之间具有良好的泛化能力，并取得了最优的照片级真实感三维生成性能。项目主页：https://liangsanzhu.github.io/photo3d-page/
英文原文摘要 1534 字符
Although recent 3D-native generators have made great progress in synthesizing reliable geometry, they still fall short in achieving realistic appearances. A key obstacle lies in the lack of diverse and high-quality real-world 3D assets with rich surface details, since capturing such data is intrinsically difficult due to the diverse scales of scenes, non-rigid motions of objects, and the limited precision of scanners.We introduce Photo3D, a framework for advancing photorealistic 3D generation, which is driven by the image data generated by the GPT-4o-Image model.Considering that the generated images can distort 3D structures due to their lack of multi-view consistency, we design a structure-aligned multi-view synthesis pipeline and construct a detail-enhanced multi-view dataset paired with 3D geometry. Building on it, we present a realistic detail enhancement scheme that leverages perceptual feature adaptation and semantic structure matching to enforce appearance consistency with the realistic detail priors while preserving the structural consistency with the 3D-native geometry. While our scheme is general to different 3D-native generators, we present dedicated training strategies to facilitate the optimization of geometry-texture coupled and decoupled 3D-native generation paradigms. Experiments demonstrate that Photo3D generalizes well across diverse 3D-native generation paradigms and achieves state-of-the-art photorealistic 3D generation performance. Project Page: https://liangsanzhu.github.io/photo3d-page/