DiT360：通过混合训练实现高保真全景图像生成
DiT360: High-Fidelity Panoramic Image Generation via Hybrid Training
针对全景图像生成中边界不连续和畸变问题，提出在 VAE 前后分别进行域间转换与域内增强的混合训练策略，包括循环填充、偏航损失和立方体损失，可复用于其他需要保持几何一致性的生成任务。
原文依据 we propose DiT360, a DiT-based framework that performs hybrid training on perspective and panoramic data for panoramic image generation.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论本文提出 DiT360，一个基于 DiT 的框架，在透视图像和全景数据上进行混合训练以生成全景图像。与先前强调模型设计的方法不同，我们将保持几何保真度和照片真实感的主要挑战归因于大规模、高质量真实世界全景数据的稀缺。基本上，DiT360 具有若干关键模块，用于域间转换和域内增强，这些模块应用于 VAE 前的图像层级和 VAE 后的词元层级。在图像层级，我们通过透视图像引导和全景细化来融入跨域知识，这提升了感知质量，同时正则化多样性和照片真实感。在词元层级，混合监督被应用于多个模块，包括用于边界连续性的循环填充、用于旋转鲁棒性的偏航损失以及用于畸变感知的立方体损失。在文本到全景、修复和外推任务上的大量实验表明，我们的方法在十一项定量指标上实现了更好的边界一致性和图像保真度。我们的代码可在 https://github.com/Insta360-Research-Team/DiT360 获取。
英文原文摘要 1216 字符
In this work, we propose DiT360, a DiT-based framework that performs hybrid training on perspective and panoramic data for panoramic image generation. We attribute the main challenges in preserving geometric fidelity and photorealism to the scarcity of large-scale, high-quality real-world panoramic data, in contrast to prior methods that emphasize model design. Basically, DiT360 has several key modules for inter-domain transformation and intra-domain augmentation, applied at both the pre-VAE image level and the post-VAE token level. At the image level, we incorporate cross-domain knowledge through perspective image guidance and panoramic refinement, which enhance perceptual quality while regularizing diversity and photorealism. At the token level, hybrid supervision is applied across multiple modules, which include circular padding for boundary continuity, yaw loss for rotational robustness, and cube loss for distortion awareness. Extensive experiments on text-to-panorama, inpainting, and outpainting tasks demonstrate that our method achieves better boundary consistency and image fidelity across eleven quantitative metrics. Our code is available at https://github.com/Insta360-Research-Team/DiT360.