Unposed-to-3D：从真实世界图像学习可直接用于仿真的车辆模型
Unposed-to-3D: Learning Simulation-Ready Vehicles from Real-World Images
用相机预测检测头替代已知位姿监督，把可微渲染的光度误差当作自监督信号，使三维重建网络能从无位姿的真实驾驶图像中学习；这套「位姿预测+可微渲染」闭环可直接迁移到其他缺少相机标注的真实数据三维重建任务。
原文依据 we remove camera supervision and use a camera prediction head that directly estimates the camera parameters from unposed images
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论创建逼真且可直接用于仿真的三维资产对自动驾驶研究与虚拟环境构建至关重要。然而，现有的三维车辆生成方法通常在合成数据上训练，与真实世界分布存在显著的域差距。生成的模型往往呈现任意位姿且尺度未定义，导致其集成到驾驶场景时视觉一致性较差。本文提出 Unposed-to-3D，一个新颖的框架，仅依靠图像监督即可从真实驾驶图像中学习重建三维车辆。该方法包含两个阶段。第一阶段，使用已知相机参数的带位姿图像训练图像到三维的重建网络。第二阶段，移除相机监督，改用相机预测检测头直接从无位姿图像中估计相机参数。随后将预测的位姿用于可微渲染，以提供自监督的光度反馈，使模型能够纯粹从无位姿图像中学习三维几何。为确保仿真可用性，我们进一步引入尺度感知模块以预测真实世界的尺寸信息，以及一个协调模块，使生成的车辆适配目标驾驶场景，具有一致的光照与外观。大量实验表明，Unposed-to-3D 能够从真实世界图像中有效重建逼真、位姿一致且协调的三维车辆模型，为驾驶场景仿真与数字孪生环境创建高质量资产提供了一条可扩展的路径。
英文原文摘要 1591 字符
Creating realistic and simulation-ready 3D assets is crucial for autonomous driving research and virtual environment construction. However, existing 3D vehicle generation methods are often trained on synthetic data with significant domain gaps from real-world distributions. The generated models often exhibit arbitrary poses and undefined scales, resulting in poor visual consistency when integrated into driving scenes. In this paper, we present Unposed-to-3D, a novel framework that learns to reconstruct 3D vehicles from real-world driving images using image-only supervision. Our approach consists of two stages. In the first stage, we train an image-to-3D reconstruction network using posed images with known camera parameters. In the second stage, we remove camera supervision and use a camera prediction head that directly estimates the camera parameters from unposed images. The predicted pose is then used for differentiable rendering to provide self-supervised photometric feedback, enabling the model to learn 3D geometry purely from unposed images. To ensure simulation readiness, we further introduce a scale-aware module to predict real-world size information, and a harmonization module that adapts the generated vehicles to the target driving scene with consistent lighting and appearance. Extensive experiments demonstrate that Unposed-to-3D effectively reconstructs realistic, pose-consistent, and harmonized 3D vehicle models from real-world images, providing a scalable path toward creating high-quality assets for driving scene simulation and digital twin environments.