通过学习视角token实现文本到图像生成的相机控制
Camera Control for Text-to-Image Generation via Learning Viewpoint Tokens
把相机位姿编码为可学习的视角token注入扩散模型，使文本到图像生成获得精确的相机控制；其用3D渲染图做几何监督、用真实感增强补外观多样性的数据构造方式，以及验证视角token可跨物体类别迁移的解耦性，对做可控生成与3D感知生成的研究者可直接复用。
原文依据 we present a framework for precise camera control with global scene understanding in text-to-image generation by learning parametric camera tokens
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论当前文本到图像模型难以仅凭自然语言提供精确的相机控制。在本工作中，我们提出一个框架，通过学习参数化相机token，在文本到图像生成中实现具备全局场景理解的精确相机控制。我们在一个精心构建的数据集上微调图像生成模型以进行视角条件化的文本到图像生成，该数据集结合了用于几何监督的3D渲染图像与用于外观和背景多样性的真实感增强。定性与定量实验表明，我们的方法在保持图像质量与提示词保真度的同时达到了最优精度。与先前过拟合于物体特定外观相关性的方法不同，我们的视角token学习到解耦的几何表征，可迁移到未见过的物体类别。我们的工作表明，文本-视觉潜变量空间可以被赋予显式的3D相机结构，为文本到图像生成的几何感知提示词提供了一条路径。项目主页：https://randdl.github.io/viewtoken_control/
英文原文摘要 1084 字符
Current text-to-image models struggle to provide precise camera control using natural language alone. In this work, we present a framework for precise camera control with global scene understanding in text-to-image generation by learning parametric camera tokens. We fine-tune image generation models for viewpoint-conditioned text-to-image generation on a curated dataset that combines 3D-rendered images for geometric supervision and photorealistic augmentations for appearance and background diversity. Qualitative and quantitative experiments demonstrate that our method achieves state-of-the-art accuracy while preserving image quality and prompt fidelity. Unlike prior methods that overfit to object-specific appearance correlations, our viewpoint tokens learn factorized geometric representations that transfer to unseen object categories. Our work shows that text-vision latent spaces can be endowed with explicit 3D camera structure, offering a pathway toward geometrically-aware prompts for text-to-image generation. Project page: https://randdl.github.io/viewtoken_control/