基于光照的视频生成：以渲染器为支撑的智能体推理
Lighting-grounded Video Generation with Renderer-based Agent Reasoning
把布局、光照、相机轨迹等显式3D场景属性通过渲染器转成控制信号，再用轻量条件模块注入视频扩散模型，为可控视频生成提供了可复用的解耦条件注入范式；其场景智能体把高层指令翻译成3D控制信号的思路也可迁移到其他条件生成任务。
原文依据 We disentangles these properties by rendering control signals from a unified 3D representation.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论扩散模型在视频生成方面已取得显著进展，但其可控性仍是一大局限。布局、光照、相机轨迹等关键场景因素往往相互纠缠，或仅被弱建模，这限制了它们在影视制作与虚拟制片等需要显式场景控制的领域中的应用。我们提出 LiVER，一个基于扩散模型、可实现场景可控视频生成的框架。为此，我们引入一种新框架，以显式的三维场景属性作为视频合成的条件，并配套一个带有物体布局、光照与相机参数稠密标注的新大规模数据集。我们的方法通过从统一的三维表示渲染出控制信号，将这些属性解耦。我们提出一个轻量条件模块与一种渐进式训练策略，把这些信号集成到基础视频扩散模型中，从而保证稳定的收敛与高保真度。该框架支持广泛的应用，包括底层三维场景完全可编辑的图像到视频与视频到视频合成。为进一步提升易用性，我们开发了一个场景智能体，可自动将高层用户指令转换为所需的三维控制信号。实验表明，LiVER 在实现最优的照片级真实感与时序一致性的同时，能够对场景因素进行精确且解耦的控制，为可控视频生成树立了新标准。
英文原文摘要 1497 字符
Diffusion models have achieved remarkable progress in video generation, but their controllability remains a major limitation. Key scene factors such as layout, lighting, and camera trajectory are often entangled or only weakly modeled, restricting their applicability in domains like filmmaking and virtual production where explicit scene control is essential. We present LiVER, a diffusion-based framework for scene-controllable video generation. To achieve this, we introduce a novel framework that conditions video synthesis on explicit 3D scene properties, supported by a new large-scale dataset with dense annotations of object layout, lighting, and camera parameters. Our method disentangles these properties by rendering control signals from a unified 3D representation. We propose a lightweight conditioning module and a progressive training strategy to integrate these signals into a foundational video diffusion model, ensuring stable convergence and high fidelity. Our framework enables a wide range of applications, including image-to-video and video-to-video synthesis where the underlying 3D scene is fully editable. To further enhance usability, we develop a scene agent that automatically translates high-level user instructions into the required 3D control signals. Experiments show that LiVER achieves state-of-the-art photorealism and temporal consistency while enabling precise, disentangled control over scene factors, setting a new standard for controllable video generation.