SAGE:面向具身智能的可扩展智能体式 3D 场景生成
SAGE: Scalable Agentic 3D Scene Generation for Embodied AI
把布局生成器与物体组合生成器同语义合理性、视觉真实感、物理稳定性三类 critic 组成闭环,通过迭代推理与自适应工具选择自精炼场景,可直接产出仿真器可用的 3D 环境用于策略训练;对做 3D 场景合成与具身数据扩增的视觉研究者,这是一套可复用的「生成器+评审器」自精炼管线。
原文依据 The agent couples multiple generators for layout and object composition with critics that evaluate semantic plausibility, visual realism, and physical stability.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论为具身智能体采集真实世界数据仍然成本高昂且不安全,因此需要可扩展、逼真且可直接用于仿真器的 3D 环境。然而,现有场景生成系统往往依赖基于规则或面向特定任务的流水线,会产生伪影和物理上无效的场景。我们提出 SAGE,一个智能体式框架:给定用户指定的具身任务(例如「拿起碗并放到桌上」),它能理解意图并大规模自动生成可直接用于仿真的环境。该智能体将用于布局与物体组合的多个生成器,与评估语义合理性、视觉真实感和物理稳定性的评审器相结合。通过迭代推理与自适应工具选择,它不断自精炼场景,直至满足用户意图与物理有效性。所生成的环境逼真、多样,且可直接部署到现代仿真器中用于策略训练。仅在该数据上训练的策略呈现出明显的扩展趋势,并能泛化到未见过的物体与布局,展示了仿真驱动的扩展对具身智能的前景。代码、演示和 SAGE-10k 数据集见项目主页:https://research.nvidia.com/labs/dir/sage/。
英文原文摘要 1225 字符
Real-world data collection for embodied agents remains costly and unsafe, calling for scalable, realistic, and simulator-ready 3D environments. However, existing scene-generation systems often rely on rule-based or task-specific pipelines, yielding artifacts and physically invalid scenes. We present SAGE, an agentic framework that, given a user-specified embodied task (e.g., "pick up a bowl and place it on the table"), understands the intent and automatically generates simulation-ready environments at scale. The agent couples multiple generators for layout and object composition with critics that evaluate semantic plausibility, visual realism, and physical stability. Through iterative reasoning and adaptive tool selection, it self-refines the scenes until meeting user intent and physical validity. The resulting environments are realistic, diverse, and directly deployable in modern simulators for policy training. Policies trained purely on this data exhibit clear scaling trends and generalize to unseen objects and layouts, demonstrating the promise of simulation-driven scaling for embodied AI. Code, demos, and the SAGE-10k dataset can be found on the project page: https://research.nvidia.com/labs/dir/sage/.