SceMoS：以几何接地词表规划的场景感知三维人体运动合成
SceMoS: Scene-Aware 3D Human Motion Synthesis by Planning with Geometry-Grounded Tokens
用 BEV 渲染图 + DINOv2 特征替代点云/体素做场景编码，并用局部高度图训练条件 VQ-VAE 分词器把表面物理嵌入离散词表——这套「2D 场景线索接地 3D 交互」的因子化思路可直接迁移到人体-场景交互、导航与具身动作生成中，显著降低场景编码参数量。
原文依据 a geometry-grounded motion tokenizer trained via a conditional VQ-VAE, that uses 2D local scene heightmap
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论在真实场景中合成文本驱动的三维人体运动，需要同时学习语义意图（如「走向沙发」）与物理可行性（如避免碰撞）。现有方法采用同时学习高层规划与低层接触推理的生成框架，并依赖点云或体素占用栅格等计算代价高昂的三维场景数据。我们提出 SceMoS，一个场景感知的运动合成框架，表明结构化二维场景表示可以成为物理接地运动合成中完整三维监督的有力替代。SceMoS 利用轻量二维线索将全局规划与局部执行解耦，并依赖：(1) 一个文本条件的自回归全局运动规划器，其运行在从场景高处角落渲染的鸟瞰图（BEV）图像上，并以 DINOv2 特征编码作为场景表示；(2) 一个通过条件 VQ-VAE 训练的几何接地运动分词器，它使用二维局部场景高度图，从而将表面物理直接嵌入离散词表。这种二维因子化达成了效率与保真度的权衡：BEV 语义捕获空间布局与可供性以支持全局推理，而局部高度图在无需完整三维体素推理的情况下保证细粒度物理贴合。SceMoS 在 TRUMANS 基准上取得最优的运动真实感与接触准确率，并将场景编码的可训练参数量减少 50% 以上，表明二维场景线索能够有效接地三维人体-场景交互。项目网址：https://anindita127.github.io/SceMoS。
英文原文摘要 1644 字符
Synthesizing text-driven 3D human motion within realistic scenes requires learning both semantic intent ("walk to the couch") and physical feasibility (e.g., avoiding collisions). Current methods use generative frameworks that simultaneously learn high-level planning and low-level contact reasoning, and rely on computationally expensive 3D scene data such as point clouds or voxel occupancy grids. We propose SceMoS, a scene-aware motion synthesis framework that shows that structured 2D scene representations can serve as a powerful alternative to full 3D supervision in physically grounded motion synthesis. SceMoS disentangles global planning from local execution using lightweight 2D cues and relying on (1) a text-conditioned autoregressive global motion planner that operates on a bird's-eye-view (BEV) image rendered from an elevated corner of the scene, encoded with DINOv2 features, as the scene representation, and (2) a geometry-grounded motion tokenizer trained via a conditional VQ-VAE, that uses 2D local scene heightmap, thus embedding surface physics directly into a discrete vocabulary. This 2D factorization reaches an efficiency-fidelity trade-off: BEV semantics capture spatial layout and affordance for global reasoning, while local heightmaps enforce fine-grained physical adherence without full 3D volumetric reasoning. SceMoS achieves state-of-the-art motion realism and contact accuracy on the TRUMANS benchmark, reducing the number of trainable parameters for scene encoding by over 50%, showing that 2D scene cues can effectively ground 3D human-scene interaction. Project URL: https://anindita127.github.io/SceMoS.