OpenDance：基于大规模互联网数据的多模态可控 3D 舞蹈生成
OpenDance: Multimodal Controllable 3D Dance Generation with Large-scale Internet Data
提出统一的掩码建模框架，通过解耦自编码器与多模态联合预测 Transformer，实现音乐、文本、关键点、轨迹任意组合条件下的 3D 舞蹈生成，为多条件可控人体运动生成提供了可复用的架构与大规模数据集。
原文依据 a unified masked modeling framework for controllable dance generation, including a disentangled auto-encoder and a multimodal joint-prediction Transformer
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论音乐驱动的 3D 舞蹈生成具有显著的创意潜力，但实际应用需要多样化的多模态控制。由于舞蹈动作高度动态且复杂，涵盖多种风格和流派，舞蹈生成需要满足音乐之外的多种条件（例如空间轨迹、关键帧手势或风格描述）。然而，缺乏大规模且标注丰富的数据集严重阻碍了进展。本文构建了 OpenDanceSet，这是一个广泛的人类舞蹈数据集，包含超过 100 小时的数据，涵盖 14 种流派和 147 名受试者。每个样本都有丰富的标注以促进稳健的跨模态学习：3D 运动、配对音乐、2D 关键点、轨迹以及专家标注的文本描述。此外，我们提出了 OpenDanceNet，一个用于可控舞蹈生成的统一掩码建模框架，包括一个解耦自编码器和一个多模态联合预测 Transformer。OpenDanceNet 支持以音乐与文本、关键点或轨迹的任意组合为条件进行生成。综合实验表明，我们的工作实现了高保真合成，具有强大的多样性和真实的物理接触，同时提供了对空间和风格条件的灵活控制。
英文原文摘要 1294 字符
Music-driven 3D dance generation offers significant creative potential, yet practical applications demand versatile and multimodal control. Given the highly dynamic and complex human motion covering various styles and genres, dance generation requires satisfying diverse conditions beyond just music (e.g., spatial trajectories, keyframe gestures, or style descriptions). However, the absence of a large-scale and richly annotated dataset severely hinders progress. In this paper, we build OpenDanceSet, an extensive human dance dataset comprising over 100 hours across 14 genres and 147 subjects. Each sample has rich annotations to facilitate robust cross-modal learning: 3D motion, paired music, 2D keypoints, trajectories, and expert-annotated text descriptions. Furthermore, we propose OpenDanceNet, a unified masked modeling framework for controllable dance generation, including a disentangled auto-encoder and a multimodal joint-prediction Transformer. OpenDanceNet supports generation conditioned on music and arbitrary combinations of text, keypoints, or trajectories. Comprehensive experiments demonstrate that our work achieves high-fidelity synthesis with strong diversity and realistic physical contacts, while also offering flexible control over spatial and stylistic conditions.