EMMA：从多模态数据中提取多个物理参数
EMMA: Extracting Multiple physical parameters from Multimodal Data
把视频轨迹当作动力学观测，用 Liquid Time-Constant 网络在连续时间模型里联合估计显式物理参数与隐变量，为视觉研究者提供了一条不依赖分割掩码或可微渲染、直接从原始视频反演系统物理量的可复用路径。
原文依据 recovers all identifiable dynamical parameters of a system directly from raw video, audio, and image-based time-series observations
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论本文提出 EMMA，一个物理信息驱动的多模态框架，可直接从原始视频、音频以及基于图像的时间序列观测中恢复系统的全部可辨识动力学参数。以往仅依赖视频的方法在状态被遮挡、驱动输入未知，或假设初始条件与坐标系已知时往往难以奏效；与之不同，EMMA 在统一的连续时间模型内对显式参数、隐式动力学分量与标定不变量进行联合推理。EMMA 利用液体时间常数（LTC）网络从异构模态中学习潜变量动力学，同时由物理约束损失强制其与主导微分方程保持一致。统一的特征流水线使视频轨迹、声学特征与图表导出的测量之间能够实现一致对齐，从而使 EMMA 无需分割掩码、可微渲染或专用传感器，即可在受迫、隐式与多变量动力学下估计参数。在 100 多个场景中——包括五个标准动力学基准（75 段 Delfys 视频）、具有隐藏输入的真实漫游车与四旋翼系统，以及涵盖生物系统与混沌系统的仿真—图表案例研究——EMMA 均实现了稳健的多参数恢复，并显著优于现有的单模态与方程发现基线。上述结果确立了 EMMA 作为一种通用、可扩展的方案，可从机会式多模态数据中提取物理一致的模型。
英文原文摘要 1519 字符
We introduce EMMA, a physics-informed multimodal framework that recovers all identifiable dynamical parameters of a system directly from raw video, audio, and image-based time-series observations. Unlike prior video-only approaches that struggle with occluded states, hidden actuation inputs, or assumptions about known initial conditions and coordinate frames, EMMA performs joint inference of explicit parameters, implicit dynamical components, and calibration invariants within a unified continuous-time model. EMMA leverages a Liquid Time-Constant (LTC) network to learn latent dynamics from heterogeneous modalities while a physics-constrained loss enforces consistency with the governing differential equations. A unified feature pipeline enables consistent alignment across video trajectories, acoustic signatures, and chart-derived measurements, allowing EMMA to estimate parameters under forced, implicit, and multivariate dynamics without requiring segmentation masks, differentiable rendering, or specialized sensors. Across 100+ scenarios including five standard dynamical benchmarks (75 Delfys videos), real-world rover and quadrotor systems with hidden inputs, and simulation-chart case studies spanning biological and chaotic systems, EMMA delivers robust multi-parameter recovery and significantly outperforms existing single-modality and equation-discovery baselines. Our results establish EMMA as a general, scalable solution for physics-consistent model extraction from opportunistic multimodal data.