自监督与表征学习 · CVPR2026 ·2026.06.03

GeoWorld:几何世界模型

GeoWorld: Geometric World Models

Zeyu Zhang、Danning Li、Ian Reid、Richard Hartley

与你有关的部分

把 JEPA 的潜变量从欧氏空间映射到双曲流形,用几何层级结构约束表征,再配合几何强化学习做多步规划;对做视频预测式世界模型或潜空间规划的人,这是一条可直接替换潜空间几何的通用思路。

原文依据 a Hyperbolic JEPA, which maps latent representations from Euclidean space onto hyperbolic manifolds
可迁移到 视觉世界模型多步视觉规划视频表征学习

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

基于能量的预测式世界模型通过在潜变量能量景观上推理而非生成像素,为多步视觉规划提供了强有力的途径。然而,现有方法面临两大挑战:(i) 其潜变量表征通常在欧氏空间中学习,忽略了状态之间潜在的几何与层级结构;(ii) 它们难以进行长时程预测,导致在扩展的推演过程中性能迅速退化。为应对这些挑战,我们提出 GeoWorld,一种几何世界模型,它通过双曲 JEPA 保持几何结构与层级关系,该 JEPA 将潜变量表征从欧氏空间映射到双曲流形上。我们进一步提出用于基于能量优化的几何强化学习,从而能够在双曲潜变量空间中实现稳定的多步规划。在 CrossTask 和 COIN 上的大量实验表明,与最优的 V-JEPA-2 相比,3 步规划的 SR 提升约 3%,4 步规划的 SR 提升约 2%。

英文原文摘要 1069 字符

Energy-based predictive world models provide a powerful approach for multi-step visual planning by reasoning over latent energy landscapes rather than generating pixels. However, existing approaches face two major challenges: (i) their latent representations are typically learned in Euclidean space, neglecting the underlying geometric and hierarchical structure among states, and (ii) they struggle with long-horizon prediction, which leads to rapid degradation across extended rollouts. To address these challenges, we introduce GeoWorld, a geometric world model that preserves geometric structure and hierarchical relations through a Hyperbolic JEPA, which maps latent representations from Euclidean space onto hyperbolic manifolds. We further introduce Geometric Reinforcement Learning for energy-based optimization, enabling stable multi-step planning in hyperbolic latent space. Extensive experiments on CrossTask and COIN demonstrate around 3% SR improvement in 3-step planning and 2% SR improvement in 4-step planning compared to the state-of-the-art V-JEPA-2.

标签

会议 CVPR2026
页码 30952-30963
Bibkey
来源 CVF Open Access