JUMP-Hand：学习逐关节不确定性以门控多视角专家混合模型进行多视角3D手部重建
JUMP-Hand: Learning Joint-wise Uncertainty to Gate Mixture of View Experts for Multi-View 3D Hand Reconstruction
将逐关节不确定性作为显式门控信号，用于多视角特征融合与三角化初始化，可迁移到其他多视角三维重建任务（如人体姿态、物体位姿）中处理遮挡与噪声。
原文依据 explicitly models probabilistic joint-wise uncertainty as a gating mechanism for multi-view fusion
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论我们提出JUMP-Hand，一种新颖的多视角3D手部重建方法，它显式地将逐关节概率不确定性建模为多视角融合的门控机制。现有方法通常依赖简单的池化或隐式注意力，忽略了每个手部关节在不同视角下表现出不同的可见性和可靠性。这种对噪声或不可靠信息的不加区分聚合不可避免地会降低整体性能。例如，一个视角中被严重遮挡的关节在另一个视角中可能清晰可见。为了解决这个问题，JUMP-Hand从混合专家（MoE）范式中获得灵感，将每个2D视角视为一个专门的专家。其核心思想是，通过逐关节不确定性建模来量化每个视角专家的可靠性，并将其作为显式门控信号，在由粗到细的重建范式中为每个关节路由专家们部分但互补的线索。在该设计中，不确定性不仅在粗阶段引导不确定性感知的三角化以获得可靠的3D手部初始化，而且在细化阶段作为门控信号，以逐关节的方式自适应地聚合来自不同视角专家的多尺度特征，从而实现鲁棒的3D手部重建。在DexYCB-MV、HO3D-MV和OakInk-MV上的大量实验表明，我们的方法始终达到最优结果，验证了所提出的逐关节不确定性门控方法对于可靠3D手部重建的有效性。代码可在https://github.com/HaohongKuang/JUMP-Hand获取。
英文原文摘要 1592 字符
We propose JUMP-Hand, a novel multi-view 3D hand reconstruction method that explicitly models probabilistic joint-wise uncertainty as a gating mechanism for multi-view fusion. Existing approaches usually rely on naive pooling or implicit attention, overlooking that each hand joint exhibits varying visibility and reliability across views. Such indiscriminate aggregation of noisy or unreliable information inevitably degrades overall performance. For instance, a joint severely occluded in one view might be clearly visible in another. To address this, JUMP-Hand draws inspiration from the Mixture of Experts (MoE) paradigm, treating each 2D view as a specialized expert. The key idea is that the reliability of each view expert is quantified through joint-wise uncertainty modeling, serving as an explicit gating signal to route experts' partial yet complementary clues for each joint in a coarse-to-fine reconstruction paradigm. In this design, uncertainty not only guides the uncertainty-aware triangulation for reliable 3D hand initialization during the coarse stage, but also acts as a gating signal during the refinement stage to adaptively aggregate multi-scale features from different view experts on a joint-wise basis, enabling robust 3D hand reconstruction. Extensive experiments on DexYCB-MV, HO3D-MV, and OakInk-MV demonstrate that our method consistently achieves state-of-the-art results, validating the effectiveness of the proposed method with joint-wise uncertainty gating for reliable 3D hand reconstruction. Code is available at https://github.com/HaohongKuang/JUMP-Hand.