SHands:面向医学训练的手术手势与错误识别的多视角数据集与基准
SHands: A Multi-View Dataset and Benchmark for Surgical Hand-Gesture and Error Recognition Toward Medical Training
提供带帧级手势基元标注与错误类型分类体系的多视角手术视频,并给出单视角/多视角/跨视角泛化三种评测协议,可直接用于检验多视角融合与跨视角域泛化方法在细粒度手部动作识别上的表现。
原文依据 we present Surgical-Hands (SHANDS), a large-scale multi-view video dataset for surgical hand-gesture and error recognition
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论在医学生的手术训练中,熟练度的提升依赖专家主导的技能评估,这种方式成本高、时间受限、难以规模化,且其专业知识仅限于拥有相关专家的机构。基于人工智能的自动化评估提供了一种可行的替代方案,但其进展受限于缺乏包含真实受训者错误以及训练鲁棒计算机视觉方法所需的多视角变异性的数据集。为填补这一空白,我们提出 Surgical-Hands(SHANDS),一个面向医学训练的手术手势与错误识别的大规模多视角视频数据集。SHANDS 使用五台互补视角的 RGB 相机记录直线切口与缝合操作,由 52 名参与者(20 名专家与 32 名受训者)完成,每人在每项操作中完成三次标准化试验。视频在帧级别标注了 15 种手势基元,并包含一套经过验证的 8 类受训者错误类型分类体系,从而同时支持手势识别与错误检测。我们进一步定义了单视角、多视角与跨视角泛化的标准化评测协议,并在该数据集上对最优深度学习模型进行了基准测试。SHANDS 已公开发布,以支持开发稳健且可扩展的、植根于临床整理领域知识的手术训练人工智能系统。
英文原文摘要 1370 字符
In surgical training for medical students, proficiency development relies on expert-led skill assessment, which is costly, time-limited, difficult to scale, and its expertise remains confined to institutions with available specialists. Automated AI-based assessment offers a viable alternative, but progress is constrained by the lack of datasets containing realistic trainee errors and the multi-view variability needed to train robust computer vision approaches. To address this gap, we present Surgical-Hands (SHANDS), a large-scale multi-view video dataset for surgical hand-gesture and error recognition for medical training. SHANDS captures linear incision and suturing using five RGB cameras from complementary viewpoints, performed by 52 participants (20 experts and 32 trainees), each completing three standardized trials per procedure. The videos are annotated at the frame level with 15 gesture primitives and include a validated taxonomy of 8 trainee error types, enabling both gesture recognition and error detection. We further define standardized evaluation protocols for single-view, multi-view, and cross-view generalization, and benchmark state-of-the-art deep learning models on the dataset. SHANDS is publicly released to support the development of robust and scalable AI systems for surgical training grounded in clinically curated domain knowledge.