TRCoRSurg：面向手术视频三元组识别的时序-关系协同推理
TRCoRSurg: Temporal-Relational Co-Reasoning for Surgical Video Triplet Recognition
面向手术视频的三元组（器械-动作-目标）识别框架，其多尺度类别激活图引导的关系提取与双向时序-关系融合注意力可迁移到任何需要建模帧内标签共现与帧间语义一致性的视频结构化识别任务；同时提出的 TCER 指标为衡量跨实体因果/语义一致性提供了可复用的评测思路。
原文依据 a Bidirectional Temporal-Relational Fusion Attention (BTRFA) module harmonizes temporal and relational representations to achieve coherent temporal reasoning
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论理解复杂的手术场景需要识别多个相互依存的实体——例如器械、动作和目标——并在时间上保持它们之间的关系一致性。现有的手术三元组识别方法难以以统一的方式联合建模帧内标签依赖与帧间时序语义。为解决这些局限，我们提出了一个统一框架，融合空间、关系与时序线索以实现鲁棒的手术三元组识别。具体而言，首先通过多尺度编码器提取类别特定的空间先验。随后，这些先验由标签相关性建模模块结合多尺度类别激活图引导的关系提取（MS-CAMRE）进行细化，使模型能够同时捕获三元组各组件之间的静态共现与动态上下文依赖。此外，双向时序-关系融合注意力（BTRFA）模块协调时序表示与关系表示，以实现连贯的时序推理。我们还引入了一个新的评测指标——三元组一致性错误率（TCER），用于定量衡量模型在多个三元组之间保持因果与语义一致性的能力。在 CholecT45 和 ProStaTD 数据集上的大量实验表明，我们的方法取得了最优性能，AP_IVT 分别提升 5.1% 和 7.8%。此外，在 TCER 指标上，我们的方法在两个数据集上分别实现了超过 36% 和 25% 的相对降低，凸显了该框架在时序-关系协同推理方面的有效性。
英文原文摘要 1628 字符
Understanding complex surgical scenes requires recognizing multiple interdependent entities--such as instruments, actions, and targets--and maintaining their relational consistency across time. Existing surgical triplet recognition methods struggle to jointly model intra-frame label dependencies and inter-frame temporal semantics in a unified manner. To address these limitations, we propose a unified framework that integrates spatial, relational, and temporal cues for robust surgical triplet recognition. Specifically, class-specific spatial priors are first extracted through a multi-scale encoder. Then, these priors are refined by a Label Correlation Modeling module with multi-scale class activation map-guided relational extraction (MS-CAMRE), enabling the model to capture both static co-occurrence and dynamic contextual dependencies among triplet components. Furthermore, a Bidirectional Temporal-Relational Fusion Attention (BTRFA) module harmonizes temporal and relational representations to achieve coherent temporal reasoning. We also introduce a new evaluation metric, the Triplet Consistency Error Rate (TCER), which quantitatively measures the model's capability to preserve causal and semantic consistency across triplets. Extensive experiments on the CholecT45 and ProStaTD datasets show that our method achieves state-of-the-art (SOTA) performance, improving AP_ IVT by 5.1% and 7.8%, respectively. Moreover, on the TCER metric, our approach yields over 36% and 25% relative reductions on the two datasets, respectively, underscoring the effectiveness of our framework in temporal-relational co-reasoning.