频谱手术刀:通过频率选择性滤波放大相邻动作差异的骨骼动作分割方法
Spectral Scalpel: Amplifying Adjacent Action Discrepancy via Frequency-Selective Filtering for Skeleton-Based Action Segmentation
把频域选择性滤波当作可插拔模块,用于放大相邻动作在频谱上的差异,可直接迁移到其他需要锐化时序边界的密集预测任务(如视频动作分割、在线时序定位)。
原文依据 adaptive multi-scale spectral filters as scalpels to edit frequency spectra
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论基于骨骼的时序动作分割(STAS)旨在对长且未裁剪的骨骼运动序列中的多种动作进行密集分割与分类。然而,现有 STAS 方法面临类间可区分性有限和分割边界模糊的挑战,其主要原因在于相邻动作之间的时空模式区分不足。为解决这些局限,我们提出频谱手术刀(Spectral Scalpel),一个频率选择性滤波框架,旨在抑制相邻不同动作之间共享的频率成分,同时放大各自动作特有的频率,从而增强动作间差异并锐化过渡边界。具体而言,Spectral Scalpel 采用自适应多尺度频谱滤波器作为手术刀来编辑频谱,并配合相邻动作之间的差异损失作为手术目标。该设计放大了相邻动作之间的表征差异,有效缓解了边界定位模糊和类间混淆。此外,作为对长时序建模的补充,我们引入频率感知通道混合器,通过聚合跨通道的频谱来增强通道演化。本工作为 STAS 提出了一种新范式,通过引入频域分析扩展了传统的时空建模。在五个公开数据集上的大量实验表明,Spectral Scalpel 达到了最优性能。代码见 https://github.com/HaoyuJi/SpecScalpel。
英文原文摘要 1564 字符
Skeleton-based Temporal Action Segmentation (STAS) seeks to densely segment and classify diverse actions within long, untrimmed skeletal motion sequences. However, existing STAS methodologies face challenges of limited inter-class discriminability and blurred segmentation boundaries, primarily due to insufficient distinction of spatio-temporal patterns between adjacent actions. To address these limitations, we propose Spectral Scalpel, a frequency-selective filtering framework aimed at suppressing shared frequency components between adjacent distinct actions while amplifying their action-specific frequencies, thereby enhancing inter-action discrepancies and sharpening transition boundaries. Specifically, Spectral Scalpel employs adaptive multi-scale spectral filters as scalpels to edit frequency spectra, coupled with a discrepancy loss between adjacent actions serving as the surgical objective. This design amplifies representational disparities between neighboring actions, effectively mitigating boundary localization ambiguities and inter-class confusion. Furthermore, complementing long-term temporal modeling, we introduce a frequency-aware channel mixer to strengthen channel evolution by aggregating spectra across channels. This work presents a novel paradigm for STAS that extends conventional spatio-temporal modeling by incorporating frequency-domain analysis. Extensive experiments on five public datasets demonstrate that Spectral Scalpel achieves state-of-the-art performance. Code is available at https://github.com/HaoyuJi/SpecScalpel.