面向光流估计的高效全对相关体采样
Efficient All-Pairs Correlation Volume Sampling for Optical Flow Estimation
RAFT 式全对相关体采样可换成内存与计算双高效的实现，且数学上与原算子完全等价，可直接替换现有光流网络中的 cost volume 采样层，从而在高分辨率输入上以更低显存和延迟获得更精细的匹配细节。
原文依据 we propose an algorithm for both memory and compute-efficient implementation of the all-pairs correlation volume sampling
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近期的光流估计方法常从稠密的全对相关体中进行局部代价采样。这导致计算与内存复杂度随像素数呈二次增长。尽管存在一种按需计算代价的内存高效替代实现，但其实际运行速度明显更慢，因此许多先前方法在降采样分辨率下处理图像，从而丢失了细粒度细节。为解决这一问题，我们提出一种算法，使全对相关体采样在内存与计算两方面都高效，同时仍与 RAFT 所定义的数学算子完全一致。我们的方法比按需采样快最多 92%，并保持同样低的内存占用；与默认实现相比性能至少相当，而内存占用最多降低 99%。由于代价采样占整体运行时间的显著部分，这可使高分辨率输入上的端到端模型推理总耗时最多节省 63%。我们对现有方法的评测包含一个 8K 超高分辨率数据集，以及对 SEA-RAFT 方法的推理时扩展。由此，我们在高分辨率下于精度和运行时间两方面均取得最优结果。
英文原文摘要 1261 字符
Recent optical flow estimation methods often employ local cost sampling from a dense all-pairs correlation volume. This results in quadratic computational and memory complexity in the number of pixels. Although an alternative memory-efficient implementation with on-demand cost computation exists, this is significantly slower in practice and therefore many prior methods process images at downsampled resolutions, missing fine-grained details. To address this, we propose an algorithm for both memory and compute-efficient implementation of the all-pairs correlation volume sampling, still matching the exact mathematical operator as defined by RAFT. Our approach outperforms on-demand sampling by up to 92% while maintaining equally low memory usage, and performs at least on par with the default implementation with up to 99% lower memory usage. As cost sampling makes up a significant portion of the overall runtime, this can translate to up to 63% savings for the total end-to-end model inference on high-resolution inputs. Our evaluation of existing methods includes an 8K ultra-high-resolution dataset and an inference-time extension of the SEA-RAFT method. With this, we achieve state-of-the-art results at high resolutions both in accuracy and runtime.