Fast-FoundationStereo:实时零样本立体匹配
Fast-FoundationStereo: Real-Time Zero-Shot Stereo Matching
把基础模型(FoundationStereo)的零样本能力通过知识蒸馏+分块 NAS+结构化剪枝压到实时帧率,这套「大模型→单学生骨干+延迟预算下搜索代价滤波+剪枝迭代精化」的加速流水线可直接复用到其他稠密预测基础模型(深度、光流、分割)的实时化。
原文依据 we present Fast-FoundationStereo, a family of architectures that achieve, for the first time, strong zero-shot generalization at real-time frame rate
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论立体基础模型具有强大的零样本泛化能力,但其计算开销使其难以用于实时应用。另一方面,高效立体架构以牺牲鲁棒性换取速度,并且需要昂贵的逐域微调。为弥合这一差距,我们提出 Fast-FoundationStereo,这是一族首次在实时帧率下实现强零样本泛化的架构。我们采用分而治之的加速策略,包含三个部分:(1) 知识蒸馏,将混合骨干网络压缩为单个高效学生模型;(2) 分块神经架构搜索,在延迟预算下自动发现最优的代价滤波设计,将搜索复杂度指数级降低;(3) 结构化剪枝,消除迭代精化模块中的冗余。此外,我们引入自动伪标签流水线,用于整理 140 万对野外立体图像对,以补充合成训练数据并促进知识蒸馏。所得模型运行速度比 FoundationStereo 快 10 倍以上,同时其零样本精度与之接近,从而在实时方法中确立了新的最优水平。项目页面:https://nvlabs.github.io/Fast-FoundationStereo
英文原文摘要 1274 字符
Stereo foundation models achieve strong zero-shotgeneralization but remain computationally prohibitive forreal-time applications. Efficient stereo architectures, on the other hand, sacrificerobustness for speed and require costly per-domain fine-tuning.To bridge this gap, we present Fast-FoundationStereo, a family of architectures that achieve, for the first time, strong zero-shot generalization at real-time frame rate. We employ a divide-and-conquer acceleration strategy with three components: (1) knowledge distillation to compress the hybrid backbone into a single efficient student; (2) blockwise neural architecture search for automatically discovering optimal cost filtering designs under latency budgets, reducing search complexity exponentially; and (3) structured pruning for eliminating redundancy in the iterative refinement module. Furthermore, we introduce an automatic pseudo-labeling pipeline used to curate 1.4M in-the-wild stereo pairs to supplement synthetic training data and facilitate knowledge distillation. The resulting model can run over 10x faster than FoundationStereo while closely matching its zero-shot accuracy, thus establishing a new state-of-the-art among real-time methods. Project page: https://nvlabs.github.io/Fast-FoundationStereo