统一球面前端:从任意相机学习球面图像的旋转等变表示
Unified Spherical Frontend: Learning Rotation-Equivariant Representations of Spherical Images from Any Camera
USF 提供了一种将任意标定相机图像映射到单位球面并进行球面卷积/池化的通用前端,其仅依赖距离的卷积核实现旋转等变,可替代传统球面 CNN 中的球谐变换,为处理鱼眼、全景等宽视场图像提供高效且镜头无关的表示。
原文依据 performs spherical resampling, convolution, and pooling canonically in the spatial domain
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论现代感知系统日益依赖鱼眼、全景及其他宽视场相机,然而大多数流程仍将针对针孔图像设计的平面 CNN 应用于二维网格上,其中像素空间邻域无法正确反映物理邻接关系,且模型对全局旋转敏感。传统球面 CNN 部分缓解了这种不匹配,但需要昂贵的球谐变换,限制了分辨率和效率。我们提出统一球面前端(USF),一个无畸变、镜头无关的框架,通过光线方向对应关系将任意标定相机的图像变换到单位球面上,并在空间域中规范地执行球面重采样、卷积和池化。USF 是模块化的:投影、位置采样、值插值及分辨率控制完全解耦。其可配置的仅依赖距离的卷积核提供旋转等变性,与平面 CNN 中的平移等变性相呼应,同时完全避免了谐波变换。我们在合成数据集(Spherical MNIST)和真实数据集(PANDORA、Stanford 2D-3D-S)上,针对分类、检测和分割任务,比较了多种标准平面骨干网络及其球面对应版本,并压力测试了模型对极端镜头畸变、变化视场及任意旋转的鲁棒性。USF 能够高效扩展到高分辨率球面图像,在测试时随机旋转下(无需训练时旋转增强)性能下降小于 1%,并实现对任何未见过的(宽视场)镜头的零样本泛化,性能退化极小。
英文原文摘要 1630 字符
Modern perception increasingly relies on fisheye, panoramic, and other wide field-of-view (FoV) cameras, yet most pipelines still apply planar CNNs designed for pinhole imagery on 2D grids, where pixel-space neighborhoods misrepresent physical adjacency and models are sensitive to global rotations. Traditional spherical CNNs partially address this mismatch but require costly spherical harmonic transform that constrains resolution and efficiency. We present Unified Spherical Frontend (USF), a distortion-free lens-agnostic framework that transforms images from any calibrated camera onto the unit sphere via ray-direction correspondences, and performs spherical resampling, convolution, and pooling canonically in the spatial domain. USF is modular: projection, location sampling, value interpolation, and resolution control are fully decoupled. Its configurable distance-only convolution kernels offer rotation-equivariance, mirroring translation-equivariance in planar CNNs while avoiding harmonic transforms entirely. We compare multiple standard planar backbones with their spherical counterparts across classification, detection, and segmentation tasks on synthetic (Spherical MNIST) and real-world (PANDORA, Stanford 2D-3D-S) datasets, and stress-test robustness to extreme lens distortions, varying FoV, and arbitrary rotations. USF scales efficiently to high-resolution spherical imagery and maintains less than 1% performance drop under random test-time rotations without training-time rotational augmentation, and enables zero-shot generalization to any unseen (wide-FoV) lenses with minimal performance degradation.