面向高级视觉感知的任务感知图像信号处理器
Task-Aware Image Signal Processor for Advanced Visual Perception
把 ISP 从「还原好看图像」改成「为预训练视觉模型生成任务友好表征」，用少量多尺度调制算子替代稠密卷积，可直接插在 RAW 输入的下游检测/分割模型前，兼顾精度与延迟。
原文依据 TA-ISP predicts a small set of lightweight, multi-scale modulation operators that act at global, regional, and pixel scales
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近年来，计算机视觉领域出现了一种日益增长的趋势，即利用 RAW 传感器数据，因为与传统的低位深 RGB 图像相比，RAW 数据保留了更丰富的信息。早期研究主要关注提升视觉质量，而近期的努力则旨在利用 RAW 数据中的丰富信息来提升目标检测与分割等视觉感知任务的性能。然而，现有方法仍面临两个关键局限：大规模 ISP 网络带来沉重的计算开销，而基于调优传统 ISP 流水线的方法则受限于有限表征能力。为解决这些问题，我们提出任务感知图像信号处理器（TA-ISP），这是一个紧凑的 RAW 到 RGB 框架，可为预训练视觉模型生成面向任务的表征。TA-ISP 不采用沉重的稠密卷积流水线，而是预测一小组轻量级的多尺度调制算子，这些算子作用于全局、区域和像素尺度，以在不同空间范围上重塑图像统计特性。这种因子化控制显著扩展了可表征的空间变化变换的范围，同时将内存占用、计算量和延迟严格约束在较低水平。在白天与夜间条件下的多个 RAW 域检测与分割基准上评估，TA-ISP 持续提升下游精度，同时显著减少参数量和推理时间，使其非常适合部署在资源受限的设备上。
英文原文摘要 1546 字符
In recent years, there has been a growing trend in computer vision towards exploiting RAW sensor data, which preserves richer information compared to conventional low-bit RGB images. Early studies mainly focused on enhancing visual quality, while more recent efforts aim to leverage the abundant information in RAW data to improve the performance of visual perception tasks such as object detection and segmentation. However, existing approaches still face two key limitations: large-scale ISP networks impose heavy computational overhead, while methods based on tuning traditional ISP pipelines are restricted by limited representational capacity.To address these issues, we propose Task-Aware Image Signal Processor (TA-ISP), a compact RAW-to-RGB framework that produces task-oriented representations for pretrained vision models. Instead of heavy dense convolutional pipelines, TA-ISP predicts a small set of lightweight, multi-scale modulation operators that act at global, regional, and pixel scales to reshape image statistics across different spatial extents. This factorized control significantly expands the range of spatially varying transforms that can be represented while keeping memory usage, computation, and latency tightly constrained. Evaluated on several RAW-domain detection and segmentation benchmarks under both daytime and nighttime conditions, TA-ISP consistently improves downstream accuracy while markedly reducing parameter count and inference time, making it well suited for deployment on resource-constrained devices.