论文标题
LW-ISP：具有ISP和深度学习的轻量级模型
LW-ISP: A Lightweight Model with ISP and Deep Learning
论文作者
论文摘要
在硬件前景，错误积累和成像效果方面，低级任务的基于深度学习（DL）的方法具有许多优势。最近，深度学习替换图像信号处理（ISP）管道的应用已接近出现。但是，对于真正的着陆，还有很长的路要走。在本文中，我们展示了基于学习的方法在ISP管道中实现实时高性能处理的可能性。我们提出了LW-ISP，这是一种新颖的体系结构，旨在隐式地学习从原始数据到RGB图像的图像映射。基于U-NET体系结构，我们提出了适用于低级任务的细粒度注意模块和插件播放的Upsmpling Block。特别是，我们设计了一种异质蒸馏算法来提炼清洁图像的隐式特征和重建信息，以指导学生模型的学习。我们的实验表明，与先前的最佳方法相比，LW-ISP的PSNR提高了0.38 dB，而模型参数和计算已减少了23次和81次。推断效率至少加速了15次。在没有铃铛和哨子的情况下，LW-ISP在ISP子任务中取得了非常具竞争力的结果，包括图像降解和增强。
The deep learning (DL)-based methods of low-level tasks have many advantages over the traditional camera in terms of hardware prospects, error accumulation and imaging effects. Recently, the application of deep learning to replace the image signal processing (ISP) pipeline has appeared one after another; however, there is still a long way to go towards real landing. In this paper, we show the possibility of learning-based method to achieve real-time high-performance processing in the ISP pipeline. We propose LW-ISP, a novel architecture designed to implicitly learn the image mapping from RAW data to RGB image. Based on U-Net architecture, we propose the fine-grained attention module and a plug-and-play upsampling block suitable for low-level tasks. In particular, we design a heterogeneous distillation algorithm to distill the implicit features and reconstruction information of the clean image, so as to guide the learning of the student model. Our experiments demonstrate that LW-ISP has achieved a 0.38 dB improvement in PSNR compared to the previous best method, while the model parameters and calculation have been reduced by 23 times and 81 times. The inference efficiency has been accelerated by at least 15 times. Without bells and whistles, LW-ISP has achieved quite competitive results in ISP subtasks including image denoising and enhancement.
