Low-light color image enhancement algorithm based on conditional generative adversarial network

Objective With the rapid development of photographic and photographic equipment, it has become easier for people to obtain full-color high-definition images and video data in different scenes. Low illumination image enhancement has gradually become one of the most frontier issues in the field of night vision imaging and detection. Exploring low illumination image enhancement methods with high fidelity and high operational efficiency is of great value in military reconnaissance, emergency search and rescue, public safety and other fields. However, achieving real-time full-color low-light images under low-light imaging conditions still poses significant challenges, typically including long exposure time, low image contrast, significant loss of detail, and severe noise contamination. To solve the problem of color image enhancement under low illumination condition, a low illumination image enhancement algorithm based on conditional generative adversarial network (CGAN) is proposed.

Methods CGAN utilizes an adversarial process to build a model, and takes the random noise and the pre-processed low illumination images as the input to the generator, and then generates the generated images which are as close as possible to the normal illumination through the generator network. Then the normal illumination images and the generated images are input into the discriminator at the same time, and the discriminator network is utilized to output the probability value between 0 and 1, and the parameters are updated by the computational error (Fig.1). Secondly, in order to avoid the problem of gradient vanishing due to too deep network structure, the generative network introduces the residual-in-residual dense block (RRDB) module (Fig.4). The RRDB module contains three residual dense block modules (Fig.5), each of which contains five layers of convolutional networks, and the low illuminance image features extracted by each layer of the convolutional network are supplied to the subsequent convolutional layers, allowing the feature signals to be arbitrarily propagated from the shallow to the deep layers. The generative network also introduces a convolutional block attention module (CBAM) (Fig.8), which consists of a cascade of a channel attention module (Fig.6) and a spatial attention module (Fig.7). The channel attention module compresses the spatial information of the low-illumination image by using global maximum pooling and global average pooling, respectively, and feeds the compressed results into the multi-layer perceptron to adaptively adjust the channel weights of the low-illumination image. The spatial attention module adjusts the weights of the spatial dimensions of the low-illumination image using the channel information of the global maximum pooling and global average pooling compressed images. Then, SK-Net, a discriminator network based on selective convolution kernel, is constructed (Fig.9). It enables the discriminator to adaptively adjust its receptive field size according to the input, which is closer to the human eye's judgment of the normal illumination image. SK-Net utilizes convolution kernels of different sizes to convolve the input images to obtain multiple branches. The multi-branch information is fused and compressed into a one-dimensional vector using global average pooling, and then Softmax is used to obtain multiple weight coefficient matrices to weight the convolved multi-branch images, and the output feature images are obtained after summing. Then, the network model is enhanced by designing Prewitt loss function and YUV loss function to enhance the ability of extracting image edge details and eliminating image color distortion, respectively.

Results and Discussions The algorithm is tested qualitatively and quantitatively on LOL public dataset. The experimental results show the advantages in low-light image enhancement compared with the current deep learning-based low-light color image enhancement algorithms JED, KinD_New, Retinex-Net, SNR and URetinex-Net (Fig.10). The algorithm proposed in this paper improves 32.7%, 57.5% and 48.45% in peak signal-to-noise ratio, structural similarity and color difference, respectively(Tab.1). The algorithm is able to better overcome the problem of image noise and color bias interference under the low illumination imaging condition.

Conclusions A conditional generation adversarial network is proposed to solve the problem of image enhancement under low illumination. Firstly, the RRDB module is introduced to optimize the network structure of the generator in order to solve the problem of gradient disappearance in deep layer networks. Secondly, the introduction of CBAM attention mechanism aims to alleviate strong noise interference in low illumination environments by enhancing the attention weight of important features in enhanced images. Additionally the SK-Net network structure is designed so that the receptive field of the discriminator network can be adjusted adaptively, so as to improve the discriminant ability of generating color images. Finally, the loss function including Prewitt edge term, YUV chromaticity term and Content term is designed to solve the problem of edge sharpness degradation and color deviation. Qualitative and quantitative tests show that, compared with the current method based on deep learning algorithm, this method achieves improvements of 2% in SSIM, 26.22% in PSNR and 41.22% in CD respectively. It has excellent performance in noise suppression, color difference elimination and effective information retention.