Abstract

In order to improve the clarity and color fidelity of traffic images under the complex environment of haze and uneven illumination and promote road traffic safety monitoring, a traffic image enhancement model based on illumination adjustment and depth of field difference is proposed. The algorithm is based on Retinex theory, uses dark channel principle to obtain image depth of the field, and uses spectral clustering algorithm to cluster image depth. After the subimages are divided, the local haze concentration is estimated according to the depth of field and the subimages are adaptively enhanced and fused. In addition, the illumination component is obtained by multiscale guided filtering to maintain the edge characteristics of the image, and the uneven illumination problem is solved by adjusting the curve function. The experimental results show that the proposed model can effectively enhance the uneven illumination and haze weather image in the traffic scene and the visual effect of the images is good. The generated image has rich details, improves the quality of traffic images, and can meet the needs of traffic practical application.

1. Introduction

Transportation is an important part of the national economy. With the changes of weather and environment, the images collected by traffic monitoring system and electronic driving instrument have different degrees of blur, low resolution, and uneven illumination [13]. These phenomena will make the electronic equipment unable to recognize and judge the useful information, bring inconvenience to traffic management and life, and bring severe challenges to the following image processing works such as feature extraction, target tracking, and recognition [4, 5]. Therefore, relying on artificial intelligence and machine vision technology to enhance the traffic images obtained under haze weather and uneven illumination conditions so as to improve the clarity, contrast and visual effect is a hot topic in recent years and has important theoretical research significance and application value.

The traditional histogram equalization (HE) method [68] is simple and efficient. The haze image has a narrow centralized histogram and low contrast. This method makes the histogram evenly distributed, expands the dynamic range of the image, and enhances the contrast, but it will lose some details. Since then, many scholars have improved the HE algorithm and proposed the bilateral histogram equalization algorithm BBHE, the adaptive histogram equalization algorithm CLAHE with limited contrast, and the algorithm ESIHE based on exposure sub-histogram equalization [913]. Homomorphic filtering algorithm enhances the high-frequency signal region of the original images and achieves clearer images, but the images do not conform to the human visual system, and the processed images show color distortion [14]. The recognition algorithm based on neural network [1517] can recognize images under different illumination conditions with high accuracy, but it needs a large number of training samples, setting the number of network layers and iterations, so the training time of network model is long and the real-time performance is poor.

Retinex algorithm [1820] is a common image enhancement method. Based on the theory of color constancy, the algorithm considers that the surface information of the object in the image is determined by its own characteristics and has nothing to do with the surrounding environment. The principle of Retinex algorithm is shown in Figure 1. The image can be regarded as composed of incident image and reflected image. The incident light irradiates on the reflected object, and the reflected light enters the human eye through the reflection of the reflected object, which is the image seen by human beings. The method can be expressed as follows: I (x, y) = L (x, y) R (x, y). I (x, y) is the image to be enhanced, which is composed of reflection component R (x, y) and irradiation component L (x, y). In the traditional Retinex algorithm [21, 22], Gaussian function is used to estimate the irradiance component in the image, and then the irradiance component is subtracted from the original image in the logarithmic domain to obtain the reflection component, that is, the enhanced image. This method is feasible in the uniform illumination image, but because the global illumination has a great change, for the nonuniform illumination image, removing the illumination component will lead to the global pixel value imbalance of the reflection component, which will affect the enhancement effect. So how to calculate the radiation component correctly greatly affects the image enhancement effect. Many improved Retinex algorithms focus on how to calculate the radiation component correctly [2326].

Retinex algorithm based on bilateral filtering improves the SSR and MSR algorithms with Gaussian function as the center-surround function, which can keep the image edge and overcome the halo, but it has high computational complexity and cannot deal with the blur effect of the image. The multiscale Retinex algorithm of double-guided filtering proposed by Fang et al. [27]. It assumes that the illumination component is piecewise smooth. The wavelet transform domain image fusion is used to enhance the image details and color fidelity, but the information fusion calculation cost is too high. Hong et al. [28] proposed Retinex image enhancement method based on feature extraction. The main feature images of three different scales were weighted and fused to estimate the irradiation component and then multiplied with the reflection component to construct the enhanced image, but the scale needed to be manually set. Zhuang and Ding [20] proposed a filtering Retinex algorithm based on edge preserving, which embeds the gradient domain filtering (GGF) prior to reflection and illumination into the Retinex-based variational framework to improve image structure and reduce artifacts and noise. Yoon and Choe [29] aimed at the problem that illumination and reflectivity cannot be well decomposed due to the fast change of illumination and illumination in the image, which violates the constraint of illumination smoothness. Based on the color constancy of human visual system, Yoon and Choe used sparse convolutional coding to represent reflectance, which can decompose illumination and reflectivity more accurately, so as to narrow the human-computer perception gap. Zotin [30] proposed a fast image enhancement algorithm based on multiscale Retinex for HSV color model. The linear correlation of RGB color from V component of HSV model was used to calculate the multi-scale Retinex only in the low-frequency region obtained by wavelet transform, which accelerated the image processing speed and enhanced the local contrast.

On the basis of previous studies, the paper proposes a traffic image enhancement model based on illumination adjustment and depth of field difference. Firstly, based on Retinex theory, the depth of field is obtained by using dark channel principle. Then, the image depth is clustered by spectral clustering algorithm. After dividing subimages, the local haze concentration is estimated according to the image depth of field, and the subimage is enhanced adaptively and fused. In addition, the illumination component is obtained by multiscale guided filtering to keep the edge characteristics of the image and adjust the curve function to solve the problem of uneven illumination.

2. Retinex Algorithm Theory

In Retinex algorithm, I (x, y) = L (x, y) R (x, y), where L (x, y) represents the illumination component, the noise, haze, and other interferences of the surrounding environment on the image, which needs to be removed. R (x, y) represents the reflection component, which is the original appearance of the image [26, 31]. The algorithm uses the original image I (x, y) to estimate and removes the illumination component L (x, y) to obtain the actual feature information. In order to reduce the computational complexity, it is converted to logarithmic domain, and the above equation is changed from multiplication operation to addition operation, and the expression is shown in the following equation:

In the single-scale Retinex algorithm [32, 33], the irradiance component is estimated by convolution of the original image and Gaussian surround function . The reflection component can be obtained according to equation (1). The Gaussian surround function is shown in equation (2), and the reflection component is shown in equation (3). In Retinex algorithm, the larger the scale parameter of Gaussian surround function is, the larger the Gaussian template is. Although the enhanced image is smoother and the color is fidelity, the image details are poor. The value of generally does not exceed 300. When the scale parameter is small, the detail enhancement is prominent, and the pixels are easily affected by the surrounding, resulting in color difference and halo. In order to compromise the advantages and disadvantages, as shown in the following equation, the multiscale Retinex algorithm (MSR) is obtained by multiscale weighting method:

In equation (4), , takes low, medium, and high three scales, is the weight, and .

The multiscale Retinex algorithm [30, 34, 35] eliminates the halo and has good sharpening effect, which can improve the balance between detail enhancement and high color fidelity of single-scale Retinex algorithm. However, the restored RGB channels cannot maintain the original proportion, the processed image color is often distorted, and the image color is grayish. In order to solve this problem, Rahman et al. introduced a color recovery factor to compensate for color distortion and obtained Retinex algorithm with color recovery (MSRCR). MSRCR has better control of color deviation and has certain color fidelity, as shown in the following formula: where is the color recovery factor of the jth color channel, and its expression is shown in formula (6). In formula (6), α and β are the adjustment parameters, and is the distribution of the original image in the jth color channel:

3. Image Depth Clustering Based on Spectral Clustering Algorithm

3.1. Image Depth Acquisition

Compared with SSR, MSR and MSRCR consider the influence of different scales on haze image enhancement, but they are still global enhancement in essence, without considering the local differences of images. In order to better enhance the local area of haze image adaptively, the paper calculates the image depth based on the principle of dark channel [3638]. The principle of dark channel is that the image has three RGB channels, and at least one pixel value is low. The principle is based on the atmospheric scattering model [39, 40], and its mathematical expression is shown in equation (7), where represents the image with fog. represents the image without fog, represents the global atmospheric illumination, and is the perspective ratio. In equation (8), is a color channel of and is the dark channel of .

The transmission estimation is obtained by dark channel, and the formula is shown as follows:where is the regulating factor, which is within the range of (0, 1). According to the relationship between the atmospheric scattering coefficient β and the depth of field d, the image depth is calculated as follows:

According to the prior dark channel theory, the depth of field is related to the distance between the object and the photographer, and the haze concentration is positively correlated with the image depth. In this paper, the haze concentration distribution in the image is obtained according to the image depth, and the subimage is enhanced with different scales according to the depth of field of the image, which can enhance the image details and improve the defogging effect.

3.2. Spectral Clustering Algorithm for Image Depth Clustering

K-means clustering algorithm [41, 42] uses distance as evaluation index, finds the best clustering center through iterative calculation, and classifies the data. In the process of calculation, the measurement distance of all data is calculated for clustering. When dealing with high-dimensional data, the distance difference between samples is no longer obvious, resulting in high time and space complexity of the algorithm. Compared with distance-based algorithms such as k-means, spectral clustering keeps the relevant information among samples as much as possible and has higher computational efficiency.

Based on the idea of graph theory, spectral clustering algorithm [43, 44] regards the sample as points in the space and then connects the vertices through edges. The weight of each edge represents the similarity degree of sample points, and the larger the weight, the higher the similarity between samples. Otherwise, the similarity is smaller. A series of subgraphs are obtained by cutting the constructed graph, which satisfies the goal of maximum edge weight and minimum edge weight in subgraph as far as possible. Each subgraph corresponds to a cluster. Spectral clustering method maps the relationship between high-dimensional data and two-dimensional graph to achieve the purpose of dimension reduction. In this paper, the full connection method based on Gaussian kernel distance is adopted, and the normalized cuts (Ncut) method is used to introduce the flow of spectral clustering algorithm.

Input: Sample set ;
The number of clusters is K;
Output: Cluster partition ;
(1)Calculate affinity matrix ;
(2)Calculating degree matrix D;
(3)Computing Laplacian matrix L;
(4)Construction of standardized Laplacian matrix ;
(5)Calculate the first k small eigenvalues and corresponding eigenvectors in ;
(6)Feature matrix F is constructed from the feature vector and the feature matrix is normalized by row;
(7)K-means clustering is used to cluster n samples in F;
(8)Get cluster partition C.

In this paper, spectral clustering algorithm is used to cluster the image depth, so as to obtain subimages with different depth of field in haze scene.

3.3. Adaptive Image Enhancement

Due to the positive correlation between the depth of field and the fog concentration, the local image enhancement can be realized under different haze concentrations. In Retinex algorithm, when the scale is small, the image enhancement details are better, and the overall situation is poor. Retinex algorithm is suitable for the scene with high haze concentration. When the scale is large, the enhancement effect is poor, but the fidelity of image color is better. According to the analysis, image depth and scale also have a positive correlation. In this paper, a variable filter is designed according to the self-adaptive value mapped by different depths of field in different images. After spectral clustering as shown in Algorithm 1, the image is divided into n subgraphs with different depths according to the number of clusters. According to equation (11), the average depth of each subgraph is calculated. , is the total number of depth of field values, and is the jth depth of field value of block i subgraph. The mapping of scale is designed according to the average depth, as shown in equation (12). The scale , is the total number of subgraphs, is the maximum depth of field, is the minimum depth of field, and is the average value of the ith subgraph. Each pixel in the subgraph has a similar depth. is the maximum of scale and is the minimum of scale . Take , where :

The variable filter is as follows:

Spectral clustering as shown in Algorithm 1 is used to cluster the depth image. The scale σ of different depth regions is calculated according to the variable filter. Retinex algorithm with color gain weighting is used for the image according to different scales for many times. According to different depth subimages, n defogging images can be obtained. Each image is enhanced adaptively, and the final output image is obtained by fusing the subimages. The final fusion image R is expressed as . , is the sub image in the ith defogging image.

4. Illumination Estimation Based on Multiscale Guided Filtering

Because the Gaussian filter in Retinex algorithm does not have the ability to preserve the edges and corners, when the filter size is enlarged, the high-brightness area in the illumination component will also be enlarged accordingly, so the phenomenon of white overenhancement around the high-brightness area appears. In order to solve the problem of over enhancement, this paper uses multiscale guided filtering to obtain the illumination component and adjusts the illumination component through the curve function to achieve the purpose of low-brightness enhancement, high-brightness suppression and maintain the edge characteristics.

4.1. Obtaining Illuminance Component by Multiscale Guided Filtering

The maximum value of the three channels at each pixel position is used to estimate the illumination component. As shown in equation (14), represents the maximum value of three channels of image on . A multiscale guided filter with edge and corner preserving characteristics is selected to estimate the illuminance component as shown in equation (15). K is the guide image, r is the filter size, and is the guided filter used on . The illuminance component is obtained by guided filtering.

The size of the filter is set as shown in the above-given formulas, where h and are the height and width of the image. The more detailed information is contained in the illuminance component, the larger size of the filter is.

4.2. Illumination Adjustment

In order to suppress the pixel value of the high brightness area in the traffic image and improve the pixel value of the low brightness area, it is necessary to balance the illumination components after the above processing. The common methods can only adjust the overall image brightness, but many images are nonuniform illumination images, which will cause overenhancement. In this paper, the curve function is used to adjust the brightness of the illumination component to achieve the suppression of the high brightness area and the enhancement of the low brightness area.

As shown in equation (19), is the output image after equalization, is the image to be processed, and are the adjustment coefficients, and the value range is [0.1, 2.0]. Figure 2 is a curve function diagram, the x-axis represents the image to be processed, and the y-axis represents the image after processing. It can be seen that in the curve function, the coefficient γ controls the rate of change of the function. When x ∈ [0, 0.55], the curve is a convex function to enhance the pixel value. When x (0.55, 1), the curve is a concave function to suppress the pixel value. Therefore, by adjusting the coefficient in the function, the illumination component is acted on, and the balanced illumination component Ladj (x, y) is obtained. The traffic haze image is dark as a whole, so it is necessary to improve the illumination of the dark area and suppress the highlight area. Therefore, the value of this paper is set to 0.6, which can reasonably adjust the illumination according to the nonuniform characteristics of the traffic image.

The reflection component of the image can be obtained by equation (20). is the illuminance component, is the pixel value of the color channel, and is the corresponding reflection component. For the convenience of calculation, it can be completed in the logarithm field by equation (21).

5. Experiment and Performance Analysis

MATLAB 2018 is used to compile the algorithm code. The computer is configured with i7 processor, 8G RAM, and Windows 10 operating system. The following figures show the comparison results of our algorithm in haze environment with other algorithms. The experimental scenes include sidewalk, fields, house, intersection, woods, and crossing.

Figure 3 gives the dark channel images of the new algorithm in Figure 4. The images in Figures 3 and 4 are part of the dataset established by our research group. As can be seen in Figure 4, the overall brightness and contrast of the image are relatively low, and some images have the characteristics of relatively high brightness value in small areas. Some areas of HE become noise due to excessive contrast enhancement; some areas become darker or brighter after adjustment and lose detail information. In the nonuniform illumination environment, it is easy to produce color distortion and overenhancement. MSR has better detail and local contrast than HE algorithm, but when estimating the illumination image, MSR always assumes that the initial illumination image changes slowly; that is, the illumination image is smooth. However, this is not the case in reality, so the illumination change of the image is not smooth at the edge of the area with large difference in brightness. Therefore, in this case, the enhanced image in the area with large brightness difference will produce halo, and the image processed by MSRCR is generally white. For the uneven distribution of haze concentration in the image, the MSRCR algorithm cannot achieve local dynamic enhancement, the texture is not clear enough, and the details are not significantly improved. In this paper, the image enhancement algorithm based on illumination adjustment and depth of field difference has a good effect on haze and nonuniform illumination images captured in different scenes. It can enhance the brightness and contrast of the images, and the results of processing have higher brightness and more obvious details. At the same time, it can maintain the natural color, improve the brightness and contrast of the dark area, and suppress the high brightness area.

Next, the obtained image performance indicators are evaluated objectively, and their peak signal-to-noise ratio (PSNR), average local standard deviation, average local information entropy, and structural similarity (SSIM) and Brenner gradient function values are calculated, respectively, to analyze and evaluate the image contrast, detail information richness, and clarity.

The peak signal-to-noise ratio (PSNR) quantifies the damage degree of noise to the original image. The higher the PSNR is, the better the denoised image signal is. For image I and K, the mean square error and PSNR are defined as follows, where the size of the image is :

Local standard deviation is defined as equation (23). When local window block size n is (2d + 1)  (2d + 1), the average value of local standard deviation of the whole image is the average local standard deviation.

Local information entropy refers to the mean value of local information entropy of the whole image. The definition of local information entropy is shown in the following equation, where p (r) is the probability of the pixel value r in the local window and rmax is the maximum value of the pixel value:

Structural similarity measure (SSIM) is used to measure the similarity and distortion of two images. The definition of structural similarity of two images x and y is shown in the following equation: where and represent the mean values of x and y, represents the covariance of x and y, and represent the variances of x and y, and and are constants to maintain stability.

The definition of Brenner gradient function is shown in the following equation, where m and n represent the numbers of rows and columns of the image ƒ. The larger the value of Clty is, the clearer the image is. thr is the limit of Brenner gradient function.

As shown in Table 1, the image performance index of the new algorithm is better than other algorithms, and the contrast, clarity, effective noise suppression, and texture details of the enhanced image are improved.

6. Conclusion

According to the characteristics of complex traffic scene images, based on Retinex algorithm, a new traffic image enhancement algorithm based on illumination adjustment and depth difference is proposed. The algorithm processes the illumination component and the reflection component separately, obtains the image depth of field by using the dark channel principle, divides the image depth into the molecular graph through the spectral clustering algorithm, and estimates the local haze concentration according to the image depth of field. The improved multiscale guided filter estimates the illumination component and adjusts the curve function to solve the problem of uneven illumination. The results show that the algorithm can effectively enhance the uneven illumination and haze images in road traffic scene and avoid overenhancement and halo blur in local area. Compared with other algorithms, the effect of the new enhancement algorithm is better.

Data Availability

All data used to support the study are included within the article.

Conflicts of Interest

The authors declare that they have no conflicts of interest regarding the publication of this paper.

Acknowledgments

This work was partly supported by the Key Laboratory of Intelligent Industrial Control Technology of the Jiangsu Province Research Project (JSKLIIC201705) and General Project of Natural Science Research in Universities of Jiangsu Province (20KJB170023).