I LLUMINATION E NHANCEMENT OF N IGHTTIME I MAGES U SING A R EGULATED S INGLE S CALE R ETINEX A

People are active during the nighttime and capture many photos. Due to the low-light nature of the environment, captured images tend to appear dimmed with imbalanced illumination, low contrast, noise, and a lack of vibrant colors. For this purpose, this paper presents a direct and practical approach to improving the lighting of night images based on the single-scale retinex model and using image processing methods and other statistical approaches. The proposed algorithm begins by converting the image from the RGB to the HSI model. Then, it enhances only the I channel while preserving the H and S channels. Thus, processing on the I channel begins with estimating the illumination version of the image and calculating the logarithms of both the illuminated and original image, like the SSR model. Then, it subtracts these two images using a subtraction of logarithmic image processing. Subsequently, the cumulative distribution function of the Gumble probability distribution is applied, and the resulting output is further treated using a logarithmic transformation method. That produces the processed I channel, which combines with the conserved H and S channels to give the HSI image. Ultimately, it converts the image to RGB format. The proposed algorithm is applied to two datasets of nighttime images, compares their performance to seven different contemporary algorithms, and evaluates the results of the compares using three specific metrics of quality evaluation based on the results generated, deducing that the suggested algorithm enhanced the brightness of night images and surpassed other algorithms in terms of execution time, and image assessment methods. Additionally, it exceeded them in all metrics used.


INTRODUCTION
Nighttime is the period from dusk to dawn [37].Images taken at nighttime are of defective quality and characterized by unbalanced illumination, unpleasant colors, limited contrast and undesirable noise [1].Due to the significant increase in nighttime photography to visualize large-scale events, such as personal activities, surveillance and speed cams, there has been an urgent need for efficient nighttime imageenhancement algorithms.Thus, this topic has attracted widespread attention by various beneficiaries.Since hardware is constantly improving, most modern devices and computer-vision applications are required to deliver high-quality images [2].Image enhancement (IE) refers to the operations applied to an image to improve its perceived quality and make it more visually pleasing to the recipient.The primary goal of IE is to change the characteristics of an image to enhance its suitability for a particular activity and viewer without introducing errors [3].IE techniques must seek to consider two crucial factors: 1) There may be hidden noise in dark areas of nighttime images, so it must be ensured that the noise is suppressed or kept from being amplified when improving the illumination [4].2) Preserving the brightness in the already bright areas from being amplified to avoid the state of over-enhancement [5].
Different algorithms have been introduced to help improve the quality of digital images.One concept of interest is the Retinex theory, which is commonly used for image enhancement and owns many versions, such as the single-scale retinex (SSR), multi-scale retinex (MSR) and multi-scale retinex with color restoration (MSRCR) [6].This research introduces a well-developed SSR algorithm using statistical and image-processing methods for nighttime-image enhancement.The performance of the proposed algorithm has been tested on two datasets, compared to ten contemporary algorithms explained in the related-work section, in addition to evaluating and discussing the results thoroughly.The paper is organized as follows: the 2 nd section explains a literature review of recent years' research work, the 3 rd section explains the developed algorithm in depth, the 4 th section presents the attained results and the 5 th section gives a brief conclusion.

RELATED WORK
In recent years, numerous studies have been introduced on improving nighttime images due to their high importance in different real-world applications.The selected studies are reviewed in a newer to older style.In 2024, a method that utilizes gamma correction and merged color spaces is introduced [35], in that the algorithm starts by determining a transmission map (TM) that includes the saturation information of the degraded image in two different color spaces.Next, the calculated TM is transformed into a function that contains the max and mean values and these values are approximated from a poor illumination image by utilizing a gamma-correction approach.After that, an adaptive valuedetermination algorithm is applied to enhance the image, prevent the over-enhancement phenomenon and generate the output.In 2023, a Gaussian-based model (GM) was developed [34] and this algorithm starts by creating the GM to get the estimated reflectance and illumination information based on the retinex theory.Then, based on the retinex theory, a decomposition in the GM-based operation is applied to the illumination layer and a gradient descent-based approach is implemented to enhance the image's illumination.Lastly, a denoising process based on the total-variation concept is executed on the reflectance layer to reduce the noise and generate the output.
In 2023, a triangle similarity-based algorithm (TS) was presented [33], in that it begins by transforming the image into the HSI color domain and maintaining the hue channel while processing the saturation and intensity channels.Next, a translation-based operation is applied to the saturation channel to improve the color representation.After that, various scaling operations are implemented in the intensity channel to improve the illumination and visual information.Lastly, a transformation to the RGB domain is applied to create the output.In 2022, a structure preservation-based variation model (SPV) was provided [32] and it started by utilizing a variation-coefficient-based concept to improve the illumination information.Next, a total-variation concept is implemented to reduce the noise information in the image.Lastly, these two images are mixed using the retinex concept in an iterative way to generate the output.In 2021, a progressive-recursive network-based algorithm was established [36], in that the method begins by getting the degraded image and sending it to a dual-attention approach to extract the global features.After that, a mixture of residual blocks and recurrent layers is utilized to extract the local features.Based on the extracted local and global features, several recursive operations are applied to enhance the image and create the output.
In 2020, a semi-decoupled decomposition (SDD) algorithm was proposed [7], in that it decomposes the image using the retinex model into reflectance and illumination components in a semi-decoupled manner.The illumination layer is enhanced progressively and the reflectance layer is improved jointly using a specialized total-variation concept.These components are united to create the output.In 2020, a retinex-based multi-phase (RBMP) algorithm was proposed [8], which is initiated by computing the illumination image in a manner akin to the standard SSR algorithm, subtracts the log of the illumination image from the log of the original image using a modified method and then processes the output through a gamma-corrected sigmoid and normalization approaches to generate the output.In 2019, an adaptive image-enhancement (AIE) algorithm was presented [9], where it first transforms the image to the HSV domain and the V channel is processed to isolate the illumination component of the scene through a multi-scale Gaussian function.Afterward, a correction function is implemented via the Weber-Fechner law and two outputs are generated by adaptively adjusting the parameters according to the distribution profiles of the illumination components.The output is created by combining both images using a specially-developed approach.Similarly, in 2019, an algorithm named LECARM was developed [10], which began by utilizing illumination-estimating algorithms to calculate the exposure ratio for every pixel.After that, the chosen camera-response model is employed to modify each pixel to achieve the required exposure based on the estimated exposure-ratio map.Lastly, the output is obtained using a specific mapping method.Moreover, in 2018, a robust retinex model (RRM) algorithm was presented [11], which starts by applying advanced regularization terms for illumination and reflectance approximation.More precisely, it employs one norm to limit the smoothness of the illumination in different regions, joining a fidelity term to highlight the structural details in low-light areas with the gradients of the reflectance to estimate the noise map using a robust Retinex concept.Next, the enhancement is applied using a Lagrange multiplier-based approach to build the output image.In 2016, a fusion-based enhancement (FBE) algorithm was developed [12], which utilizes an illumination-estimation algorithm based on morphological closure to separate an observed image into reflectance and illumination components.This algorithm generates two images from the illumination image, one with brightness enhancement and the other with contrast enhancement, by applying sigmoid transform and adaptive-histogram equalization.Moreover, two weights are created using a multi-scale process.Lastly, the two images are fused using the determined weights to create the output.In 2016, an algorithm called LIME was proposed [13], which starts by determining the max values of the RGB image, followed by the determination of reflectance and illumination information via the retinex model.The illumination information is enhanced using a structure-processing concept, followed by implementing different maps to further boost illumination.The output image is generated by joining the improved-illumination and reflectance components.
As seen from the studied algorithms, different notions were used and the computational cost of each algorithm varies.Most of the proposed algorithms in this field did not reach the required level of enhancement.Thus, the chance remains to develop a new method that can improve the illumination of nighttime images more efficiently.The proposed algorithm differs from existing algorithms in several aspects.First, low computational developments are utilized to make the proposed algorithm efficient and particularly fast in filtering different nighttime images.Second, the utilized developments improve the illumination in a direct and non-iterative way while considering minimal noise augmentation, which is needed as many existing algorithms utilize the iterative feature and their utilized processing steps may amplify the noise, which leads to the requirement of another major step for image denoising, making such algorithms slow and require high computational cost.

PROPOSED ALGORITHM
Land and McCann initially proposed the retinex theory [30].The term "retinex" is derived from the combination of the root terms "retina" and "cortex," which are both essential components of human vision.Retinex is more visually consistent with human vision.This is predicated on the notion that the reflectance and illumination components' collective influence creates the image, as shown in Figure 1.Specifically, when light illuminates an object, it creates a reflection that is then seen as an image by the human eye [6].Various algorithms have been designed based on the retinex theory, such as the SSR [14] and the MSR [15].Both algorithms utilize a specific Gaussian function to modify a given image.Therefore, the brightness level of the output image is determined by using the natural logarithm of reflectance.Nevertheless, it may exhibit a color-distortion effect, which poses a difficulty in both the SSR and MSR.A potential solution to address this problem is the implementation of a "Multi-scale Retinex with Color Restoration" (MSRCR) technique [16].Here, the inclusion of MSRCR allows for the handling of color distortion and restoration by utilizing the color ratio of the red, green and blue (RGB) channels.However, due to the universally-applied mapping curve, this method tends to diminish the level of detail in the image, particularly in the areas of high brightness.
The primary motivation of the proposed regulated SSR (RSSR) algorithm is to improve the quality of night images by improving lighting in dark areas without intensifying brighter regions using noncomplex concepts.The SSR model and other less-complex statistical concepts and methods were used among these concepts.The original SSR model is used for illumination estimation of degraded images by applying convolution (*) between the input image I(x,y) and the Gaussian function G(x,y), which is calculated in the following manner [17]: Let Q be a normalizing factor; x and y represent the coordinates of the digital image; U and V denote two grayscale gradients, where U is horizontal and V is vertical.U and V hold the same size as I(x,y).
Additionally, M and N represent the dimensions of I(x,y), (•) denotes a multiplication operation and σ is a parameter that addresses the brightness.Then, the logarithm of the illumination image resulting from the previous step and the logarithm of the degraded image are taken to produce an enhanced version of the degraded image called reflectance image R(x,y) by subtracting the log of the illumination image from the log of the degraded image [14], in the following manner: where R(x,y) is the output of original SSR.Experiments have been conducted on applying the standard SSR on different nighttime images to determine its filtering abilities with this type of image.Some results are demonstrated in Figure 2. From the conducted experiments, the SSR provided resulting images with defects, including extra dimming for the darkened areas, which led to the loss of visual details, as well as amplification of brightness in the bright areas and the production of unrealistic colors, leading to overall unacceptable results.Regardless of these defects, the standard SSR model is characterized by low computational cost, which is a key aspect and has a high development potential [14].
The proposed RSSR algorithm aims to improve illumination while producing appropriate colors and avoids the over-amplification of the latent noise.The RSSR algorithm begins its first phase by converting the image from the RGB form into the HSV color model [18].This color model is designed to efficiently separate the color information from the brightness (value) information, making it intuitive to improve brightness by simply modifying the value component.Supposing that the input image I(x,y) has three color channels of red (R), green (G) and blue (B) and R, G, B ∈ [0, Wm], with Wm being the max.range value (typically 1), assuming the range ∈ [0,1], the conversion to the HSV color domain can be achieved using the following equations [31]: with Wh, Wl and Wr defined as Wh = max(R, G, B), Wl = min(R, G, B), Wr = (Wh -Wl), where S is the saturation channel and V is the value channel.What is more needed is to determine the hue (H) channel, wherein if the three RGB channels contain a similar value, then it is the case of a gray pixel.In this situation, Wr = 0, S = 0 and H is undefined.To calculate H when Wr > 0, each channel is normalized in the following manner: , , .
Next, the initial hue (Ĥ) is calculated based on the notion of which color channel contains the max.value in the following manner: The outcome value of Ĥ is in the range of [-1, 5] and the final H channel is obtained in the range of [0,1] as follows: The operations are performed only on the value channel, because the key requirement here is to improve the illumination, as the HSV color domain separates the color information from the illumination information.Thus, the processing becomes rapid and efficient.In the second phase, the Gaussian function G(x,y) is calculated using Eq. ( 1) and Eq. ( 2), where (σ = N×M).The third phase includes the computation of the illumination image in the following manner [17]: where, M(x,y) is the illumination image.To apply the convolution (*) in the frequency domain, first, the Fourier transform is used to convert the inputs from the spatial domain into the frequency domain.Then, the element-wise multiplication between two inputs of the same size is computed.It often needs a frequency shift to return the high frequencies in the middle and the low frequencies in the edges and finally convert the image from the frequency domain into the spatial domain [19].In the fourth phase, the log of the illumination image M(x,y) and the log of the input image I(x,y) are determined as follows: Here, (ε = 0.001) represents a minor value added to prevent the computation of the log of zero, which is infinite.The fifth utilized phase includes the application of a logarithmic-subtraction approach [20], as logarithmic image processing has been utilized in dynamic range manipulation, improving the visibility of details in both dark and bright regions, replacing the standard-subtraction method in Eq. ( 3) to produce the reflectance image, as follows: After that, the sixth phase is implemented, which includes the utilization of a slightly modified cumulative distribution function of the Gumble probability (CDF-GP) approach.The standard CDF-GP approach can be mathematically expressed as follows [21]: This approach redistributes the values across the image, emphasizing certain brightness levels over others and improves the contrast depending on β.With a slight heuristic modification to simplify the calculation, its equation becomes as follows: (14) where β > 0 is the parameter that controls the image illumination and contrast, in that lower β values compress the range of values, reducing illumination and contrast.In comparison, higher β values spread out the intensity values, potentially enhancing illumination and contrast.Next, the log transform is applied as the seventh phase to further improve fine details in low-density areas.This transformation is appropriate for an excessively dark image, as it increases the values of dark pixels and decreases the values of highly-illuminated pixels [22], resulting in a well-balanced, visually pleasing outcome.The log transform can be computed as follows [23]: where S(x,y) represents the resulting value channel and c is a luminance parameter that is set to 2.5.In the final eighth phase, a conversion from HSV to RGB is applied.To convert the HSV image, where ∈ [0, 1], to the corresponding RGB image, the following is applied [31]:   ˆ6 mod 6 HH  (16) where (0 ≤ Ĥ < 6) is initially obtained, then the intermediate values are calculated as follows: Using these pre-determined values, the normalized RGB channels are computed as follows: Finally, scaling the channels to the range of [0, A-1] (normally A = 256) is done in the following manner: where RGB is the final algorithm output.The flowchart of the proposed RSSR algorithm is demonstrated in Figure 3.
Figure 3. Flowchart of the proposed RSSR algorithm.

RESULTS AND DISCUSSION
In this section, the results of the proposed algorithm are presented and its performance is evaluated on low-light nighttime images.These results are also discussed and compared to the results of other algorithms.In this study, two datasets were used.The first is the MIT-Adobe FiveK dataset which contains five thousand images captured using single-lens (SLR) cameras by different photographers, wherein the images are all in RAW format, meaning that all data captured by the camera sensor is pristine.Photoshop was used to convert these images from the DNG format into the JPG format.The second one is the exclusively Dark (ExDARK) dataset [25], containing approximately seven thousand images captured in low-light conditions.
When it comes to the comparison, the proposed algorithm is compared with ten advanced methods; namely, SDD [7], AIE [9], FBE [12], LECARM [10], LIME [13], RBMP [8], RRM [11], SPV [32], TS [33] and GM [34].Moreover, the results of the proposed method and other methods are evaluated using one reduced-reference (RR) metric, called the lightness order error (LOE) and two no-reference (NR) metrics that are natural image-quality evaluator (NIQE) metric and blind/referenceless image spatial quality evaluator (BRISQUE).The LOE [27] is utilized to measure the error of the lightness order (i.e., illumination quality) between the degraded image and its filtered counterpart.The output of the LOE is a numerical value, where lower scores represent a better illumination quality.The LOE is defined as: The variables W and H represent the image dimensions and D(x,y) denotes the relative order difference in luminance between two given images.Moreover, the NIQE [28] measures the naturality and evaluates the quality based on measurable deviations from statistical patterns found in natural images without considering expected distortions or human subjective judgments.The quality of the distorted image is quantified by measuring the difference between the statistical properties of the model and the distorted image.The output NIQE is a numeric value, where lower scores represent better naturality.Likewise, the BRISQUE [29] measures the distortions and perceived quality and utilizes natural scene statistics (NSS) to construct a distortion-agnostic no-reference metric for image quality that functions in the spatial domain.NSS focuses on analyzing the statistical patterns seen in "natural scene" photos and developing metrics to quantify the extent to which the statistical properties of an unfamiliar image differ from those of typical natural scene images.The output BRISQUE is a numeric value, where lower scores represent low distortion and high quality, which is deemed better.In brief, the NIQE measures the naturality, the LOE measures the illumination quality and the BRISQUE measures the existence of distortions.
As for computational complexity, CPU runtime can deliver insights into an algorithm's efficiency and complexity [26].Let's dissect it: the computational complexity measures the number of resources required by a method to solve a problem.It's usually quantified in terms of space and time complexity.The CPU runtime, on the other hand, denotes the real time needed by a CPU to implement a specific method.It relies on numerous aspects, such as the method's complexity, the input size and the hardware utilized.Comparative analysis (CA) can be applied to this case.CA means that comparing the CPU runtimes of various methods for the same task provides a sense of relative computational complexities, in that a method with a lower runtime for the same input size denotes a lower computational complexity.Thus, CPU runtimes have been considered as a computational complexity measure and provided in this study in Table 4 and Figure 13.The computer on which experiments and evaluations were performed had specifications of 16 GB of RAM, a Core i7-8650U 2.11 GHz processor and MATLAB 2020a.1-4 show the recorded scores and implementation times for the compared algorithms.Finally, Figure 12 and Figure 13 display the graphs of the average performance in Tables 1-4.
As in the given samples of the conducted experiments, the proposed algorithm succeeded in improving the quality of nighttime images in that it illuminates dark areas while preserving the illumination of the bright regions from being extremely amplified, in addition to emphasizing the visual details of the filtered images.This balance benefits in maintaining the natural illumination while improving visibility in darker image parts.Moreover, the output images from the proposed RSSR algorithm have vibrant, eye-comforting colors with acceptable contrast, bearing in mind that the proposed method does not add any distortion or unwanted artifacts during the processing procedure and prevents the noise from being massively augmented.This guarantees that the processed images stay true to the pristine scene without    presenting any visual irregularities.Moreover, this also indicates that the RSSR algorithm not only enhances visibility, but also improves the images' visual appeal.In addition, the calculations are low and therefore, the proposed method has great potential in night-image processing, making it suitable for resource-constrained applications.From the outcomes of the performed comparisons, it is observed that each method provides different enhancement modes due to the different used processing notions, wherein the analysis of each method depends on aspects, such as quality of illumination, contrast, colors, sharpness, in addition to the generation or increase in noise, artifacts or errors.SDD provided insufficient illumination with a smoothed appearance and brightness amplification.It's why the metrics readings are low and the processing speed is slow due to the implementation of the noise-reduction process.AIE delivered the second-best reading in terms of LOE compared to the other methods.However, the unnatural tonality and noise generation led to scoring poorly in NIQE and not good in BRISQUE.Still, it recorded the second fastest method in terms of processing time.FBE recorded low and unusual brightness and contrast but with adequate sharpness.Thus, LOE readings were not good, but BRISQUE and NIQE readings were agreeable and the processing speed was considered acceptable.
Likewise, LECARM produced images with insufficient lighting and had white shadows around the edges.Thus, LOE readings were unacceptable, but the BRISQUE and NIQE readings were reasonable with relatively fast processing speeds.LIME introduced brightness amplification, unusual illumination, processing errors and boosted colors.That is why the LOE readings were the worst among the competitors, yet they averaged in terms of BRISQUE and NIQE with above-average processing speed.
In addition, the RRM algorithm provided average illumination with over-smoothness.Due to that, the LOE readings were mediocre, but due to the over-smoothness, the readings of BRISQUE and NIQE were very low.As for the processing time, it was the worst as it took an extremely long processing period.Moreover, RBMP delivered adequate illumination with somewhat pale colors.Thus, the LOE readings were satisfactory as well and the BRISQUE and NIQE metrics recorded the second-best results, considering that it did not generate distortions, provided slightly pale colors and was noticeable fast.
GM delivered results with limited brightness, imbalanced contrast and slightly pale colors, scoring below average in LOE, low in BRISQUE and NIQE and with slow performance according to the average runtime.TS proved to have low illumination and artifacts in the results, leading to low LOE, BRISQUE and NIQE readings with fast runtimes.SPV increased the illumination and surged the difference between the brightest and darkest regions in the image, leading to somewhat average readings according to the utilized metrics.When it comes to the proposed RSSR, it outperformed all the other comparison algorithms subjectively and objectively, as it recorded the best readings according to LOE, BRISQUE and NIQE metrics with the fastest execution time.It is essential, because it is infrequent to have an algorithm that produces high-quality results rapidly without generating distortion or massive noise presentation.In this context, the proposed algorithm excels and its performance is considered positive and distinctive for the desired purpose, improving the illumination of nighttime images.When developing the RSSR, it has been affirmed that proper illumination must be provided with balanced contrast and attractive colors and focused on avoiding the generation of unwanted processing errors in addition to evading noise amplification.Thus, the methods used in the development of the RSSR were added and adapted successfully to introduce a fast and efficient algorithm.Despite the accomplishments of the algorithm, it still has one limitation; that is, it is not fully automatic and the human operator should manually choose the value of β to produce the resulting image with the desired illumination.

CONCLUSION
This research proposes an algorithm to improve the illumination of nighttime images.This algorithm works on the HSV color model and estimates the illumination image in a similar way to the standard SSR model.Still, it differs in the subtraction process, as it uses logarithmic subtraction in addition to the utilization of two statistical approaches for further visual enhancement, in that the first is a modified CDF-GP approach, which applies a curvy transform and the other one is a non-complex log transform.The performance of the proposed algorithm is assessed by utilizing two different datasets.By performing a comparison with ten contemporary algorithms, the obtained results are then evaluated using three metrics and recorded CPU runtimes.The study's outcomes showed that the proposed RSSR algorithm improved the quality of nighttime images and properly illuminated the details in dark areas while avoiding over-illumination of bright areas, producing images with natural and balanced brightness, adequate colors and adjusted contrast.As a result, the proposed RSSR algorithm outperformed the other algorithms in the used objective measures and recorded the fastest runtime.This is essential, as it is challenging to find an algorithm that is uncomplicated and fast and, at the same time, generates satisfactory results.In future work, it is likely to embrace developments by including AI for automation.

Figure 2 .
Figure 2. Outputs of the standard SSR model when applied to different nighttime images.

Figures 4 -
7 show the experimental results of the proposed algorithm with various degraded nighttime images, Figures 8-11 demonstrate the comparison results.Moreover, Tables

Figure 13 .
Figure 13.Average readings of NIQE and runtimes.The proposed RSSR outperformed the other algorithms in the metrics used because of the careful development and attentive analysis of the drawbacks of the related-work methods, knowing what advantages to consider and what disadvantages to avoid.When developing the RSSR, it has been affirmed that proper illumination must be provided with balanced contrast and attractive colors and focused on avoiding the generation of unwanted processing errors in addition to evading noise amplification.Thus, the methods used in the development of the RSSR were added and adapted successfully to introduce a fast and efficient algorithm.Despite the accomplishments of the algorithm, it still has one limitation; that is, it is not fully automatic and the human operator should manually choose the value of β to produce the resulting image with the desired illumination.