Big Data-Oriented Novel Background Subtraction Algorithm for Urban Surveillance Systems

: Due to the tremendous volume of data generated by urban surveillance systems, big data-oriented low-complexity automatic background subtraction techniques are in great demand. In this paper, we propose a novel automatic background subtraction algorithm for urban surveillance systems in which the computer can automatically renew an image as the new background image when no object is detected. This method is both simple and robust with respect to changes in light conditions.


Introduction
Big data research is attracting a great amount of attention due to the seemingly infinite generation of huge data worldwide. As one example, big dataoriented techniques have emerged as an important research topic for smart cities and urban surveillance systems [1] [2].
Urban surveillance systems are important applications for smart cities [3] [4]. These systems use automated object detection methods whereby cameras are installed to automatically detect vehicles and other objects. Because of the tremendous volume of data generated by urban surveillance systems, to obtain useful information from the huge numbers of images and videos, low-complexity techniques that can automatically identify objects from various sources are in high demand. Hence, automated or so-called automatic object detection algorithms have become an important research topic. Currently, the algorithms under investigation can be classified into three main groups: frame difference, background subtraction and optical flow calculation methods.
The frame difference algorithm is used to analyze the image sequences of two or more adjacent frames to identify moving targets by calculating the differences in continuous frames. For every pixel, if the difference is larger than a set threshold, the result is labelled 1, else 0. The larger is the threshold value, the less noise there is [5] [6]. To detect and track moving targets, some authors have proposed combining the frame difference method with the particle filter algorithm [7]. The shortcoming of the frame difference method is that it is not able to detect nonmoving objects, since there is no difference between adjacent frames.
The second type of automatic object detection algorithm is the background subtraction method. This method first builds a scene background model and then uses the current frame image to subtract the background model. In this way, targets are identified. This is a widely used method for retrieving targeted objects. Background subtraction performance depends mainly on the background modelling technique used, and Gaussian mixture models (GMMs) are currently the most popular. In [8], the authors combine a GMM with a classified foreground in an interpolated red, green, and blue (RGB) domain. Since one problem of GMMs is that they cannot effectively model noisy backgrounds, to address this shortcoming, the researchers in [9] utilize an advanced distance measure based on support weights and a histogram of gradients to achieve background suppression. Current background subtraction methods (such as GMMs) are also limited by their high computational complexity [8] [9], e.g., the large number of calculations required limits their practical application in real-time as reconstructions and background updates take too long.
The third type of automatic object detection algorithm is based on optical flow (OF) calculation [10]. This method estimates motion scenes and combines similar motion vectors to detect moving objects. However, the OF method is also very computationally expensive. Another major shortcoming of the OF method is that it assumes that an object is always moving, which is not necessarily true.
In this paper, we propose a novel low-complexity automatic background subtraction method that is computationally efficient and does not require that objects be continuously moving.

OUR PROPOSED METHOD
The basic idea of our proposed algorithm is to detect whether or not there are objects (e.g. cars, humans) in each image. If no object is detected, we update the current image as the background. In this way, the background is renewed in real time, which is robust with respect to changing light conditions. Compared to existing methods, which are computationally expensive, our method is computationally efficient. Another advantage of our method is that it does not depend on object motion; even nonmoving objects are detected automatically. In [11], we proposed a simple filter for detecting vehicles or license plates from captured images. We found that an object (for example, a vehicle or human being) is normally the highest energy frequency aspect of an image and the energy frequency curves decrease sharply outside object boundaries. In this paper we improve filter design for detecting and updating the background. Our proposed filter can automatically determine whether there are objects in the images.

Pre-treatment of the captured image
Taking the example shown in Figure 1, our objective is for a computer to automatically determine whether there are objects on a road based on this captured image. To do so, the computer determines whether or not this image can be used as a background. Using the pretreatment method proposed in [11], we first transfer the original image into a gray-scale image to reduce the image data that must be calculated.

Fig. 1 Original image.
After generating the grayscale image, we calculate its gradient using P x to denote differences in the x direction (horizontal) and P y to denote differences in the y direction (vertical).
As in [11], after calculating the gradients of both the horizontal and vertical directions, for every pixel, we obtain the overall gradient of each pixel as follows: (1) Figure 2 shows the gradient of this grayscale image, in which we can see that some large areas of constant grayscale fade into darkness due to the slow gradient movement. Only the outline of vehicles and some other background objects are visible.

Design of our effective filter
In [11], we designed a smart filter based algorithm that scans and identifies the object areas in an image. This method was inspired by the mathematical definition of intercorrelation and the function pulse δ(x), which has the following important characteristic: This means that the function δ(x − a) can be utilized to filter out another function ϕ(x) at the x-axis when x=a. We use this property to filter out high frequency areas in an image.
We use a matrix array to calculate intercorrelations in an image. Since the matrix array is centrosymmetric, the image intercorrelation process can be considered to be a convolution process as follows: where G(n) represents the original image and h(n) is the matrix array. F (n) denotes the image after convolution.
As we know, the results obtained after convolution reflect the overlapping relationship between two convolution functions. In mathematics, convolution can be regarded as a weighted summation.
In the filtered image F (n), since spectrum information is displayed on the horizontal axis as brightness, by simply choosing the middle horizontal array line of the filtered image, we obtain the onedimensional brightness values. These values clearly reflect high frequency areas of objects on the horizontal axis. Our proposed filter successfully reduces the two-dimensional image into a one-dimensional curve, which significantly reduces the data required.
Since the matrix filter proposed in [11] has a bar shape, in this paper, we refer to it as the bar filter. As we know, in the bar filter, all the numbers are set as 1's and can be considered to be constituted by some line arrays.
The bar filter incurs a lot of calculations. To reduce the computational load and still obtain satisfactory convolution results, we propose a new and improved filter in which we have removed some line arrays from the original matrix array. As such the new filter looks like a grate, so we refer to it as the grate filter, a diagram of which is shown in Figure 3. In this study, we set the width of the bar filter as eight, which means there are eight columns of 1's integrated into this filter. In the grate filter, we set the first and the fourth columns as 1's, which means there are three columns of 0's between the 1's columns and a total of only two columns of 1's remaining. From the definition of convolution, we know that in this case the computational load of the grate filter is just one quarter that of the original bar filter. Therefore, it is clear that the first advantage of the grate filter is its significant reduction in the computational load. We filtered the pre-treated gradient image in the horizontal direction using the two different filters, and then plotted the one-dimensional brightness values for the straightforward view by choosing the middle horizontal array line of the filtered image. The resulting values obtained in the x-axis are shown in Figures 4 (bar filter) and 5 (grate filter).
We simply scan the horizontal brightness values from left to right and choose the threshold at 0.5 for the horizontal axis. To eliminate the impact of unwanted brightness, we check the widths between the horizontal points against a threshold value and discard very narrow results. If at least one of the detected widths is wider than the predefined width, the system indicates the presence of at least one object in this image. In this case, since at least one object (i.e. a vehicle) is detected, the computer reaches the decision: "Object Detected", which means the original image cannot be used to renew the background. Figures 4 and 5 both indicate that at least one object was detected. Considering the reduction in computational complexity, the grate filer outperforms the bar filter. Moreover, the grate filter obtains more detail in the brightness values of the horizontal axis.   The time period for capturing a new image may range from several seconds to several minutes, as the setting is dependent on the actual situation in the surveillance area.

Flow chart of the proposed algorithm
In the process of searching for objects in an image, sometimes several different objects are detected. Because our task is to decide whether or not this image can be used to renew the background, it is not necessary to show all the objects in a resulting image. We use a left-to-right rule and choose to show the first object detected in the image. In this situation, the previous background image is unchanged, and after a set time period, a new image will be taken into consideration.
On the other hand, when no object is detected in the image, we choose this image to update the background image, and our program shows the original image with the lable "Object not detected". Then the same action is repeated, i.e., after the set time period, a new image will be taken into consideration.

EXPERIMENTAL RESULTS
Next, we tested our filters to test their performance on different images. We used various images captured on different roads to test the two filters. Figures 7 to 15 show the original images and identified results.  There are several cars in Figure 7 and the corresponding horizontal axis' brightness values (Figures 8 and 9) clearly show several high-frequency areas, which indicate that there is at least one object in the image. Both filters identify these cars and show the first detected red car. The difference between them is that the grate filter shows more details in the horizontal axis's brightness values. Both images obtain the result "Object Detected", which means that the original image cannot be used as a background image.

Original Image
Original Image There are numerous vehicles (buses and cars) inside Figure 10 and the corresponding horizontal axis' brightness values (Figures 11 and 12) indicate that there are several high-frequency areas, from which we can conclude that there are some objects in this image. Both filters detect these vehicles properly. Again, the grate filter shows more details than the bar filter in the horizontal axis's brightness values. And again it is clear that the original image cannot be used as a background image. Fig. 13 Cars on the road.

Original Image
In Figure 13, we see that there are two cars travelling very close to each other. The corresponding horizontal axis' brightness values (Figures 14 and 15) show two high-frequency areas, that are very close to each other. Although both filters detect the fact that there are objects on the road, the filters achieve different degrees of precision. Since there are some smoothing effects associated with the bar filter, the two cars detected are mixed together in Figure 14. In contrast, the grate filter outperforms the bar filter in detecting the details. The two cars are separate, and the first car is clearly selected by the grate filter, as shown in Figure 15. Next, we used some images of an empty road to test our proposed algorithm, the original images and In Figure 16, there are no vehicles or people on the road. However, we see that there are several flower beds and lamp posts in the image which may affect our search process. Figures 17 and 18 show that the flower beds and lamp posts are not wide high-frequency areas. The high impulses at the right end of each figure is from the edge of the original image. Although the noise level does not meet our threshold requirement, we can see some differences in the results of the two filters. With the grate filter, the noise values are lower, not even reaching 0.5, which is a better result. Lastly, both results show the label: "Object not detected". So this image is properly recognized as a background image. Figure 19 shows another image of an empty road in which the background contains some trees, buildings, and several lamp posts. Figures 20 and 21 show that these objects are associated only with some narrow  high-frequency areas and, again, the high impulses at the right end in each figure is from the edge of the original image. Although the grate filter provides more detail in the resulting brightness values, the noise level does not meet our threshold requirement, so they will not be considered as objects. Lastly, both filters conclude: "Object not detected", which properly indicates that the original image can be used to update the background.   (Figures 23 and 24) reflect the presence of this high-rise transmission tower. We can see that the grate filter yields lower brightness values than the bar filter, which makes the computer's decision more accurate. Lastly, the decisions from both filters are "Object not detected", so the original image can be used to update the background. noise and the decisions of both filters are correct, i.e., "Object not detected". Since no objects were identified in this image, it can be used to update the background.

Conclusions
In this paper, we proposed a new grate filter that can identify whether there are objects in images captured by cameras in urban surveillance systems. With this technique, background images are updated from time to time when no object is detected. The proposed method provides a low-complexity solution that facilitates the setup of the background image for urban surveillance systems, and thus makes the foreground detection process easier. This method requires no complex computation for background detection. In addition, it is robust with respect to the impact of changing light conditions and does not depend on the motion of objects. We tested and compared the proposed grate filter with the bar-shaped filter. The results confirm that the grate filter obtains greater detail on the intercorrelation results of images with a lower computational load. His main research lies in the area of Internet of Things, big data analysis and cloud computing. He had published over 20 papers in these areas and had successfully pushed multiple research findings into the market.