F AST DETECTION STUDY OF FOREIGN OBJECT INTRUSION ON RAILWAY TRACK

: The foreign objects intrusion on railway track has seriously affected the safe operation of the train, and it is extremely urgent to monitor them in real time. In order to improve the detection accuracy and rapidity of foreign objects intrusion on railway track, the new detection method of foreign object intrusion on railway track based on multi-background modeling, multi-difference and proportion method of black and white pixels is put forward in this paper. The multi-background modeling method that includes the historical background modeling, the multi-frame average background modeling and the previous frame of current frame background modeling method is used to model background modeling, and the three backgrounds are updated respectively to achieve background updating. The improved Canny method and Hough transform method is used to extract track edge, and get the final track edge image. Based on track edge image, the railway track dangerous area was established through the image segmentation method to reduce the amount of information in image processing and improve the processing speed. And then, according to the structure method of multi-background modeling, the detection method that fuses the historical background difference, average background difference and inter-frame difference is used to detect foreign object intrusion on track, and the detection result was processed by the morphological open processing. Finally, for the foreign objects intrusion, the decision is done by the quantitative proportion method of black and white pixels of image. The experimental results show that this method has better noise immunity performance and environmental adaptability, and the accuracy and rapidity of foreign objects intrusion detection is improved effectively.


Introduction
The foreign objects intrusion on railway tracks has seriously affected the safe operation of trains, and some foreign objects intrusion have also caused serious accidents, such as the rolling stones that suddenly fall from the valley and tunnel entrance, the illegal intrusion of pedestrians, animals, vehicles, etc., which greatly threaten the safety of passenger and train running, once an accident occurs, it may lead to serious consequences (Hou & Li, 2017;Hu et al., 2012;Wang et al., 2014;Yao et al., 2015;Zhang, 2015).On January 31, 2018, a train collided with a garbage truck in Crozet County, Virginia, America, one person on the garbage truck was killed and one person was seriously injured, some people on the train suffered minor injuries.On January 25, 2018, a train derailment occurred on the 10452 trains with 350 passengers on the roads between Segrate and Pioltello in Milan, Italy, resulting in three fatalities and 46 injuries.In February 2011, a train and bus collision occurred in Bangladesh, causing more than 50 casualties.In April 2010, an Italian train derailed due to the collapse of the mountain, resulting in more than 30 casualties.In May 2010, the railway track from Shanghai to Kunming in China was buried by landslides, causing K859 passenger trains to derail, 19 people died and 17 people were injured.On July 29, 2009, in China, 1473 passenger trains derailed due to the collapse of the mountain, resulting in the death of 4 passengers and injured more than 50 passengers.In January 2008, the D59 train from Beijing to Qingdao was colliding with construction workers in Shandong and caused a major traffic accident, 19 people were killed and 9 people were injured.It can be seen that railway safety is very important and railway safety technology should be strengthened (Kardas-Cinal, 2014; Burdzik et al., 2017, Wieczorek et al., 2018).In recent years, with the rapid development of modern science and technology, plenty of scholars have studied a lots of object detection methods of track foreign object intrusion.The fiber-optic sensors have been used to monitor foreign objects intrusion in Japan's Shinkansen, and each monitoring point judges foreign objects intrusion and damage levels through the attenuation of optical signals.It has an early warning function (Cheng, 2010).A metal protective net with double cable sensors has been adopted to monitor foreign objects intrusion on highway bridges and tunnel entrances over railways in France (Cheng, 2010).The track foreign objects intrusion have been supervised through cameras and laser radar installed on trains in German, this method is simple to implement, low cost, but limited monitoring range, and limited by speed (Mockel et al., 2003).British scholar Arvind Hari Narayanan uses MIMO (Multiple input multiple output) radar installed on the side of the road to realize foreign objects intrusion monitoring on level crossing (Narayanan et al., 2011).Spain's Alvarez et al. (2004) used Ultrasound detector installed on both sides of the track to monitor foreign objects that had fallen into track.García et al. (2010) of the University of Alcalá in Spain has used infrared detectors installed on both sides of the track to form infrared barriers to monitor intruders and used data fusion techniques to calculate the size and orientation of foreign objects.Sehchan et al. (2007) in South Korea have used image processing methods to detect foreign objects at a station, this method mainly uses image difference to distinguish the foreground from the background, and it is simple and easy to implement, however, it is difficult to distinguish the foreground from the background of static foreign objects, and the detection reliability is low.In Hou&Li (2017) and Li (2016) using double background modeling and differential image to identify foreign objects intrusion in track, the method is simple and the accuracy is high.In Guo et al. (2012) Fast DBSCAN clustering method using the extremum of measurement sequence as the core object to classify, detect and identify intrusion objects on the track, and can better recognize foreign objects and trains.In Shi et al.(2015) a target classification and tracking algorithms based on SVM and Kalman filtering is used, Integrate the behavior of the target into foreign object discrimination, effectively improve the accuracy of foreign object invasion recognition, and have early warning function.In Wei (2014) using the detection method of stereo vision to detect foreign objects intrusion, a better detection effect has been achieved.At the same time, technologies such as deep learning, artificial intelligence, and driverless technology have also developed by leaps and bounds.Some experts and scholars have used convolutional neural networks on deep-learned to realize target detection (Girshick et  Due to the influence of light, fog and the changes of other weather, although many methods of them have been applied to track foreign object intrusion detection, they have advantages and disadvantage, especially, the detection precision, the adaptability to environment change and rapidity still need to be improved.For the study of the railway track foreign objects intrusion detection methods, aiming at the shortage of the existing detection methods, the detection method of foreign objects intrusion on railway tracks based on multi-background modeling and multi-difference was put forwarded in this paper.The decision is done by the black and white pixels quantitative proportion of image.In this method, matrix operations and convolutional neural networks are not used, it's processing speed is fast.Therefore, the method not only can improve effectively the accuracy and rapidity of foreign object detection, but also improve environmental adaptability, and provides a technical support for the safety of train running.

Multi-background modeling and updating 2.1. Multi-background modeling
In general, to judge whether there is a foreign object in the current frame image, the key is to find a reference object, through comparing the current frame image with the reference object image to judge whether there is a foreign object, and the reference object image is called background modeling.Multiframe average background modeling method is simple and efficient, but it is apt to be affected by noise (Yuan et al., 2013;Peng et al., 2013;Xie et al., 2013), adjacent frame background modeling method can adapt to the environment well, but, sometimes, the similar area of the foreground and background will be mistaken for back-ground.So, a multi-background modeling method that includes the historical background modeling, multi-frame averaging background modeling and the previous frame of the current frame background modeling method is put forward.This method takes into account the advantages of the historical background modeling, multi-frame average background modeling and continuous frame background modeling methods, and makes up for the deficiencies, it can provide good conditions for foreign object detection.Background image 1: Background image 2: The multi-background modeling structure diagram is shown in Fig. 1. x y to update background image 3.If there are foreign objects in current frame image, the alarm will ring, after the alarm and foreign objects are Processed, Using the new images to reset the all information in Fig. 1.

Railway track dangerous area segmentation 3.1. Edge extraction of the track
The main purpose of railway track foreign object intrusion detection is to identify the obstacles which affect the safety of train running, in order to improve the detection accuracy, it is necessary to divide a dangerous area based on the track, and merely detects the foreign object in the dangerous area.The most important thing of dangerous area division is to extract track edge, and judge the position of the track.Hough transform (Yang et al., 2013) is a feature extraction technique in digital image processing, it's basic idea is to use the duality property of point and line that the collinear points in the image space coordinates correspond to the intersection lines in parameter space coordinates, at the same time, in the parameter space coordinates, the straight lines that intersect a point are matched by collinear points in the image space coordinates, the task of edge extraction is completed by a simple cumulative statistic in the space.In view of the obtained track image which may contain buildings, trees and other more backgrounds except for track, in order to obtain a complete track edge image.Before detecting railway track foreign object intrusion, the edge detection of images is very important.The main purpose of edge detection is to extract the edge features of the target image and reduce the amount of data processed.The edge detection operators are Roberts operator, Sobel operator, Prewitt operator and Canny operator (Ruan, 2007).These operators are commonly used methods.The results of the edge detection operator for gray images are shown in Fig. 2.
It can be seen from Fig. 2, the results of the edge detection operators are different, there are some significant differences.The conditions of various edge detection operators are summed up as follows: (1) Roberts operator: Edge positioning accuracy of Roberts operator is high, but it is easy to lose part of the edge, and the image is not smoothed, so the ability of suppress noise is poor.(2) Prewitt operator and Sobel operator: They have a certain ability to suppress noise and has good edge positioning accuracy, but it is prone to multi-pixel wide edges and can not completely remove false edges.(3) Canny operator: Compared with other operators, the effect of Canny operator is best, and the edge detection accuracy is high (Li & Wu, 2014).From the comparison of the experimental results, the Canny operator is better, but it does not achieve the expected results for the railway track edge extraction.So, the Canny operator is improved in this paper.The improved Canny method (Li et al., 2015) is used to preprocess track image, and Hough transform is adopted to extract track edge.This algorithm realization process is as follows: (1) Images enhancement: the obtained images is filtered by extremum median filter method to enhance image details; (2) Gradient magnitude calculation: the image gradient is determined by introducing a weighted operator into the gradient magnitude which is detemined by the direction of 0°, 90°and 45º, 135º; (3) The image edge points are determined by suppressing the non-maxima of image; (4) The high and low threshold values are determined by the improved iterative threshold segmentation method that the image is divided into a high gray value region and a low gray value region by the mean value of maximum and minimum gray value of the image, and the optimal high threshold value is obtained by the iterative method in the high gray value area, similarly, the optimal low threshold value is obtained by the iterative method in the low gray value area; (5) Edge detection and connection; (6) Hough transform method is used to extract track edge and then to get the final track edge image.The basic idea of the Hough transform is the duality of the point to line (Wang & Li, 2006).The line in the rectangular coordinate system is a point in the polar coordinate system.In the rectangular coordinate system X-Y, the line equation y=kx+b, k is the slope of the line and b is the intercept of the line.Therefore, in practical applications, the linear equations are usually described in the form of polar coordinate equations.the line equation ρ=xcosθ+ysinθ, ρ is the normal distance from the origin of the coordinates to the line, θ is the angle between ρ and the X axis.As shown in Fig. 3.

Dangerous area segmentation
The segmentation of dangerous area which is used to insure the safety of train should try to avoid detecting the area where is not dangerous to the train running.The track gauge in common use is 1.435 meter in China, while the width of train is greater than track gauge, so in order to protect the safety of train running, the dangerous area is divided to be greater than the width of the train.

Foreign object detection and decision 4.1. Foreign object detection based on difference method
Average difference is a foreign object detection method by comparing the current frame with background modeling, the method can accurately detect, and easy to implement, but it is influenced greatly by the obtained background image; the inter-frame difference is a foreign object detection method by comparing two adjacent frames or multi-frame, the method is simple, high real-time capability, less sensitive to the changes of environment, such as light, and it has good stability, but this method can not completely detect foreign object (Zheng & Li, 2014).In view of the advantages and disadvantages of the two methods, in this paper, the three difference methods that historical difference, average difference and inter-frame difference are fused.According to the structure method of the multi-background modeling, if 1 ( , ) d x y is the historical difference image between the current frame and background image 1, 2 ( , ) d x y is the average difference image between the current frame and background image 2, and 3 ( , ) d x y is the inter-frame difference image between the current frame and background image 3, then calculating the average image 4 ( , ) d x y by the three difference images to compensate for their shortcomings.Formulas of the method are as follows:    ( , )  ( , ) 0, ( , ) Where, ( , ) d x y is the foreign objects detection re- sults.According to formulas 8, ( , )=1 d x y , it means there is a foreign object, otherwise it does not exist.

Bogus object processing
Because of being influenced by the changes of environment, such as light, the detection result may have edge burrs and isolated points, therefore, the detection result is processed by the morphological open processing and regional Unicom.The method realization process is as follows: (1) Swelling and corrosion.Swelling and erosion are two basic morphological operations.Many other morphological algorithms are compounded by these two operations.Swelling is the process of merging all the background points that an object touches into the object and expanding the boundary to the outside.The swelling can serve to fill small holes in the image and small sunken at the edges of the image.The image A was swelled by structural element B, denoted as AB  , defined { : } =   .In this paper, morphological opening operation is used to swell and corrode the detection results, remove the pseudo-boundary points of the target object, smooth the contour of the target object, and perform regional interconnect processing to effectively eliminate noise interference and obtain complete foreign objects contour.It can be seen that the bogus object processing can effectively eliminate noise, obtain a complete foreign object contour, and lay a foundation for foreign object detection.

Foreign object pixels calculation and decision
The purpose of foreign object detection is to identify obstacles which are dangerous to the train running, the track image is processed by the above method, foreign object in the image is white, non foreign object in the image is black, according to the ratio of the number of target pixels and the total number of image pixels(or the ratio of the number of white pixels and the total number of white pixels and black pixels) can effectively identify the dangerous foreign object.The calculation formula is as follows.
Where, a n is the number of white pixels, b n is the number of black pixels, R is the ratio of the number of white pixels and the total number of black pixels and white pixels.In the method of black and white pixels foreign object detection, the threshod , it can be judged that there is a dangerous foreign object, the alarm will ring; if S RR  , it can be judged that there is no dangerous foreign object, the alarm will not ring.Foreign object detection structure diagram is shown in Fig. 6.

Experimental results and discussion
The track images were processed through the above methods and algorithms in the Matlab7.1.Comparing the experimental results that the historical difference, average difference, inter-frame difference method, the three method fusion and the morphological open operation have been shown in Fig. 7. Fig. 7 is the detection results of track foreign object, comparing the results of Fig. 7(a), Fig. 7(b) and Fig. 7(c) with the result of Fig. 7(d), it can be seen that the detection results of historical difference (between background image 1 and the current frame image), the result of inter-frame difference (between background image 3 and the current frame image) and the result of average difference (between background image 2 and the current frame image) have some noises, the detected contour of foreign object is not good enough and been affected by the environment.At the same time, the result of historical difference and inter-frame difference is similar.The fusion detection result of three method difference is wonderful, the anti-noise ability is strong, it is obvious that the paper algorithm is greatly improved, the foreign object contour is more complete.Fig. 7(e) is the detect result that there is not the foreign object intrusion, Alarm bell does not ring.During the process, a large number of simulation experiments were designed and executed, and these results were compared, analyzed and improved.The comprehensive experimental results show that the method can achieve good effects, improve the detection accuracy and rapidity, improve the adaptability to environment change, restrain the noise interference effectively, and the real-time performance is better.This method provides a good technical support for studying of foreign object detection on track.

If
is the previous frame of the current frame background modeling image 3.The n, m and k is the sequence of frame, in general, 35 n ,the multi-background models are shown in formula 1, formula 2 and formula 3.

Fig. 2 .
Fig. 2. The extraction result of track edge of railway track under four operators: (a)The gray image, (b)The Roberts operator, (c)The Prewitt operator, (d)The Sobel operator, (e)The Canny operator

Fig. 3 .Fig. 4 .
Hough transform duality diagram: (a) the rectangular coordinate system, (b) parameter space The extraction result of railway track edge on the improved Canny and Hough transform is shown in Fig. 4. Fig. 4(a) is the extraction result of track edge on the improved Canny, Fig. 4(b) is the extraction result of track edge on Hough transform.Through comparing Fig.4(a) with Fig.2(e), it can be seen that the improved Canny has less disturbing edges and it is helpful to extract the edge of the track.The extraction result of track edge of railway track: (a) The extraction result of track edge on the improved Canny, (b) The extraction result of track edge on Hough transform

Fig. 5
Using the Hough transform to extract track edge, the linear equation of track edge can be obtained, adjusting the track equation based on track position, extending edge from left and right on track ensure that the train can be in the dangerous area to determine the dangerous area.The dangerous area division is shown in Fig.5.Fig.5(c) is dangerous area.The division results of track dangerous area: (a) The gray image, (b) The gray image of foreign object intrusion, (c) The result of division, (d) The result of division of foreign object intrusion

+ 4 d
is the current frame.Combined with the actual situation of the railway track and the train safety running, the smaller foreign object does not affect the safe operation of the train, and the pseudo target is also same.Therefore, a threshold s R is set through a large number of experiments, if the value of image 4 d is greater than threshod s R , it is regarded as 1, if the value of image is less than threshold s R , it is regarded as 0, the result is shown in formula 8.

2 )
Corrosion and swelling are dual operations.Corrosion is the process of eliminating boundary points and shrinking boundary points inwards.The image A was corroded by structural element B, denoted as AB  Opening and closing operations.Opening and closing operations are based on swelling and corrosion.The opening operation is a process of performing a swelling operation after the image is corroded.The image was opened operation by structural el-.The closing operation is a process of performing a corrosion operation after the image is swelled.The image A was closed operation by structural element B, denoted as AB, defined () A B A B B

sR
is designed by a large number of experiments, and calculates the ratio R in the detected current image or in the morphological open operation image, then compares R with

Fig. 7 (Fig. 6 .Fig. 7 .
Fig. 6.Track foreign object detection structure diagram6.ConclusionsBased on traditional detection method for foreign object intrusion on railway track, the new detection method that based on multi-background modeling and multi-background updating, track edge extract, dangerous area establishing on improved Canny and Hough transform method, foreign object detection based on multi-difference method, bogus object proand decision based on ratio of the black and white pixels are studied and designed.