Research on FOD Detection System of Airport Runway Based on Artificial Intelligence

With the rapid development of the global aviation industry, the number and scale of construction of civil airports around the world are increasing. In this context, the difficulty of aviation safety assurance work has also increased. Foreign object debris on the airport runway is one of the important factors affecting aviation safety and security. Any foreign material, debris or small objects that appear on the airport runway may pose a serious threat to the ground operation safety of the aircraft. Therefore, research on FOD detection is very important.


INTRODUCTION
According to a special survey conducted by the Civil Aviation Administration of China, more than 4,500 cases of aircraft tire puncture caused by foreign bodies on the airport runway have occurred in the country from May 2007 to April 2008, and the common FOD of the airport runway is shown in Fig. 1 [1,2].  The global aviation operation department has paid sufficient attention to the detection of airport runway FOD to ensure the safety of the aircraft. For example, major airports in China have established special FOD governance committees to carry out FOD prevention propaganda and increase FOD prevention efforts.
The traditional airport FOD detection technology usually adopts manual screening method as shown in Fig. 2. The automatic detection of FOD not only has low efficiency, high rate of missing detection and high cost, but also can't realize full real-time detection, which has great hidden danger, so it has attracted more and more attention. Foreign bodies on the airport runway can not only damage the aircraft and endanger precious lives, but also cause serious operational problems for airlines and airports and bring huge economic losses.
In 1996, the American Air Transport Association counted the FOD incidents of 23 member airlines within three years. The data shows that the average annual loss of FOD caused by each airline is 7.4 million US dollars, and the average annual loss of all member airlines is 170 million US dollars.
If coupled with the indirect losses caused by flight delays, interruption of takeoff and runway closure, these losses will be astronomical.
In the year from May 2007 to May 2008, there were as many as 4,500 incidents of aircraft tire damage caused by FOD in China. About 13% of the tires exchanged by most airlines are punctured by FOD. It is a big expense to scrap and repair tires early.
In 2008, the Civil Aviation Administration organized relevant units to carry out special investigations on the prevention of foreign objects, and launched a series of special activities in the industry to prevent and manage foreign objects.

RESEARCH CONTENTS
In recent years, some companies have developed FOD automatic detection systems [3] through radar detection and optical video detection technologies, such as QinetiQ's Tarsier system [4] (as shown in Fig.  3), Xsight's FODetect [5] (as shown in Fig. 4), and Stratech's iFerret [6]. Although radar detection has the characteristics of long detection distance and high resolution, it cannot provide the color characteristics of the detected objects and intuitive video images, which brings inconvenience to the subsequent removal of FOD.
The FOD detection system based on optical video is difficult to effectively describe and extract the characteristics of FOD due to the influence of traditional image processing target detection technology, and there is still much room for improvement in detection performance.   Target detection is one of the important tasks in computer vision technology. Typical target detection algorithms are often designed for larger targets such as pedestrians, vehicles, and faces.
However, FOD targets often have various damages and no fixed shape, so the features that can be used for detection will be significantly different from traditional targets.
Moreover, the detection process tends to have a higher degree of attention to the target materials and hazard levels. The hazard levels of targets with different materials will be significantly different, and for targets with different levels of hazards, the subsequent treatment process will also be significantly different.
For example, low-risk items can be removed within a time that does not affect the aircraft's landing and landing arrangements.

RESEARCH STATUS AT HOME AND ABROAD
The current mainstream FOD detection technology mainly includes radar-based technology, optical image technology and multi-sensor hybrid detection technology. The radar-based method uses milli-meter-wave radar and uses radio transmission and reception data for FOD detection. This type of method has the characteristics of high resolution and high detection accuracy. It can accurately locate the target and is not affected by weather conditions. However, it cannot reflect the visual color characteristics of the target object, which brings inconvenience to the subsequent removal of FOD.
Based on the optical image method, the intelligent detection and processing of the video image collected by the optical camera realizes the detection of the FOD target and determines the FOD. At the same time, it can provide an intuitive video image display, which is convenient for subsequent confirmation and removal.
However, the optical imaging system is susceptible to weather and light and other factors, and it is easy to miss detection when the color of the target is close to the road surface. For the characteristics and deficiencies of radar detection and optical image detection technology, the hybrid detection technology adopts radar, Multi-sensor technologies such as visible light and infrared realize complementary advantages.
For example, Israel 's FODetect system uses a combination of millimeter-wave radar and visible light video, while the United States 'FOD Finder [4] uses a combination of radar and infrared cameras, and has achieved good detection performance.
With the development of artificial intelligence technology, Argos AI introduced it into FOD detection applications and launched the intelligent FOD detection system A-FOD [7]. By capturing high-resolution FOD images, artificial intelligence technology is used to achieve efficient Detection.
Compared with the mature commercial FOD automatic detection system that has been deployed abroad on a large scale, most of the FOD detection in domestic airports adopts the traditional manual screen screening method. This method is inefficient and cannot guarantee accuracy. Security risks.
Due to the complexity of the FOD automatic detection system, no domestic manufacturer has passed the FAA certification. In response to this situation, many domestic research institutions and related manufacturers are actively developing this system.
According to Hong Kong's "Ta Kung Pao" news, the Hong Kong AA and the ground equipment company jointly designed the FOD automatic detection system, which uses artificial intelligence-related technologies to be deployed at Hong Kong airports.
For FOD detection, the detection time is shortened from 2 hours each time for manual screening to 2 minutes each time, which greatly speeds up the efficiency of FOD detection. At the same time, even if it is rainy or windy, it will not affect the system operation.
It can be seen that artificial intelligence technology has great potential in the field of FOD detection. An important technology for FOD detection is based on computer vision algorithms, through image sensors, real-time monitoring of airport runway images and FOD target detection [8].
FOD detection is mainly to find the FOD target as the task, and at the same time give the semantic information of the target's hazard level and material type.
The hazard level information is used to help the system to divide the processing response level, according to different hazard levels, different levels of processing High-risk items need to be dealt with in time, and low-risk targets can be dealt with at an appropriate time.
Semantic information of material types can help to reasonably handle FOD targets. Different material targets are processed in different ways to make the system truly intelligent and achieve unmanned operation as much as possible with the cooperation of some automated equipment [9].

RUNWAY FOREIGN OBJECT DETECTION AND RECOGNITION SYSTEM BASED ON ARTIFICIAL INTELLIGENCE
A complete runway foreign object detection and recognition system based on image analysis is composed of three parts, namely airport runway image acquisition module, airport runway foreign object detection module and airport runway foreign object recognition module. Although several parts are independent of each other, they are closely connected . For example, the selection of cameras, different installation schemes, and the installation of other equipment such as lights will affect the research of the corresponding processing algorithm, and the difficulties encountered in the algorithm research will also require the improvement of hardware equipment.
The foreign body detection and identification system of the airport runway is divided into two major modules of detection and identification, in which the detection module is the basis of the entire system. The detection module uses sensors as the main tools.
The physical parameters of the sensors determine their installation position and angle on the patrol car. The traveling speed of the patrol car also puts forward higher requirements for real-time detection of the system.
The work flow of the runway foreign object detection and identification system can be implemented in two ways: real-time data collection and real-time foreign object detection processing, real-time data collection and offline foreign object detection.
Combined with the current status of foreign object detection on the runway in China, the current domestic hardware conditions and our actual situation, this system adopts the method of simultaneous data collection and detection for foreign objects above 5cm × 5cm, and data collection for foreign objects below 5cm × 5cm Separate from testing.
The overall design of the airport runway foreign object detection and recognition system in this paper is: During the routine inspection of airport patrol cars, the image of the road surface in front of the vehicle is collected by the optical camera installed on the vehicle, and the foreign object information is extracted using image processing technology The optical information of the area of interest is returned to the onboard computing data center for comprehensive processing, real-time foreign object detection and determination of the type of foreign object, and feedback to the staff.
The optical camera technology is mature and the price is low. It is often used in various video surveillance occasions, and the detection capability of the high-resolution optical sensor can meet the detection requirements of small foreign objects on the runway, and the feature information extracted by the optical camera is also used to classify the foreign objects Recognition laid the foundation.
The collection of pavement images requires that the images can cover as much of the runway width as possible. The travel speed of the patrol car can reach 60km / h or more.

Multiple cameras
Generally, multiple cameras are installed side by side at the front of the vehicle, each camera can cover a certain range, and a wider field of view can be obtained by multiple cameras. The camera can be mounted on a high roof or mounted on the bumper at the front of the vehicle.

Single camera
In order to obtain a wider field of view, this solution needs to be erected at a higher position and use a high-resolution wide-angle camera.
Based on the above discussion, this system chooses the scheme of installing multiple cameras on the roof of the car.
The system needs to be able to intelligently distinguish and classify the detected foreign objects, so in-vehicle data centers should process the foreign object information returned by the previous stage through real-time and effective algorithms to identify the detected foreign objects.
FOD targets are used to alert airport staff according to different types of foreign objects. The brief working principle of the airport runway foreign object detection and identification system is described as follows: (1) The system starts a self-check to confirm that each sensor is working properly.
(2) The optical sensor works to monitor the road surface in front of the body during the road patrol.
(3) If a foreign object is found, the foreign object ROI area of the optical image is fed back to the FOD data processing computer for comprehensive processing to determine the foreign object range.  (4) The pattern features of the foreign body are extracted, compared and identified by the FOD data processing computer, and the danger level of the foreign body is judged and fed back to the staff.
(5) The staff cleans up the foreign objects in time according to the danger level, and records the foreign body information, which is saved to the system server.
The actual object of the vehicle runway foreign object detection and identification system developed as shown in Fig. 5. The airport runway foreign object detection and recognition system is applied to the vehicle-mounted system. It can turn on the camera in real time, process the runway pavement scene situation captured by the camera in real time, detect the runway foreign object therein, and then identify the type of runway foreign object and its hazard level, and carry out Real-time alarm.

CONCLUSION
Artificial intelligence technology has great potential in the field of FOD detection. An important technology for FOD detection is based on computer vision algorithms, using image sensors to monitor airport runway images in real time and perform FOD detection.
The main task of FOD detection is to find the FOD target, and to provide semantic information of the risk degree and substance type of the target.
The hazard level information is used to help the system to divide the processing response level, according to different hazard levels, different levels of processing High-risk items need to be dealt with in time, and low-risk targets can be dealt with at an appropriate time.
Material type semantic information can help to reasonably handle FOD targets. Different material targets are processed in different ways to make the system truly intelligent and achieve unmanned operation as much as possible with the cooperation of some automated equipment.