Fuzzy-Based Method of Detecting the Enviroment Character for UAV Optical Stabilization

An optical stabilization of UAV (UAS) is a very important part of a structure in their control systems. Not only as a backup stabilization system in a case of IMU failure, but also as a main system, used for stabilization or navigation. In this paper the concept of a system for environment character detection is presented. The system can classify a surrounding environment depending on chosen characteristics. Such system can be used for a better horizon detection due to switching to a correct horizon detection algorithm, which can be used for determining the position of UAV. The system is based on Takagi Sugeno fuzzy inference system and fuzzy artificial neural networks. An earlier work on this subject was presented last year, but concept of the system was redesigned with a usage of fuzzy artificial neural network for a more precisive outputs and automatic determination of characteristics of fuzzy sets on input.


1.
IMU and their Weaknesses

Electromagnetic Noise and GPS Denied Environment
An inertial measurement unit (IMU) is an electronic device that measures and reports a craft velocity, orientation, and gravitational forces, using a combination of accelerometers and gyroscopes, sometimes also magnetometers.Currently, the use of magnetometers in IMUs, is more common.Magnetometers using changes in the Earth's magnetic field for momevement determination.Magnetic fields are vector quantities characterized by both strength and direction.Using those values we can evaluate speed and acceleration in 3D space same way, like using accelerometers.In a case of an electromagnetic inferences, magnetometer provides incorrect data so evaluations of position are incorrect.Such IMU failure may cause an incorrect reaction of UAV control system and as result crash or damage of a vehicle.
Electromagnetic inferences lead to an uncontrolled movements of UAV in a case of algorithm failure (for example position hold).In such case, GPS signal cannot be used for correction due to low accuracy (usually in meters).Drift of UAV, for example small helicopter, can lead to PID controller oscillations which can cause an uncontrolled movements in area of five meters and more radius.
The most common sources of electromagnetic inference are: • UAV electric parts -for example electric motors, transmitter.
• Magnetic anomalies -such as a storm, electric wiring, electric trains.
For the reasons mentioned above, the need for a backup system is obvious, especially for UAV.Unlike normal airplanes and helicopters, which can be controlled by a human pilot in a case of sensor failure, the autonomous UAV do not have such an option.

Optical Stabilization of the UAV
The most common alternative of IMU is an optical stabilization.Usually optical flow, corner and feature detection algorithms are used.Those algorithms have limitations especially due to the low resolution of onboard cameras.Even high resolution cameras cannot provide enough usable data in high altitudes.In such altitudes, small changes and image shifting, which are needed for optical flow determination, cannot be detected.
High altitudes of UAV and bad atmospheric conditions are reasons for a feature detection algorithm failures.In such conditions, image features usually cannot be repeatedly detected.For applicable UAV optical stabilization we need some fixed points (clusters) or detected information in captured image.Such point or information should be present in most captured images so this information can be used for determinations of movements.
The question is, how many points or information is needed for UAV stabilization in 6 degrees of freedom (DOF).In ideal condition we have one detected feature for every degree of freedom.For backup system of IMU it is possible to determine a compromise and define minimum number of such features which we can use for short time emergency stabilization.

Horizon Based UAV Stabilization
As a horizon we can usually understand a dividing line between sky and ground.Usually we cannot be sure that sky and ground are always visible, so lets use the definition of horizon as the line, parallel with water surface.Position and angle of horizon can be determined by human eye in most of scanned images.There are many detection algorithms for various conditions.Problem occurs with only one onboard UAV camera.Usage of only one camera cannot ensure that it will always capture the scene in which the horizon is located, or the position of the horizon that can be determined.Such problem can be partially solved by usage of more than one onboard camera or with usage of panoramic onboard camera.In a case of bad atmospheric conditions, which reduces visibility, we can use thermal vision, UV or HR camera.
Detected horizon provides information, which can be used for emergency UAV stabilization.From horizon position and angle we can calculate information about changes of pitch and roll of UAV.Information about altitude, rotation and drifting on horizontal plane cannot be determined.From those reasons we need to add an auxiliary image detection algorithms such as optical flow and feature detection.
The problem with low image resolution in high altitudes cannot be effectively solved.But proposed stabilization system will work as backup emergency system, so this problem may not be significant.In a case of emergency situation we can presume, that scanned image resolution will be greater when UAV is close to the ground or obstacle, so stabilization system will have more precise output.Such feature can avoid collisions in emergency situations.
From information above we can understand that for efficient optical stabilization, based on the horizon detection, we need to provide certainty of horizon detection in every situation and environment in which UAV will operate.Such feature can provide proposed method of automatic environment detection.

Usual Problems with the Horizon Detection Algorithms
Currently, there are some general advances in horizon detection.Each of them candetect the horizon under certain conditions from the detection of horizon using Hough transformation to more complex methods.
There is an overview of some of the techniques and problems depending on the conditions in Tab. 1. Intentionally we do not mention the detection horizon in urban areas here, because in such conditions all listed procedures fail virtually.
There are a number of horizons detection solutions with different percentage success of the detection.For small UAVs, which reach maximum heights of hundreds of meters, the horizon detection is not a trivial task though.At this point it is necessary to define the difference between the horizon and the visible horizon.We consider as the horizon the dividing line between sky and earth, that is, a horizontal line that is level to the ground.We understand the visible horizon as the real dividing line between sky and earth, which is visible from a given location in the scene.Visible horizon may therefore not always be horizontal with the ground plane.Another problem arises naturally in urban areas, where the horizon must be detected for example from vanishing points of detected building outlines.
The ideal objective is to find a universal approach how to detect the actual position of the horizon and any environment.Such procedure would not fail in any area, which the UAV can get in.
To detect horizon in urban areas there are several methods that are based on the monitoring of main lines in the image and the detection of the vanishing points, from which the likely position of horizon and its angle can be calculated.
Tab. 1: Some problems of standard horizon detectors depending on conditions.

The method
The best results The possible weaknesses Hough transformation, edge detectors.Great heights, straight horizon.Fails at low altitudes and indented areas Searching of the centers of gravity of the ground and sky.

Equally contoured horizon
In the indented surroundings and a higher cloudiness Optical flow.Great heights, a little contoured horizon.
May fail at low altitudes and indented surroundings.
Algorithms that work with edges however fail in open landscape, because there is no correct detection of horizontal lines and the subsequent derivation of the vanishing points because they are virtually absent in the scene.

The Detection of the Environment
For the problems described in the previous chapters a simple solution can be used, based on the idea of detecting the environment, in which the UAV device is moving, and the subsequent correct choice of the algorithm for detecting the horizon.At this point, it is irrelevant what kind of algorithms for horizon recognition we will use; the important thing is to choose at least one for landscape without build-up area, one for urban areas and one for indoors.The following pseudocode represents the basic algorithm for environment detection.
Algorithm 1 end if 11: end for However, the choice of correct algorithm which correctly calculates the decision variables based on the detected parameters, which will lead to the correct environment identification is still a problem.For correct decision is better to use some soft-computing method.
We decide to use fuzzy -based approach.Basic problem is, of course, selecting the right input parameters.

The Choice of Detection Parameters
The information listed in the text above implies that one of the key parameters for the decision-making algorithm will be straight lines detection and their characteristics.Quantity of detected straight lines with specific lengths is quite different in open scenery and urban development.However, this may not always be decisive.The straight lines, which can provide false information, such as location of the vanishing point (power lines, fences, roads), can be detected even in open landscape.Therefore, it is necessary to provide additional input parameters.We must include only the lines that have a certain length to the calculations, though.It is necessary to exclude short lines, which may occur in both types of environments.We can select minimum length of straight lines using for example the size of the processed (see equation bellow), c long is a constant, which defines the minimum size of the detected lines.
It is a good idea to use the direction of detected lines to calculate the next parameter.It is specifically the number of lines that have the same direction as this attribute corresponds to a high degree of urban environment.It is appropriate to use the number of detected rectangles is the last parameter.It is essential to use an algorithm that is able to detect rectangles with perspective [1].Representation of rectangles in the images from urban environment is usually high.Because we detect the straight lines and their directions, there is no problem to find the lines that form the edges of detected rectangles.For detection of environment it is necessary to count with the fact that quadrangles are also detected outside urban areas, but their number will be low.For determination of a difference between indoors and outdoors environment we can use position of detected vanishing points.Lets separate input set of vanishing points to two groups, inside image and outside.Therefore we have four input parameters: • number of parallel lines clusters, • number of detected rectangles, • number of vanishing points inside of image.
Output parameters for decision making are three: • variable of urban environment, • variable of non urban environment, • indoor variable.
The final output from decision process is classified environment (indoor, outdoor, city).

The Proposed Method
For solving this problem we can choose from many possible soft-computing approaches.First experiments were made with usage of fuzzy Takagi -Sugeno inference system.During implementation of such system some important problems occurred.The biggest problem was with fuzzification of input and choosing a right fuzzy sets for linguistic variables.fuzzy sets had to be specified manually, and fine tuning also had to be made by the programmer.Also rules behaviour was not easy to predict.From those reasons we decided to use an artificial neural network with supervised learning.Our system should be able to learn from given examples of environments.As the best approach seems to be usage of fuzzy classification neural network (NFC), which approximates behaviour of Takagi -Sugeno inference system [5].
Scheme of proposed method can be seen in Fig. 1.Images from onboard UAV camera is processed by image recognizing procedures and numbers of lines, clusters, rectangles and vanishing points are determined.Those values are used as an input to fuzzy classification neural network.Based on NFC output, the correct horizon recognition algorithm is chosen.Detected horizon is used for further optical stabilization of UAV.Output is z = ax + by + ci 3: end if For a zero-order Sugeno model, the output level z is a constant (a = b = 0).The output level z i of each rule is weighted by the firing strength w i of the rule.For example, for an AND rule with Input 1 = x and Input 2 = y, the firing strength is: where F 1() and F 2() are the membership functions for Input 1 and Input 2. The final output of the system can be the weighted average of all rule outputs.
More about the Sugeno fuzzy inference systems can be found in [3] or [4].Let us denote the input and output parameters as: • SEG number of line segments, • CL number of parallel lines clusters, • RC number of detected rectangles, • VPI number of vanishing points inside image.Initial input fuzzy membership function were set as we can see on Fig. 2, we have chosen Gaussian function.Output of inference system should be three variables, one for the urban environment, one for non-urban environment and one for indoors.
Results from an early experiments with the fuzzy IS can be seen on Fig. 3 and Fig. 4. We used the input parameters without vanishing points detection.URB and NURB values are the fuzzy IS outputs from range <0,1> and represents detected environment membership.As we can see, there were problems with detection overlap, so the results were not usable in real conditions.Many configurations were tested, but for real world use we need more robust and reliable solution.From those reasons we choose to aproximate the fuzzy IS with a fuzzy artificial neural network.
Next step was to approximate fuzzy IS with usage of fuzzy neural network for classification (FNNC).

The Fuzzy Neural Network for Environment Classification
Scheme of used FNNC can be seen in Fig. 5.As mentioned above, for i th rule, s th sample and j th feature we are using Gaussian membership function with center c, which is defined as follows [5]: For fuzzyfication we are using following rule, where N is number of features: Defuzzyfication is done with usage of following rule: Weight w representing degree of belonging to k th class.Finally, determining of output class C for s th input is done simply by choosing the maximum: For supervised training of the fuzzy classification neural network we can use variety of learning algorithms.During the neural network training a chosen algorithm should update the values of membership function centers and widths, also all weights on connection from fuzzyfication to defuzzyfication layer.Usually variants of the SGG algorithm is recommended [5] for training however in this case we have problem with variable number of inputs.Also the learning algorithm must be able to work online.The main principle of supervised learning is to manual switching of correct environment, which causes to generate designated output.UAV now can collect data and learn from given examples.At present, the online learning algorithm for our purpose is the subject of research.For testing functionality of proposed system we are using temporary approach, which consider collecting data and then make offline learning.Those values are used as initial weights of NFC.Next step is to use modified online learning algorithm, based on error gradient descent principles.

Conclusion and Future Work
The ability to detect the character of the environment in which autonomous entity operates should be an important feature for future autonomous systems, especially for UAVs.Usage of the fuzzy classification neural network for such detection is probably an appropriate approach as it corresponds to the character of the input parameters that have a high degree of uncertainty.Also, this approach seems to be appropriate for the possibility of using the amount of offline and online learning algorithms for fuzzy neural network.
Our recent research indicates that the method presented in this article is applicable for practical use in UAVs stabilization systems.In the near future we expect the publication of comparative results using different types of online of learning algorithms and neural network configurations.
c 2015 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING • number of line segments,

First 2 1:
step was to create an Takagi -Sugeno based fuzzy inference system.A typical rule in a Sugeno fuzzy model has the form [3]: Algorithm if Input 1 = x and Input 2 = y, then 2:

Fig. 1 :
Fig. 1: Scheme of proposed system for environment detection and stabilization.

Fig. 5 :
Fig. 5: Scheme of fuzzy neural network for environment classification.