Protall (An Intelligent, multi-sensor, comprehensive obstacle avoidance system for automobiles and UAVs)

In this era of Artificial Intelligence and Automation, manufacturing and testing of self-driving cars and autonomous delivery of parcels to the desired location with the help of UAVs have a considerable amount of growth in the industry. This advancement in technology also raises safety issues due to the failure of sensors to detect the object or sometimes because of the dynamic environment. Protall (Protect-all) is an integrated solution for UAV and automobile vehicles to provide ultimate safety to both itself and its surroundings. With the strategic sensor integration and its intelligent processing, it aims to produce controlled output and hence, ensures to prevent any possible failures from occurring. The system constantly reacts to the environment and maintains comprehensive interaction with the user thus, enabling it to handle any dynamic situation and hence, it emerges as an optimal solution for Vehicles and UAVs.


Introduction
India consists of 1 percent of the world's vehicle population and accounts for around 11 percent of the global death in road accidents, as per the report by the World Bank. Almost 4.5 lakh road crashes per annum and 1.5 lakh casualties [1]. This is a significant figure, and some measures are necessary to be taken to decrease the count of road crashes that will ultimately reduce the death ratio. Also, many UAV collision cases are caused by the intervention of birds, extremely low hardware reliability and low software stability. Protall proves to be an innovative idea to tackle all deadly crashes inspired by biomimicry from a chameleon. Chameleon's eye can provide them almost full 360 degrees of vision. Moreover, they can process different images at the same time. It helps them to hide by changing their color and saving lives from predators. Using this powerful technique in the technological world, accidents can be detected before happening, and safety measures can be taken. The Protall system is used in the UAVs and automobiles to detect various parameters such as distance and speed of the surrounding obstacles. By using multiple sensors, data parameters can be processed, and at the same time, safe solutions are implemented to avoid collisions. Protall is further categorized into UAVs and Ground vehicles named as Roadall and Dronall. Roadall system is designed in a way that it assures maximum protection for automobiles from all sorts of collision cases. It not only alerts the user from pre-crash warnings but also activates the braking system through actuators and slows down the vehicle. Roadall accommodates Time of light sensor to detect the distance of surrounding objects, Lidar sensor radar around the car and Camera sensors for users to view and analyze the surroundings. Speed differential sensor is used to estimate the speed of the front and side vehicles, night vision camera for more safety in fog and other conditions. An active braking system IOP Publishing doi: 10.1088/1742-6596/2161/1/012056 2 helps to slow down the whole car. Roadall also features the hill hold system, which allows the driver to start and move the vehicle safely without any crash on the inclined slope or hill.
Dronall is an intelligent system made for UAVs to detect stable and dynamic objects in their surroundings and quickly respond to the changes by tackling them. It receives the surrounding data through the Time-of-flight Sensors and Cameras. After processing the data, it alerts the user for necessary control changes. If no actions or commands are passed, it automatically moves away from the obstacle and maintains its position and altitude. In case of any damage or battery failure, the UAV will switch to failsafe mode and land on a flat surface with the help of Time-of-flight sensors on the base mount. The operator will receive the message of technical difficulties in the UAV and can pick up the UAV from the landing site.
Protall is a complete solution for Car owners and UAV manufacturers as it can provide total safety from accidents and collisions. The user can take decisions and manually make changes if necessary while driving the car and controlling the UAV. Protall will therefore have a bright future in the 21st century and safeguard humans with the power of technology.

Literature Review
This section briefly describes various UAV's and Automotive collision detection and avoidance system that was proposed and implemented in the past few decades. Most of the collision avoidance system accommodates conventional obstacle recognition technologies. Most of the existing methods only alert the user when an obstacle is detected very close. Then everything depends on the user's response and if the user does not intervene in the warning, then there is a possibility of a collision. So, this method does not ensure total safety [2]. The proposed method not only alerts the user but also stops the vehicle whenever there is a possibility of collision. This system is designed to be robust and flexible as to work in any conditions and provide safety. The method as proposed in [3] illustrates recent advancements in Autonomous Vehicles regarding two critical concerns that play the most vital role in successfully implementing and working an Autonomous Car like Obstacle Detection and Track Detection. The work in [4] demonstrates the performance of LIDAR, RADAR, vision cameras, ultrasonic sensors and IR. It provides feedback on their capabilities in harsh weather conditions, rough and uneven terrain, detection of potholes and bumpers during daylight visibility and night darkness. Considering the sensor analysis in the paper, suitable measures are implemented in Protall to work in a dynamic environment while also actively maintain safety through an active alert and braking system. A lot of research and development of UAV's obstacle avoidance using some standard sensors, such as radar, LIDAR, ultrasonic, and infrared rangefinders, have been widely used on UAVs to detect the existence and position of UAVs obstacles [5]. The researchers in [6] makes the usage of event cameras, bioinspired sensors that distinguishes between static and dynamic objects, which is an efficient strategy to generate motor commands to avoid obstacles. Based on biomimicry, Protall comprises of Lidar range sensor, which has a 360-degree vision similar to a chameleon and can process multiple images at the same time. The existing UAV's obstacle avoidance system can detect static objects and maneuvers as per the collected data, but there is a high chance of collision in dynamic conditions. Dronall functions so that it not only detects the object's presence but also responds by changing its path to prevent a crash. Dronall also features the emergency landing in case of system failure and takeoff/landing assistance for smooth flight performance.

3
Proposed Methodology

System Architecture
Protall is an intelligent, multi-sensor obstacle-avoiding system. Its infrastructure can be broadly classified into 3 stages: data sensing, obstacle detection, and Obstacle avoidance. These stages would be followed by both the sub-systems: Dronall, which aims to provide safety to the Unmanned Aerial vehicles (UAVs) and Roadall, which aims to provide protection to the users and all Surface vehicles (SVs). ' Figure 1' shows the process flow of the three-layered stages. Following steps describe the 3stage process flow [7]: for processing, post which the system can detect the object, its location, and its type. This layer acts as an identification of an Obstacle.  Obstacle Avoidance: Depending upon the response of above layer, appropriate action is taken to avoid the obstacle with the help of MCU and actuators attached to it. This layer deals with the action taken by the system to avoid or prevent the severity of a collision. Functionality of each step is further sub-divided to optimize the system and give the best response based on appropriate conditions. Both Dronall and Roadall follow the above-layered stages in its own unique way to tackle collisions as to make the respective systems more robust and safer to use than the already existing methods.

Structure and Schematic
Protall system undergoes the process of Sensing, Obstacle detection, Calculation of maneuver for collision avoidance and performing maneuver for successful avoidance of obstacles for UAVs and automobiles.

Dronall (UAVs):
 Multidimensional LIDAR Time of flight sensor is attached at the top of UAVs which creates a radar around the UAVs as to provide 360° positional monitoring. This sensor surrounds the whole top part of UAVs as to acquire data from different axes at high speeds and also to maintain the Lateral position of UAVs. It also helps detect both dynamic and static objects when entered in the field of view [8][9].  Time of Flight (TOF) sensor, an efficient distance sensor is attached at the bottom of the UAVs which helps them to maintain a constant height above the ground when user doesn't intervene or UAVs tends to change their direction, which adds the autopilot capability into them so that they do not fall and get damaged.  In Addition to the above point, 2 TOF sensors is attached at the bottom of UAVs, both at the rear ends rather than one which provides safe landing capability into them. While landing if both TOF sensors detects the same distance of the ground which will ensure a safe landing surface for the UAVs to land because sometimes surfaces are not even and drone tends to fall and gets damaged. Usage of Multidirectional LIDAR and TOF sensor will enable the UAVs to adjust its lateral position without ascending or descending due to pitch and roll.  Camera sensors are used for the user to view, analyze and identify obstacles around the path so as to reach their destination without colliding to an obstacle. IR sensors is also being mounted on UAVs which provides an extra safety feature that helps the UAVs to detect objects in lowlight, foggy and rainy conditions.  System detection sensors are used to detect internal faults in the system or some critical conditions that may arrive while using UAVs. This sensor when triggered the UAVs has to land immediately and it makes all distance sensors inactive so as to help the user land immediately with the help of camera sensors without any conditions to satisfy for obstacle avoidance.  When an obstacle enters into the field of view an alert will be sent to the user with the help of buzzer so that user can take appropriate measures by looking in the camera. If user does not intervene to respond in spite of alert the Dronall system will help the UAVs to avoid the obstacles [10].  Figure 2' represents different conditions and how the Dronall collision system tackles each condition to avoid collision from obstacles. In first situation there are no obstacles detected by LIDAR senor, so the UAVs will keep on moving according to user instructions and will keep on checking obstacles in their vicinity.
In second condition while moving, when an obstacle is detected within the field of view, the LIDAR sensor will sense the angle of obstacle and will provide it to the internal accelerometer that will decelerate the UAV. If user gives the input as the same angle at which the obstacle was detected, the UAV stops immediately and will not receive any input from transmitter until the obstacle remains in the field of view. If user provides some different angles where there are no obstacles detected, the decelerated UAV will move gradually until obstacle remains in the field of view [11].
In third condition, when a dynamic obstacle is detected with the help of speed and LIDAR sensor, the UAVs will not receive any input from transmitter and will detect some angle with the help of LIDAR sensor where there are no obstacles and will start moving in that direction. While moving if in case the UAVs are not able to receive instructions from users end, with the help of TOF sensor which is attached at the bottom end of UAV, the vertical thrust is adjusted autonomously to maintain desired height.  Figure 3' represents the pictorial view of Roadall system and how Roadall system is implemented using sensors and active brake system in an automobile.  Multidirectional LIDAR Time of flight sensor is used in this system which creates a radar around the vehicle as to provide 360° positional monitoring. It helps in detecting the obstacle when they enter into the vicinity of the vehicle.  Time of Flight (TOF) sensor which is an efficient distance sensor is attached at the front of the vehicle facing downwards at an optimum angle. It helps in identifying bumps and potholes of the road. The angle at which sensor should be connected to the vehicle would depend on the height of the vehicle as to detect bumps and potholes in advance and alert the user.  Speed differential sensor is attached at the front of vehicle to detect the speed of the obstacle in front if any. Based on the sensed value it detects whether it is a static or dynamic object and is used with camera sensor as to detect the obstacles with their respective speeds.  Camera module is also used at the rear end of the vehicle for the user to view, analyze and respond while going in reverse direction. Reverse gear sensor detects when vehicle in reverse mode and displays the output of camera sensor in the monitor.  When an obstacle enters into the field of view, Buzzer beeps which alerts the user to respond by applying brakes. Brake sensor detects whether the user has responded to the beep and had applied brakes. If not, Active Braking System will become active and will stop the vehicle.  Active Braking System (ABS) is used to apply brakes automatically to avoid collisions in critical conditions, as to when an obstacle is detected and user does not respond to that. ABS in combination with angle sensor also acts as the Hill-Hold system as to when the vehicle is at halt in incline road the brakes must be applied so that vehicle does not move in the backward direction. ' Figure 4' represents the usage of the Roadall system to prevent the collision from obstacles. Obstacle sensing with the help of the attached sensors acts as a crucial part of the system. If no obstacle is detected, the vehicle keeps on moving according to user instructions and keeps on checking obstacles in their vicinity.

Figure 4. Working Of Roadall
Speed differential sensor detects the speed of the obstacle in front and sends the information to ABS (Active Braking System) via Microcontroller board to respond accordingly based on the type of object. If a static obstacle is detected at the front and the user does not intervene despite the warning, the ABS tries to avoid the collision by applying the brakes. If a dynamic obstacle is detected at the front, depending on the speed of that obstacle, the vehicle will move gradually until it remains in the field of view [12].
If obstacles are detected on left or right side of the vehicle, the system first decelerates the vehicle and alerts the user regarding the same. Even after the alert, if the user tries to take a turn on the same side which will be detected by the steering wheel sensor, the steering wheel will be locked and will not allow the user to take a turn until that obstacle remains in the field of view. The Bump and Potholes are detected by the Time-of-flight sensor to alert the user in advance to reduce the vehicle's speed so that the user experiences a great ride even on bumps and cracks. When the vehicle is at a halt in an incline road, the Hill-Hold system becomes active and applies brakes independent of the user. The amount of brake that must be applied so as to stop the vehicle from moving behind depends on the weight of the vehicle, which is sensed by the pressure sensor. The Hill-Hold system deactivates only when the required amount of torque is applied by the user, which is perceived by the torque sensor. This feature prevents the collision of the vehicle on incline roads.

Practical Development
 ' Figure 5' shows the rudimentary data flow diagram of Protall which is subsequently followed by Droanll and Roadall sub-systems with some modifications and advancements as discussed in the above sections.  The data flow defines the set of instructions which would be followed by UAVs and automobiles as to detect objects and take appropriate decisions to avoid obstacles while moving [13].  Continuous proximity sensing is achieved with the smart usage of intelligent sensors strategically placed at optimal positions of the system.  If any obstacle is detected within its vicinity, the system decelerates the vehicle and alerts the user with the help of an alarm. The vehicle will correspondingly move at slow rate until the obstacle is within the field of view. The speed differential sensor helps detect the type of obstacle and its motion. In case the user fails to respond to the alert, corresponding actions is taken by Dronall and Roadall sub-systems for UAVs and automobiles respectively.  Dronall sub-system brings the vehicle to a halt in case static obstacles are detected in the way and in case of dynamic obstacles it moves away from the obstruction. On the other hand, Roadall sub-system gradually comes to a halt based on the speed of the detected dynamic object.  Figure 6' represents the circuit block diagram of Dronall system and Roadall system respectively which helps to understand the blue-print of the system and various module involved to give a comprehensive analysis of each of them and real time data logging of these parameters. The onboard intelligence is the Protall obstacle avoidance processor which becomes a generic model for Dronall and Roadall system.  ' Figure 7' represents the circuit diagram of Protall which consists of a Arduino Uno which is the on board intelligence used in the system, LIDAR time of flight sensor which helps in detecting obstacles, Camera sensor which helps the user to view and analyse the surrounding, speed differential sensor which helps in detecting the speed of the obstacle and buzzer to alert the user when an obstacle is in the vicinity of the vehicle.  The role of the microcontroller board in this system is to decide based on the collision risk whether or not to intervene based on the data received from different sensors. Based on the decision made by the microcontroller board, it is passed on to actuators or human-machine which performs the required actions.  The reason for choosing Arduino Uno controller is because it is provided with an open-source board that uses ATmega328 microcontroller and the Arduino IDE software is very compatible. Arduino Uno has SPI, I2C and Serial Communication protocols. It also has Analog to Digital converter (ADC) pins so that almost all the Sensors working on any communication protocol can be Embedded with it. It does not require an Internet connection to operate and also it is very cheap, unlike Raspberry pi which is ideal for this application. The pins used in the board acts as an oscillator that has a frequency of 16MHz which is suitable for almost all applications. Arduino IDE (Integrated Development Environment) software is open-source and is very compatible with all the Sensors, Actuators and Cloud platforms because of the Libraries which it provides and makes it suitable to use in this desired application.  LIDAR time of flight sensor works on I2C communication protocol and consists of 5 header pins Vcc, Gnd, SCL, SDA, GPIO1, Xshut. The Vcc pin is connected to the Vcc of microcontroller board as the operating voltage is 3.3v or 5v and Gnd pin to the gnd of board. The SDA and SCL pin of the sensor is connected to the SDA and SCL pin of microcontroller board respectively. GPIO1 is an interrupt pin that can be used to send to send a signal to the microcontroller to notify when the measurement is complete.  TTL Serial JPEG Camera sensor works on Serial communication, so the Transmitter pin is connected to the Receiver pin of microcontroller and Receiver pin is connected to Transmitter pin of microcontroller. Vcc and Gnd of sensor are connected to the Vcc and Gnd of microcontroller respectively.  Speed differential sensor consists of 4 header pins Vcc, Gnd, A0, D0. The A0 provides the Analog output of the sensor and D0 provides the Digital output of the sensor, therefore A0 pin is used and connected to the A0 of the Arduino uno to get the exact speed of the vehicle. Buzzer is directly connected to the microcontroller by connecting its Vcc and Gnd to the Vcc and Gnd of microcontroller.  Figure 8' represents the implementation of the Dronall system in the UAV, which helps understand and analyse the system practically. The circuit is rigged up module-wise to ensure every part is properly working and able to give accurate results. All the sensors are powered by the Arduino Uno with its on board 5v supply and Gnd connections. The sensors help to convert physical phenomenon into real-time human-readable data. Arduino ide software has been used to implement and insert the codes in the microcontroller to accept the sensor values and, based on that, give a response to actuators to implement the following mechanical motion. The data collected from all the sensors are being sent to the Arduino Uno, which in turn is connected to the internal microcontroller board through an I2C communication protocol, which means the data collected from Dronall system sensors are directly being sent to the internal system of the UAV as to take appropriate actions when an obstacle is detected in the field of view.

4.1.2
Roadall (automobiles): Figure 9. Circuit Realization of Roadall ' Figure 9' represents the implementation of the Roadall system in the Bot, which helps understand and analyze the system practically. All the sensors were tested and analyzed to ensure that they are working properly and provide accurate results. The sensors used in the circuit implementation are LIDAR range sensor, which is placed at the top of the Bot for detection of obstacles, Camera sensor which is connected at the front of Bot; angle sensor, which helps detect the angle of the road, Buzzer is used which beeps when any obstacles are detected, speed differential sensor to detect the speed of the obstacles, pressure sensor to detect the whole weight of the vehicle, IR sensor which is placed at the front of the Bot as to detect the obstacles in low light and foggy conditions. Active Braking System is implemented by connecting servo motor to rear drum brake parking cable to ensure optimal braking force and minimum braking distance, which helps stop the car when an obstacle is detected and the user does not intervene. The Arduino Uno powers all the sensors with its onboard 5v supply and Gnd connections. ' Figure 10' represents the images taken from the camera sensor by Roadall and Dronall systems respectively. The camera sensors are the eyes of this system which helps the user to view and analyze the surrounding and also helps in identifying obstacles. It is able to identify and distinguish cars, pedestrians, bridges, traffic signals, cyclists and road markings with the help of its smart object detection algorithms. Figure 11. 2D Mapping of room using a LIDAR ' Figure 11' represents the 2D mapping of a room using a LIDAR which helps in measuring the distance of an obstacle. To detect the correct distances of obstacles it illuminates the target with laser and then analyzes the reflected light. The red dot in the mapping represents the LIDAR itself and the black dots in the outline represents the distance readings of different obstacles present in the room, so this sensor helps in achieving 360° positional monitoring of the surrounding and this sensed data will be sent to the actuators through microcontroller and help the UAVs and automobiles to avoid the obstacles.

Conclusion
After analyzing and exploring the system, it is seen that Protall can handle any dynamic situation and act as a protecting shield for UAVs and automobiles to avoid collisions or decrease the severity of it in a few seconds before it occurs. Advancements and modifications have been made in the Protall system to make the Roadall system for automobiles and the Dronall system for UAVs as both the vehicles have different modes of working and they face different types of obstacles in their vicinity. Both the subsystem provides safety from all types of barriers that may occur during the journey. It not only aims to provide safety to manned vehicles but also unmanned aerial vehicles. The system makes a transition from providing safety to users in automobile to protecting UAVs to promote technology, reducing the use of automobiles, and eventually reducing the death rates due to road accidents.

Future Scope
The Protall system has many more features than just avoiding obstacles. A lot of development is still going on to make this system more integrable, robust and autonomous. A user-friendly application is in action with the help of the Blynk app to combine IoT technology with this system and this app will help to remotely control the systems. This app will also monitor the number of collisions and death rates that happen with and without this Protall system.
Integrating the system with intelligent machine learning algorithms to improve the system responses such that the system learns from its failure and performs better with every usage. Integrating Computer Vision technology with the help of Stereo vision sensors to this system as to detect dynamic objects and to estimate a 3D model of a scene, which can then be integrated with Artificial Intelligence to help find the vehicle a new path for them. This Technologies can be inculcated into the system to make the vehicles self-driving vehicles which will be able to detect the objects, find the appropriate path based on condition arrived and reach the destination safely all by their own with no human input [14]. As the Usage of UAVs is increasing day by day, research is going on to add wireless charging docks for UAVs so that they can be used for long-range applications. Thus, this system provides sustainable, robust, smart, ergonomic solutions to decrease the number of collision rates and accidents with the help of technology to make the vehicles smart and provide the users a great experience.