An unmanned aerial vehicle (UAV) - aka drone, unmanned aircraft system or remotely piloted aircraft system - is an aircraft without a human pilot on board. Its flight can be controlled autonomously by computers in the vehicle or by remote control. UAVs can uniquely penetrate areas, which may be too dangerous or too difficult to reach for piloted craft. The UAV cyber-physical system comprises all the subsystems and interfaces for processing and communication functions performed by the embedded electronic system (avionics) and the ground control station. To accomplish the desired real-time autonomy, the avionics is highly tied with aerodynamics sensing and actuation. An entirely autonomous UAV can (i) obtain evidence about the environment, (ii) work for an extended period of time without human interference, (iii) move either all or part of itself all over its operating location devoid of human help and (iv) stay away from risky situations for people and their assets. This chapter intends to introduce the material addressed in further chapters of this book. The next sections go through some concepts that are recurrent in the book.

Optical flow (OF) plays a decisive role in visual situation awareness, detection and obstacle avoidance systems for unmanned aerial vehicles (UAVs), which are cyberphysical systems (CPSs) that interact with the environment through sensors and actuators. The use of cameras allows the integration of computer vision (CV) algorithms with the inertial navigation systems (INS). The movement of characteristics of the image fused with the dynamic of the UAVs allows us to improve the process of remoting sense, avoid obstacles or estimate the position and velocity of the UAV. In the literature, there are various algorithms to locate characteristics points between two consecutive images. However, the computation time and consumption of physical resources such as memory features are due to embedded systems. This chapter shows (i) how to integrate the movement of the pixel textures (OF) in the image with INS data, (ii) compares different algorithms to match points between consecutive images, (iii) implements a process to encounter points between consecutive images and (iv) implements a computationally less expensive and with less memory consumption algorithm. A case study about using the field-programmable gate array (FPGA) as part of the visual servoing is discussed showing how to integrate results into the CV hardware system of a UAV and addressing the need to handle issues such as multi-resolution.

The modelling of a system's dynamics and performing simulation are commonly used techniques and almost critical components in the modern development of manned and unmanned systems alike. This chapter addresses the need for modelling and simulation of unmanned air vehicle systems and reviews historical developments in the field, current techniques used and future developments.

As the use of unmanned aerial vehicles increased, studies regarding their autonomous flight became an academic field of great interest for researchers. Until recently, most studies based their developments using an inertial measurement unit (IMU) and a global navigation satellite system (GNSS) as the main sensors to calculate and estimate the UAVs pose. These sensors, on the other hand, have several limitations, which can affect the navigation, therefore, the fully autonomous aspect of the system. Images captured during flight, computer vision algorithms, and photogrammetry concepts have become a core source of data to estimate the UAVs pose in real-time, therefore, composing new alternative or redundant navigation systems. Several algorithms have been proposed in the scientific community, each one working better in specific situations and using different kinds of imaging sensors (active and passive sensors). This chapter describes the main visual-based pose estimation algorithms and discusses where they best apply and when each fails. Fresh results depict the development of new strategies that will overcome the remaining challenges of this research field.

This chapter presents the concepts, advantages, and practical examples of computer vision on robot operating system (ROS), applied explicitly on unmanned aerial vehicle (UAV) domain. ROS was built with the concept of abstraction (using messages interfaces) to allow re-use of software. This concept allows the integration with already built source code for computer vision tasks with the ROS environment. In conjunction with the available simulators for ROS, it is possible to design UAV solutions in a model-based concept, anticipating many operational and technical constraints prior to operational release.

This chapter explores how the type of environmental impacts the CV techniques, algorithms and specific hardware to be used. Indoor environments, also known as controlled environments, generally rely on solutions based on beacons, proximity sensors and image processing for data acquisition. In this case, as the environment is controlled, the illuminance of the scene is adjusted and sensors are previously positioned, which facilitates the development and execution of these systems. In outdoor environments, generally known for uncontrolled environmental variables, frequently require solutions based on image processing techniques to provide the data acquisition. In this environment, the non-constant variation of the illuminance of the scene and the great variation of the background of images are important complicating factors for the operation of the image processing algorithms. In addition, constructions and buildings block the signal of sensors and global positioning systems making it even harder to treat the exceptions caused by these factors. Each exception being treated in a CV system has a computational cost that can be high. If this is considered in applications using embedded hardware, some projects simply become infeasible. Researchers put great effort attempting to optimise the software for high performance and better use of the hardware resources, so that less processing power is demanded as well as positively impacting the energy savings. This chapter presents a review of the main CV techniques currently used in the development of mission control software for the use in indoor and outdoor environments, providing autonomy navigation and interaction for these aerial robots.

In typical intelligence surveillance and reconnaissance (ISR) missions, persistent surveillance is commonly defined as the exercise of automatic intelligence discovery by monitoring a wide area coverage for hours of operation at a high altitude leveraging aerial platforms (manned or unmanned). The platform can be large enough to carry a matrix of high-resolution sensors and a rack of high-performance computing equipment to process in real-time all sensors' feeds. With the current ISR growing in capability, engineering and optics-based aerial surveillance to find a suitable design solution became a design challenge. More onboard processing is desired for an increasing fidelity/resolution sensors' feed, while matching a constraining SWaP (size, weight, and power) budget requirements in a bandwidthconstrained operating theatre. The advent in small unmanned aerial vehicle (sUAV) technology, able to carry sophisticated optics payloads and to take aerial images from strategic viewpoints has become unavoidable in nowadays battlespace contributing in moving forward the ISR capabilities. The constrained on-board processing power in addition to the strict limit in the flying time of sUAV are amongst the serious challenges to overcome to enable cost-effective persistent surveillance based on sUAV platforms. All previous examples show that tailoring the sensors to match the platforms' environment is a challenging endeavour and therefore architects have shifted their design methodology to be based on hardware and software open architectures as a centrepiece of their approach in building cost-effective surveillance solution design. This chapter is a brief introduction to hardware and software building blocks for developing persistent surveillance systems. In our context, the focus is in particular on Electro-Optic (EO, visual spectrum) and Infrared (IR) integrated solutions leveraging computer vision techniques for surveillance missions.