Remote inspection with multi-copters, radiological sensors and SLAM techniques

Activated material can be found in different scenarios, such as in nuclear reactor facilities or medical facilities (e.g. in positron emission tomography commonly known as PET scanning). In addition, there are unexpected scenarios resulting from possible accidents, or where dangerous material is hidden for terrorism attacks using nuclear weapons. Thus, a technological solution is important to cope with fast and reliable remote inspection. The multi-copter is a common type of Unmanned Aerial Vehicle (UAV) that provides the ability to perform a first radiological inspection in the described scenarios. The paper proposes a solution with a multi-copter equipped with on-board sensors to perform a 3D reconstruction and a radiological mapping of the scenario. A depth camera and a Geiger-Muler counter are the used sensors. The inspection is performed in two steps: i) a 3D reconstruction of the environment and ii) radiation activity inference to localise and quantify sources of radiation. Experimental results were achieved with real 3D data and simulated radiation activity. Experimental tests with real sources of radiation are planned in the next iteration of the work.


I. INTRODUCTION
E XPERIMENTAL fusion reactors are aiming for a new and viable source of energy. The normal functioning of these reactors or unforeseen problems will lead to planned and unplanned shutdowns. During this period, Remote Handling Systems (RMS) will proceed with maintenance operations [1]. The RHS require special design for regular operations of inspection and maintenance in hard conditions, such as the high levels of radiation, temperature and residual magnetic fields. The remote maintenance operations are performed during the shutdown of the reactor and shall be previously planned, given the time-consumption and costs.
A Unmanned Aerial Vehicle (UAV) can be deployed in the reactor building to quickly acquire data for a first inspection and insight view. This prompt diagnosis allows to plan ahead the sequence of maintenance operations. However, given the current state-of-the-art of the technologies, the application of a UAV inside reactors for remote inspection during the reactor's shutdown is still an ambitious goal in the near future. Nevertheless, a UAV can be deployed for remote inspection in other scenarios with lower levels of radiation. For instance, in facilities where the material used in nuclear medicine is A. Vale, H. Carvalho, R. Marques, Y. Brouwer and Bruno Goncalves are with the Instituto de Plasmas e Fusão Nuclear, Instituto Superior Técnico, Universidade de Lisboa, Av. Rovisco Pais 1, 1049-001 Lisboa, Portugal R. Ventura is with the Institute for Systems and Robotics, Instituto Superior Técnico, Universidade de Lisboa, Av. Rovisco Pais 1, 1049-001 Lisboa, Portugal stored, in locations where possible nuclear weapons are hidden for terrorism attacks or even for search-and-rescue (SaR) like operations in scenarios of nuclear disasters. The outcomes of the research and development performed using a UAV in these scenarios are important to proceed with the remote inspection in nuclear reactors.
The multi-copters are the most common UAV with the powerful capability of a loiter navigation mode, i.e., the vehicle slows to a stop and holds the position during flight while doing data acquisition. This type of UAV can operate autonomously or remotely controlled. In autonomous operation, the predefined trajectory can be optimised according to the dynamics of the vehicle and the target goal of the inspection mission. The main goal of the radiological inspection of a given scenario is to localise and quantify the sources of radiation. For the localisation purpose, a 3D representation of the environment is required.
The proposed solution in this paper provides a methodology for the 3D reconstruction of the scenario as reference for the trajectories and location of the possible sources of radiation. The 3D reconstruction is based on Simultaneous Localisation and Mapping (SLAM) techniques commonly used in mobile robot application [2]. The multi-copter is equipped with a depth camera and a Geiger-Müler (GM) counter to measure the radiological activity along the trajectory and to infer the levels of radiation of the scenario. This paper is organised as follows. The Section II presents the problem statement of 3D reconstruction using point clouds obtained from depth images and the radiological inference. The Section III proposes a solution for combining different point clouds and the algorithm to estimate how the radiological activity is mapped. Experimental results achieved with real 3D data and simulated radiation activity are presented in Section IV. Conclusions and future work are summarised in Section V.

II. PROBLEM STATEMENT
In activated scenarios, the radiation comes from activated materials or from leaks of containers with activated material inside. During the maintenance operations in nuclear fusion reactors, or during the SaR like operations in nuclear accidents (e.g. Fukushima or Chernobyl), or even during the prevention of terrorism attacks where nuclear weapons may be hidden, the challenge is usually the same: radiological inspection. It is necessary to localise the sources of radiation and then quantify the radiological intensity of the respective sources. As a future development, not covered in this paper, the identification of the type of radiation (α/β/γ) will be addressed, in order to infer the composition of the source (e.g., α emissions from 235 U [9], β − emissions from 3 H, also known as tritium [10]), using other types of sensors, such as the Cadmium Zinc Telluride (CdZnTE) or Scintillation Radiation Detectors.
Since the focus of the radiological inspection with multicopter system is providing an inspection tool for decision makers, visualisation plays a major role. A possible approach to provide an overview of the scenario and also to support the localisation of the possible points of interest is to build a 3D model, largely used in modern modulation software. Thus, the solution concept towards a 3D reconstruction of the scenario.
Besides the geometrical details, colour and contrast are also essential in the human perception of the world. Therefore, the 3D reconstruction must also include, at least, a grey-scale representation of the scenario.
The 3D of reconstruction of the environment starts with depth images, which are converted to 3D points after calibration using the pin-hole model [3], translated by where the cam X, cam Y , and cam Z are the coordinates of the 3D points in the camera's coordinate frame, u and v are the pixel coordinates, and λ is the depth. K is the intrinsic parameters matrix of the camera. This matrix is given by the calibration, which includes the pixels scale factor f x and f y , their skewness f s and the image centre in the camera frame C x and C y . Registering two point clouds, i.e., fitting two point clouds, requires the matching process between points or surfaces, as well as calculating a rigid body transformation that minimises the distance between the points. The (2) is the point-to-plane metric used in this work, where R and T are respectively a rotation and a translation, p i are the points in the moving point cloud, q i are projection of the p i points on the static point cloud planes and n i are the normals to their respective static plane.
The 3D reconstruction of the scenario provides a point cloud and a set of points describing the path of the camera. In this approach for the radiological inference, it is assumed that each point i of the map is a radioactive source with its own radiological intensity, γ i . In addition, assuming no interaction with the atmosphere, the radiological intensity decreases at a squared distance to the source [9], [10]. Along a predefined path followed by the multi-copter, the radiological intensity σ i measured at each point p i of the path, is given by the sum of the contributions of every source m j in the map, given by (3). The contribution is proportional to the inverse square distance, "d(...)" between p i and m j [8].
The path P is defined by a set of I points, p i . The Σ is the set of measurements σ i measured at each point of the trajectory. The (4) is the matrix form of (3), where Σ is a I × 1 column vector containing in element i the radiological intensity σ i measured at point p i of the path, ∆ is the I × J matrix containing in element ij the inverse square distance between points p i and m j of the path and map, respectively, and Γ is a J × 1 column vector containing in element j the radiological activity of the j-th point of the map.
In summary, the problems to be addressed in this paper are the 3D reconstruction of the scenario and the radiation activity inference, mathematically described in (2) and (4), respectively. Possible solutions are presented in the next section.

III. PROPOSED SOLUTION
This section introduces the used sensors and describes the 3D reconstruction methodology and the algorithm to infer the radiological activity.

A. Sensors
The proposed solution is based on Kinect like sensors [12], which are sensors that provide RGB or coloured images and depth images. However, the used sensor is a Structure camera from Occipital, which only provides infra-red (IR) and depth images, [13]. Depth images provide, in each pixel of the image, the distance to the closest object in the scenario. The working principle of the sensor is to project an IR laser pattern and capturing its displacement and deformation with an IR camera. From the resulted deformation the camera internally calculates the distance to the objects in the scenario.
To quantify the levels of radiation a GM counter is used, namely the model Sparkfun SEN-09848 with an USB interface. The data acquired by both sensors are synchronized and recorded in a light computer installed on the multi-copter. The data is processed offline, in an external and more powerful computer.
The algorithms for the reconstruction and radiological activity inference are detailed on the next sections.

B. 3D Reconstruction
The 3D reconstructing of the scenario aims for two main objectives. In the first one, the 3D representation provides the means for the scenario visualisation. The reconstructed point clouds colour can create a radiological heat-map, where hotspots are centred on the possible sources of radiation. In the second objective, the position and orientation of the camera relative to a global coordinate frame along the trajectory can be estimated. It is assumed that the GM counter is attached to the camera, hence, the locations and orientations (poses) of the GM counter along the trajectory are also estimated. The poses of GM are important to detect and localise the sources of radiation in the map, as suggested in Section II and further detailed in Section III-C.
The algorithm is represented in Figure 1 and detailed in the next sections.
1) Pre-Processing: The 3D reconstruction algorithm receives a set of depth and IR images sorted along the time. The IR images include a projected pattern, as illustrated in Figure 2b, used by the camera to internally calculate depth, i.e., the λ in (1). The projected pattern is similar to salt and pepper noise. The undesired effect of the pattern is reduced using a median filter with a 3 × 3 kernel, [5].
Point clouds are obtained from the depth images, which are depicted in Figure 3b. In this example, the point cloud coordinates are calculated with (1), where λ is provided by each pixel of the depth image. The grey-scale of the pointcloud comes from the unfiltered IR image.
2) Feature Extraction and Matching: The Scale Invariant Feature Transform (SIFT) was chosen for the proposed solution. The SIFT begins with a difference of Gaussians (DoG). It is a band-pass filter that can be applied to the image and tuned to find sharp transitions of a given scale in the image. The DoG highlights the transitions in the pass-band scale, so that features are then detected in the maxima of the filtered image. The algorithm is detailed in [4]. The output is an array of positions, orientations and scales in the image and their corresponding descriptor in IR 128 .
The euclidean distance criterion is used to match features, where the nearest-neighbour is selected for each feature. To avoid a high number of false-positive matches, e.g., associations of noisy and wrong features, a threshold is used on the nearest-neighbour distance ratio (NNDR), defined by (5)  nearest-neighbour. The NNDR is a matching uniqueness factor used to define if the second nearest-neighbour is sufficiently distant in comparison with the first nearest-neighbour [5].
3) Pairwise Registration: Using a point-to-plane metric, the Iterative Closest Point (ICP) calculates planes that can be used to minimise the error in (2), as detailed in [7]. The principle of ICP is the following: by iteratively reassigning points to their nearest-neighbour and minimising the distance, the cloud of points will converge to a desired configuration. The main issue of ICP is its high dependence to the initialization. If the point clouds are not "close enough" when the ICP is initialised, the result will converge to an undesired error local minimum.
The SIFT is used to improve the ICP. The SIFT features are associated to the nearest pixel using the coordinates of each feature. Then, the 3D points are obtained with (1). Hence, the  SIFT matches can be used to associate two points in the 3D space rather than two points in the image.
Afterwards, the Random Sample Consensus (Ransac) algorithm is used to fit mathematical models to data that contains outliers and, thus, to exclude false-positive feature associations. In the feature matching context, the Ransac algorithm fits a rigid body transformation with Procrustes analysis [6] (Singular Value Decomposition), which finds the optimal rotation and translation to a set of matched 3D points.
The rigid body transformation resultant from Ransac provides a good initialisation for ICP. The error evaluated in (2) is then minimised by the point-to-plane ICP, resulting in a fine approximation of the moving points to the planes calculated in the static point cloud.

4) Global Reconstruction Approach:
For the proposed solution it is assumed that the images are sorted according to the sequence of acquisition, i.e. there is a time sequence. A simple method is adopted, associating each point cloud with the previous and with the next frame. The global coordinate frame is defined in the first point cloud. The consecutive point clouds are fit to the previous one by the order of acquisition.

C. Inferring radiological activity
The goal of the inference is to obtain the matrix Γ from (4) from the measured data as illustrated in Figure 4, i.e., to infer the radiological activity of each point of the map. To estimate the Γ given the measurement data corrupted by noise, a possible solution is given by (6), where ∆ + is given by (7), the Moore-Penrose pseudoinverse [11].
This paper presents results with real data of depth images, but with simulated data of radiological measurements. The approach to simulate the data is described as follows. The radiation sources are distributed, i.e., each point of the map will contribute with some radiation. The intensity of each point is set with a fall-off 3D Gaussian exponential. The radiation intensity of a point is so much higher as lower is its distance to the centre of the Gaussian.
Let k be the number of sources of radiation, n the index of the n-th source, a n the "strength", s n = (x n , y n ) the position, η n the width factor and m j = (x j , y j ) the j-th point of the map. Then, the radiological activity of a point belonging to the map is given by the sum of the contributions of all radiation sources. The radiological activity, γ j , is given by (8). Figure 5 illustrates a simulated planar scenario, with a rectangular shape (assumed on an horizontal plane) and three Gaussian radiation sources with different intensities. The zaxis represents the radiation intensity in the Figure 5, as well as in the next two figures. According to (8), every point m j in the map of Figure 5 shall contribute with a given amount of radiation, resulting in σ i measured at point p i along a path. The Table I   Two paths were tested in the described scenario, with 20 waypoints p i each one. The first path describing a "S" shape along an horizontal plane above the map was tested, as represented in Figure 6. This path is sparse and covers almost the scenario. The second path describing a "L" shape only covers part of the scenario's periphery, as illustrated in Figure 8.
In both trajectories, it is possible to notice a Gaussian like curve centred in the area corresponding to each point of the path. In addition, the inferred radiological activity of the scenario is higher where the measured radiological activity is higher, and lower where the measured radiological activity is lower. In Figure 6, the first trajectory covers almost the whole map, while the second trajectory in Figure 8 only covers the periphery. The inferred radiological map provides a better approximation of the simulated radiation in the first case. This result was expected, since there are more points (2500) in space than the acquisitions along the trajectory (20). Therefore, the system of equations in (6) is under-determined. The pseudo-inverse matrix ∆ + found in (7) provides different results, depending on whether the path is spread across all space or just covering part of it.

IV. EXPERIMENTAL AND SIMULATED RESULTS
The results of the proposed solution for radiological inspection are shown in this section. The proposed solution was tested in the entrance of a laboratory building, as illustrated in the panoramic picture of Figure 7. This scenario includes a corridor, walls, doors and stairs.
The experiment was performed using a total of 150 depth images and their corresponding 150 IR images. The sensor was moved at an approximately constant speed and following a path with a spiral shape. The orientation of the sensor is  perpendicular to the shape of the path and pointing to the walls or doors. The 3D reconstruction of the scenario is depicted in Figure 9. Then, a source of radiation was simulated in a corner of the scenario as described in Section III-C. The inferred radiation map is illustrated in Figure 10, as well as the estimation of the followed path (represented by a line), where the depth and IR images were acquired. The dots along the line represent the positions and the radiological information was simulated as sensor data acquisition.

V. CONCLUSION AND FUTURE WORK
The paper presented a solution for the 3D reconstruction of the scenario as reference for the localisation and quantification of possible sources of radiation. The used sensors are depth cameras and Geiger-Müller counter. The 3D reconstruction is based on techniques for association of images and cloud of   points, commonly used in mobile robot applications. After the 3D reconstruction, the radiation map is inferred given the radiation measurements, with the respective position and orientation of the sensor where the data was acquired. The solution is dependent to the trajectory, which suggests that the challenge of localise the sources of radiation is also a path planning problem. In addition, the radiological intensity is inversely proportional to the square of the distance between the sensor and the source. In addition, the Geiger-Müller counter is very small, with low efficiency and, thus, the sensor has to move closer to the sources of radiation to acquire any reliable information. For the application with drones, the proximity to the objects in the scenario (below 1 meter) increases the risk of collision and the dynamics interference on the flight. Therefore, the next work is to test the proposed solution with the drone depicted in Figure 11, flying inside a room with real sources of radiation, namely the Potassium-40. Other sensors, such as Cadmium Zinc Telluride (CdZnTe) or scintillators based sensors are being assessed as alternative radiological sensors, with better efficiency and capability to identify the sources of radiation.