Mind-Controlling Green Unmanned Vehicles through the Cloud: Building a Prototype

—Due to their practicality and low cost, unmanned vehicles have gained a lot of popularity recently. In this paper, we present our work on designing and building green unmanned vehicle prototypes powered by batteries that can be charged by photovoltaic cells. Moreover, our prototype vehicles are equipped with Arduino boards connected to different wireless communication devices (3G, WiFi, and Bluetooth) allowing them to be controlled in various ways. They can be controlled by establishing a connection between the Arduino board and the control device, which can be a smart phone or a laptop along with a neuroheadset. The connection can be direct as with the Bluetooth technology or indirect by using a dedicated cloud server. Finally, the vehicles are equipped with a camera that streams video wirelessly to the user. 


I. INTRODUCTION
Nowadays, security and safety are top priorities for most people. For tasks requiring exploring, inspecting, surveillance or monitoring of areas that are either too expensive to monitor or are not "human-friendly," robots present an appealing solution. Due to the nature of such tasks, fully autonomous systems might not be suitable [1]. Instead, researchers propose to build systems where semiautonomous vehicles equipped with cameras as well as other sensors are controlled by humans residing in safe locations. In this project, we propose to build unmanned vehicles to perform the exploration, inspection, surveillance or monitoring tasks. The control and coordination between these components can be handled by humans thousands of miles away using easy-to-use smart phone applications. The system we are proposing will be capable of monitoring large hazardous regions with minimal risk, cost and human effort.
One of the benefits of such projects is to provide a cost-efficient way to monitor large rural or hostile areas with minimal human effort and low risk. By allowing humans to use smart phones to control both UAVs and UGVs, the monitoring process will have minimal risk on human life. Moreover, the operational cost associated with training humans to control the devices and paying  Manuscript received December 1, 2014; revised April 11, 2015. their salaries will be drastically decreased. Finally, the use of smart phone applications means that there will no longer be a need for a centralized specially-equipped control station.
The contributions of this work are several. We discuss our work on designing and building green unmanned vehicles powered by batteries that can be charged by photovoltaic cells. Moreover, our designed vehicles are equipped with Arduino boards connected to several sensors such as temperature and humidity sensors, ultrasonic sensors, etc. The Arduino board is also connected to different wireless communication devices called shields (3G, WiFi, and Bluetooth) allowing the vehicles to be controlled in various ways. They can be controlled by establishing a connection between the Arduino board and the control device, which can be a smart phone or a laptop along with a neuroheadset (a brain-computer interface (BCI) device capable of reading EEG signals representing "mental commands" and transforming them into real commands). The connection can be direct as with the Bluetooth technology or indirect by using a dedicated cloud server. The ability to use cloud systems to control the vehicles allows the human controller of the vehicle to control it through the Internet thousands of miles away. This opens the door for a wide range of applications. Finally, the vehicles are equipped with a camera that streams video wirelessly to the user. Due to the various challenges in implementing the above ideas, we limit our focus in this paper to Unmanned Ground Vehicles (UGVs) and Unmanned Surface Vehicles (USVs). We are currently working on porting some of the above ideas to the more popular Unmanned Aerial Vehicles (UAVs).
The rest of this paper is organized as follows. In Section II, we present a brief review of the some of the existing works with similar objectives as ours. In Section III, we discuss in details our unmanned vehicles in terms of the design and the hardware used. Finally, we conclude the paper and discuss future work guidelines in Section IV.

II. RELATED WORKS
As discussed earlier, using unmanned vehicles to inspect or monitor hazardous areas have been the focus of numerous ambitious research projects. Such systems have been proposed since the 1950s [2]. Although it is more appealing to design autonomous systems [3]- [6], certain tasks require some level of human intervention [1]. Such systems use communication network technologies to allow humans to control surveillance vehicle thousands of miles away. Several projects [7]- [12] proposed to design such semi-autonomous surveillance systems. For a survey on these topics, the interested reader is invited to check out [13], [14].
One of the most interesting and closest works to ours is that of Surmann et al. [1], in which the authors designed a system that combines the capabilities of both UAVs and UGVs to monitor large areas. We propose to build a system similar to [1]'s in functionality, but with a looser coupling between the different unmanned vehicles. Moreover, our system will enjoy reduced complexity and cost due to the recent advances in the field of robotics since the completion of Surmann et al. [1] project in 2008.
In this work, we focus on semi-autonomous systems that are very crucial for certain tasks such as the scanning of (urban or rural) hostile regions, urban search and rescue missions (especially after natural disasters that render the environment not suitable for humans such as volcanoes), monitoring of large rural areas such as vast forests or long shorelines, inspection of industrial locations (especially nuclear plants), etc. Most of the previous works focused on either UAVs or UGVs but not both. Moreover, the works on USV are very limited. We believe that the combination of the capabilities of different kinds of unmanned vehicles and the coordination between them can significantly enhance the system's ability to perform many tasks. For example, in case of a natural disaster, such as an earthquake, rescue missions are forced to look through debris for survivors. Obviously, UGVs are suitable for such tasks, however, going under the rubbles will reduce their abilities to communicate with the human controller especially taking into account that access points and base stations are probably out of service due to the natural disaster. That is when UAVs can serve as a relay (an intermediate point of communication) between the underground UGVs and unaffected means of communication such as satellites.
One of the contributions of this work is the use of multiple control methods such as direct wireless connection between the user's laptop/PC with the unmanned vehicle (which is what most of the previous works use). Another approach is to use a smart phone to control the vehicles as used by many recent works. The most exciting choice for us was the use of a neuroheadset as a control device. A neuroheadset is a brain-computer interface (BCI) device capable of reading EEG signals representing mental commands and transforming them into real commands. According to Wikipedia, 1 research on BCI dates back to the 1970s. Since then, many applications of BCI have emerged especially in neuroprosthetics that aim at restoring damaged hearing, sight and movement.
Most recently, the continuous research on non-invasive EEG-based BCI techniques produced low-cost lightweight devices with a decent enough level of 1 http://en.wikipedia.org/wiki/Brain-computer_interface accuracy to be practically employed. One such example is Emotiv's EPOC neuroheadset. While there have been more research efforts in different fields benefitting from such neuroheadsets (such as controlling robotic arms [15]), we are yet to see wide adoption of this technology into the field of designing and controlling unmanned vehicles. Our work integrates Emotiv's EPOC neuroheadset and smart phone technologies to control our unmanned vehicles. We are unaware of many research projects with similar goals. Some works [16]- [19] have managed to integrate the use of EPOC neuroheadset and similar BCI devices with smart phones; however, their goals are different from ours and have nothing to do with robotics and unmanned vehicles.
A relevant idea to ours is to use BCI devices to control vehicle. For example, [20]- [22] used them to control wheelchairs. In [23], Bothra and Torun used EPOC neuroheadset to control an A. R. Drone (an unmanned aerial vehicle) called SWARM Extreme. Gomez-Gil et al. [24] used EPOC neuroheadset to steer a tractor and compared this method with manual steering as well as autonomous steering. Also, Raul Rojas and his team at the Free University of Berlin tested several vehicle control approaches including an iPhone, an iPad, an eyetracking device, and, most importantly, using EPOC neuroheadset. They called their system BrainDriver.
What is exciting about BrainDriver is that it was tested on a real car (called MadeInGermany, which is a modified Volkswagen Passat Variant 3c) driving in real environment (the former airport in Berlin Tempelhof) [25], [26]. For their senior design project, Perez et al. [27] created EPOC-alypse, a ground vehicle controlled by EPOC neuroheadset. They also used an Intel DH61AG motherboard, Xbee series 1 Bluetooth transmitter/receiver, and an Arduino board located on the vehicle. Finally, for a brief coverage of over uses of BCI devices in robotics, the interested reader is referred to Reyes and Tosunoglu's paper [28].

III. OUR UNMANNED VEHICLES
Before going into the details of the design and building of our vehicles, it is beneficial to discuss the main challenges/objectives of this work.

A. Objectives/Challenges
The first one is how to enable humans to easily control unmanned vehicles thousands of miles away with minimal delay, cost and effort. As mentioned previously, we equip our unmanned vehicles with the necessary components to provide the controllers with enough knowledge of their surrounding and to receive control commands from the users who will be using a smart phone application to control the vehicles. Below, we discuss the details of both major components of this work.
The vehicles are equipped with several components to allow them to successfully complete their tasks. They are equipped with cameras to capture (and stream) videos of their surroundings. This video feed will be compressed and transmitted via 3G, WiFi or Bluetooth networks, and thus, the vehicles should be equipped with proper communication devices (such as 3G-enabled antennas). The vehicles should include in their transmission their current location as well as some vital readings such as their positions, power levels, temperatures, humidity levels, etc. So, proper sensors (including a GPS device) are mounted on the vehicles. The same communication device will be used to receive the control commands from the users/controllers. To coordinate everything, small chips, with acceptable levels of processing capabilities and memory size are mounted on the vehicles.
The control application has a user-friendly interface showing the video feed from the vehicle along with its position and other vital readings. As part of our future improvements of the control application, we intend to make the control application more practical and easy to use. E.g., to accommodate the small screen sizes of some of the smart phones, we will build three types of interfaces: abstract, normal and detailed, where each type contains a level of details suitable to certain range of screen sizes. The application will make use of any feature of modern smart phone such as the tilt sensor (as well as other accelerometers) to allow the user to easily control the vehicles.

B. General Design
Since our prototype vehicles are complex with various features/capabilities, we discuss here the most important parts of the design. Inspired by [11], Fig. 1 shows the block diagram of our prototype vehicles. The figure shows how the user can interact with prototype vehicles either using a smart phone or using a laptop/PC (possible using a neuroheadset). The communication with the Arduino board is established either directly or indirectly through a cloud system. The use of cloud computing technologies is very appealing due to many reasons such as continuous availability and resilience to disasters and attacks. However, the most appealing aspect of using cloud computing technologies is the magnificent processing and storage capabilities provided at a very reasonable price [29]. This allows us to extend our prototype and add several compute-intensive features to it such as real-time processing of the video feed or the ability to make autonomous decisions. We now discuss the details of the hardware used in building the prototype vehicles. The most important parts are the Arduino boards. We use two different boards: the Arduino Uno board and the Arduino Mega board. Fig. 2 shows the pinout diagram of the Arduino Uno board. We also make use of several other parts such as breadboards, H-bridges, stepper motors, relays, regulators, etc., which are connected to the Arduino board. Most importantly, the Arduino board is connected to the proper "shields" to facilitate wireless communications such as the Adafruit HUZZAH CC3000 WiFi shield. Figure 2. The pinout diagram of the Arduino Uno board [30].
Looking back at Fig. 1 one can see that we are using two different types of motors: DC motors and servo motors to initiate the movements of UGV and USV, respectively. The servo motors are also helpful in directing the ultrasonic sensors to provide a better sense of the surrounding environment in order to avoid obstacles. The figure also mentions the sensors. The only types of sensors used in this project are the camera, ultrasonic and temperature sensors. The camera we use is the Foscam FI9821W Wireless IP Camera, which has 300 degree pan and 120 degree tilt. So, there is no need for a servo motor to direct it. However, this camera is a bit large and heavy for small prototype vehicles like ours. For the UGV chassis, we use a generic 4WD chassis bought online, while, for the USV, we manually build the body using foam board typically used for insulation. Finally, for the power, we use rechargeable batteries and install several photovoltaic cells to recharge them giving our prototype the ability to benefit from green power sources. Harvesting green power is one of the main themes of modern scientific research [31]. To be more specific, we use a solar charger and several polycrystalline silicon solar panels, each with a peak current of 1.5W and peak voltage of 6V. To obtain more green power, we also use several monocrystalline miniature solar cells, each with a peak current of 830mA and peak voltage of 0.5V.
What makes our work stand out is the various control methods provided to the user. The user can simply use a laptop/PC and connect directly to the Arduino board (e.g., using WiFi) to control the prototype vehicles. Many existing projects use this approach. A more recent trend is to take advantage of the widespread use of smart phone and design a smart phone application to control the vehicle. We implement both approaches. For the smart phone control application we choose to work on the Android mobile operating system. Additionally, we make use of two cutting edge technologies to provide the users with more flexibility and ease of use. The first one is the use of the Berg Cloud system 2 as an intermediate step between any device connected to the Internet and the Arduino board of our prototype vehicles. The control program was written in the JavaScript Object Notation (JSON) language. The second technology is the EPOC neuroheadset (shown in Fig. 3) which is developed by the San Francisco-based company, Emotiv. It is one of the first consumer-targeted BCI devices to become commercially available with an affordable price. It is a WiFi-enabled non-invasive neuroheadset equipped with a large number of electrodes. These electrodes are strategically positioned on the user's scalp to accurately measure electroencephalograph (EEG) and electromyography (EMG) signals. These signals are then mapped into actions/gestures/emotions. Other advantages of using the EPOC neuroheadset compared with competing devices (such as Neurosky Mindset, 3 OCZ Neural Impulse Actuator, 4 etc.) is the large community of researchers/developers using it, the very helpful and easy to use software development kit it comes with and its use of saline-moistened felt pads for the electrodes as an alternative for the conductive paste typically used to reduce electrical impedance and improve conductivity between the electrode and the scalp [19].
After taking some time to get used to it, this neuroheadset makes a wonderful control device that theoretically can be used by anyone even if they suffer from severe motor-impairment. The neuroheadset comes with proprietary software from Emotiv called TestBench that displays the data stream coming from the headset in real-time manner. Among many things, it shows sensor contact quality, EEG readings, FFT, gyro, wireless packet acquisition/loss display, etc. See Fig. 4. The neuroheadset offers exciting control features that we did not get a chance to explore with this prototype. It is capable of providing real-time measurements of the facial expressions (such as blink, left wink, right wink, furrow (frown), raise brow (surprise), smile, clench teeth (grimace), glance left, glance right, laugh, smirk (left side) and smirk (right side)) and the emotional state of the user (such as instantaneous excitement, long term excitement, frustration, engagement and meditation). The main functionality of the neuroheadset we exploit is its ability to detect mental commands. To do so, the software much be trained as follows. The user must repeatedly focus on a specific thought (such as moving the vehicle forward) and the software learns the patterns of the electrical activity of the user's brain measured by EEG/EMG which are associated with this thought. Then, whenever this pattern is detected, the software can simply translate it into a command (move the vehicle forward) that is sent to the Arduino board so that the vehicle can indeed move forward. Emotiv's website lists the following 13 pretrained thoughts: push, pull, lift, drop, left, right, rotate clockwise, rotate anticlockwise, rotate forwards, rotate backwards, rotate left, rotate right and disappear. In his thesis, Wright mentions that "skilled users may train and be monitored for up to 4 different thought patterns at once" [19], [32]. IV. CONCLUSION This paper discusses designing and building green UGVs and USVs. The built vehicles exploit many stateof-the-art tools and technologies such as smart phones, cloud systems, neuroheadsets, etc. They are equipped with many wireless communication devices making them practical and useful for many applications such as going into hostile environments, participating in urban search and rescue missions, etc. The vehicles are constructed using off-the-shelf parts and products with very limited budget in terms of both time and money.