Evaluation of a Heterogeneous Sensor Network Architecture for Highly Mobile Users *

,


Introduction
Sensor networks are a promising technology to gather distributed information about the physical world and transmit it to the virtual domain.Sensor networks can be divided into two main categories: wireless sensor networks (WSNs) and personal area (sensor) networks (PASNs).Wireless sensor networks normally consist of a large number of distributed low-cost, low-power sensor nodes, or motes.Motes use low power micro-controllers and are equipped with broadcast-enabled radios which enables formation of multi hop mesh networks.WSNs are usually used in large-scale, low data rate applications that requires monitoring over long periods of time, such as environmental monitoring, defense and industrial monitoring [1,2].For dynamic and mobile applications, such as sports monitoring, a personal area network (PAN) with additional sensor nodes (PASN) is a more suitable solution.Traditional PANs consist of a small number of consumer devices which have higher processing capabilities compared to a commonly used sensor nodes.For example, mobile phones, personal digital assistants, and computers communicate with each other using standard-ized protocols, such as Wi-Fi, Bluetooth and TCP/IP.A PASN placed on a human user is sometimes referred to as a Body area network (BAN).
The targeted application class investigated in this paper is the monitoring of a group of highly mobile users, for example, fire fighters or assault teams.In many cases, these teams carry out operations in very hazardous environments.Therefore it is beneficial to monitor the health status of the teams during an operation.Sensor nodes can be deployed on a human body in order to sense health status, such as heart beat rate, stress level, body temperature, activity and pulse and transmit sensor data to a command center at some remote location.The second characteristic of the targeted applications is the network scale.The scale can be considered large compared to traditional body area networks [3], but relatively small compared to traditional WSN installations.In order to gather sufficient information, several sensor nodes must be deployed on each user.A team also consists of several users.The resulting sensor network will therefore consist of several smaller body area networks, i.e. a network of networks.Totally, there could be more than a hundred sensor nodes deployed.The third noticeable characteristic is the influence of the environment to the wireless communication.The wireless transmission could fail due to unpredictable reasons in a complex environment.Another issue requiring consideration is the operational time.Fire rescue operations usually do not last for extensive periods of time, and a system life time in the range of hours up to a week is therefore considered to be sufficient.
A WSN can address the scalability and life time issues.However it cannot address the mobility issue in a trivial way since WSN need to connect to an infrastructure network, such as Wi-Fi, mobile phone access network service or satellites [4] to send sensor data to the remote command center.As a contrast, a PAN which contains off-the-shelf devices, such as mobile phones, can easily access available infrastructure networks and gain access to the Internet.However, a PAN is only applicable for single user monitoring and the device's life time can be very short comparing with a WSN.Therefore, in [5], a network architecture which combines a WSN and PANs is proposed as a solution for the targeted applications.In this architecture, all sensor nodes deployed on users use broadcast radio (IEEE 802.15.4 [6]) in order to form a mesh network (WSN).At the same time, a dual-radio (Bluetooth [7] and IEEE 802.15.4) gateway/sensor node connects to a user's Bluetooth equipped mobile phone, which constitutes a PAN.This gateway node forwards the sensor data from the WSN to the user's mobile phone which links to a 3G network, and thus can access database servers on the Internet.Furthermore, cloud computing and web services can be used to perform resource intensive computations and add intelligence to the sensor nodes without increasing their processing power.This concept is widely adopted in the smart phones today and can be applied on Internet connected sensor nodes as well.This requires that the web service protocol stack is implemented on top of TCP/IP and deployed on the motes.Although challenging, new implementation and data encoding techniques have been developed that shows promising results in this direction [8].
Benefiting from the combination of WSN and PANs, the scalability, mobility, and interoperability of a mobile sensor network can be achieved.However, the network performance which reflects the feasibility of the proposed architecture is not investigated.It is very important and challenging to guarantee that critical sensor data, such as alarms, successfully reach the infrastructure networks and are stored on a server in an acceptable delay, typically in the range of hundreds of milliseconds.Furthermore, it is necessary to investigate the scalability of the proposed architecture since the number of sensor nodes that can be supported by a resource constrained gateway node is limited compared to traditional gateways that are less restricted in resources.
This paper is organized in the following way: Section 2 discusses related work of this paper.Experimental design for the system test is presented in Section 3, and in Section 4, the experiments' results presented.Finally, future work and conclusion of the research are presented in Sections 5 and 6, respectively.

Related Work
In [9], a mobile patient monitoring system, named Mo-biHealth was introduced by Alteren et al.This system used the general packet radio service (GPRS) and universal mobile telecommunications system (UMTS) as infrastructure networks to remotely monitor mobile patients.An iPAQ H3870 was used as the infrastructure gateway of this system and also to graphically visualize sensor data in real time.The system was tested in different European countries with different telephone network operators.Milenkovic et al. implemented a personal health monitoring system based on wireless sensor networks [10].Their approach required a personal server program to reside on a personal digital assistant, a mobile phone or a computer during operation.Sensor data were forwarded from the network coordinator to the personal server via an USB connection.For the performance evaluation of the system, the investigated metric was the power consumption.The IEEE 802.15.4 standard is also well studied in many simulations.Zheng  In this article, the investigated network's scalability is simulated using Matlab.This is because the gateway node uses a bandwidth limited Bluetooth link to communicate with consumer devices.The attribution to such slow Bluetooth data rate is due to the fact of low UART speed (57.6 Kbps) between the gateway node's microcontroller and the Bluetooth module.There are current no simulation tools available that can simulate an IEEE 802.15.4 network with such a constraint.

Experiments
For evaluating the performance of the data collection chain, experiments were designed and carried out based on the following hardware and software resources:

Gateway Node
The gateway node is a dual-chip, resource-constrained platform.Figure 1 illustrates the prototype of the gateway node.It consists of a Mulle v3.1 [13] sensor node with an extra radio transceiver.The Mulle uses a Renesas M16C/62P microcontroller as its central component running at 10 MHz.The M16C MCU features 31 kB of RAM and 384 kB of flash memory.Besides an IEEE 802.15.4 module, the gateway node is also equipped with a Mitsumi's WML-C46 AHR Bluetooth module, which can provide a 2 Mbps link with other device.However, the bandwidth to the Bluetooth 2.0 module is UART limited to 57.6 kbps in the current version.This limits the system's bandwidth since Bluetooth 2.0 can support up to 2.1 Mbit/s transfer rate.

Sensor Node
The sensor node platform used for the experiments is the Mulle v5.2 which also uses a Renesas M16C at 10 MHz.The radio module is an Atmel AT86RF230 [14] which is an IEEE 802.15.4 standard compliant module.It is used for internal communication between sensor nodes and gateway nodes.Moreover, the IEEE 802.15.4 module is configured to utilize the 2.4 GHz physical layer, which can provide up to 250 kbps of data rate.Figure 2 shows two sensor nodes (on the right hand side) transmitting sensor data to a gateway node in a two hop manner.The right hand side node is a data source node and it sends unicast packets to the node in the middle.The middle node forwards the packets to the gateway node on the left hand side.This is the experimental setup which is a part of the proposed heterogeneous sensor network architecture.

User Devices
Currently, the system has been verified to work with several off-the-shelf consumer devices.Supported mobile phone brands include iPhone, Nokia, Sony Erisson, and Motorola.It should be noticed that no application software is needed to be installed on any of these mobile phones.A laptop with the Linux operating system can also connect to the gateway node.Another popular consumer device used for mobile sensor networks is personal digital assistant (PDA).Such a device has not been tested with the gateway node, but is planned as future work.For performance evaluation described in this paper, the user device used was a Sony Ericsson K810i 3G mobile phone.

3G Network
The location where the experiments were carried out is Luleå University of Technology (LTU) in northern Sweden.A Turbo 3G network, which provides 1 Mbps maximum up link data rate and 14.4 Mbps maximum down link data rate is available at the university, and was therefore used for the experiments.The operator chosen was TeliaSonera.

Software
The gateway node utilizes the lwIP and lwBT stacks for communication, which are written in standard C code.The ActiveMessage packet format in TinyOS was used between the gateway and sensor nodes.The network's sensor nodes were programmed using TinyOS [14].As the gateway node is highly resource constrained, its memory usage is also presented in this paper to show the performance impact by running three stacks: IP, Bluetooth and 802.15.4, in parallel.

Test Cases
Test cases were designed in order to analyze the architecture's performance in terms of end-to-end delay and the number of sensor node manageable in the system (network scalability).Therefore, the test cases are:  3G network round trip time statistics  1 hop end-to-end delay with three different payload sizes  2 hops end-to-end delay with three different payload sizes  simulation of packet generation rate for sensor nodes and scalability estimation The purpose of the first test case is to investigate the 3G network's real-world performance with a regular application.Such information can be used to identify the boundary of the system's performance.The second and third test cases aim on investigating the influence of packet size and network topology to the end-to-end delay.The last test case uses Matlab to simulate the packet generation rate of sensor node.Based on that, the number of sensor nodes that can be supported by a bandwidth constrained gateway, e.g. the network scalability, can be estimated.

Infrastructure Network (3G) Statistics
The first test case measures the infrastructure network (3G) performance.In this case a Bluetooth equipped laptop accessed the Internet via a 3G mobile phone as shown in Figure 3, and has been continuously transmitting ICMP PINGs to Luleå University of Technology's web server.Figure 4 depicts the performance of the Te-liaSonera 3G network at different time periods during the same day, and on different days.To make the illustration comparable, y axle's scale is limited from 0 ms to 1000 ms.Thus, the RTT instances which exceed 1000 ms are not shown in Figure 4.As it can be seen, the round trip time (RTT) between the laptop and the server is mainly in the range from 400 ms to 500 ms independent of the time period.Table 1 summarizes the performance statistics for these three trials.It includes all RTT instances.It is important to know the 3G network performance as the up link data transmission rate and high delay is one bottle neck for the proposed architecture.For example, according to the average RTT in Table 1, packet delivery delay from a sensor node to a database server on the Internet should exceed 432/2 ms, which is 220 ms if TCP/IP payload length in a gateway node is the same within the laptop.In some cases the RTT measured was very large (around 5070 ms in the third trial) comparing with common RTTs.Such behavior explains some severe situations for mobile sensor network's end-to-end delay which is shown in following subsections.This high delay in combination with the limited memory and processing resources of the sensor node acting as gateway will introduce severe bandwidth limitations.
Another point to be noticed is that the 3G network performance can vary depending on the time of the day, the location, and the number of simultaneous users.Though the time when measurements are taken in this paper is only spread over two days, and the location is a specific city, it is still predictable that the 3G network performance will not drastically vary at different times and locations under normal network operation.

Single-and Multi-Hop Data Delivery Delay
After collecting the infrastructure performance analysis, the sensor nodes and the gateway node are added to the existing 3G network.Figure 5 illustrates a single hop and two hops heterogeneous sensor network architecture.Data delivery delay is measured for these two setups respectively.Figure 6 shows packet delivery delay for the first experiment setup, single hop transmission.The delay is represented as a time difference between packets issued from a sensor node, and received by a database server on the Internet.The effect of the IEEE 802.15.4 packet payload length to the delay is investigated in three experiments with 5 bytes, 50 bytes and 114 bytes data length.Packet payload needs to contain two bytes sensor node ID, two bytes serial number and at least one byte sensor data.Thus, the minimal data length used in the experiments is 5 bytes.The maximal data payload length is 114 bytes according to the IEEE 802.15.4 standard specification.The 50 bytes is chosen as the medium of minimal and maximal data size.Generally, the end-toend delay decreases when the data length is shortened.The maximal delay is around 950 ms when the data length is 114 bytes and the minimal delay is approximately 300 ms when the data length is 5 bytes.
Figure 7 shows the measurements of end-to-end delay from sensor node to a database server on the Internet when there are two hops between a sensor node and a gateway node.As it can be seen, in most cases, the delay becomes smaller when the data length decreases.During the experiment for 50 bytes transmission, some delays were very large: around 3300 ms, 1800 ms and 2200 ms (3 spikes in Figure 7).This could happen due to unpredictable reasons, such as 3G network fluctuation or sensor node transmission problem.For example, during the infrastructure performance evaluation in 4-A, the instability (large end-to-end delay) of 3G network is captured.The RTT for pure 3G network can be more than 5000 ms.Comparing with single hop setting, the two hops endto-end delay does not increase drastically for all three data lengths used.To further evaluate the overall performance, the delay means are summarized in Table 2.For smallest data size (5 bytes), the delay mean value difference between single hop transmission and two hops transmission is 33 ms, and for largest data size (114 bytes), this difference is 45 ms.For medium data size (50 bytes), the delay mean value difference between single hop transmission and two hops transmission is 134 ms.This value is relatively large due to the spikes as mentioned before.However, for regular cases in Figure 7, the green curve locates at similar height between red and blue as in Figure 6.Therefore, ideally, for 50 bytes data transmission, the difference of delay mean value between single hop and two hops should also be between 30 ms and 45 ms.One delay mean that needs to be noticed is that for one hop and very small data size configuration, the delay is 335 ms.This indicates that a gateway node can forward around 3 packets per second to a server on the Internet.This result is used for the network scalability evaluation.

Network Scalability
In Section 4-B, the experiment results of end-to-end delay are presented.If the sensor data length is set to 5 bytes and only one hop is needed to send this data, the end-to-end delay is 335 ms on average (in Table 2).Therefore, the number of packets with 5 bytes payload can be forwarded by the gateway node to a server on the Internet via a mobile phone's 3G network in one minute is 60 s 335 ms 180   The number of sensor nodes which can be supported by a single gateway node is calculated as 180 Packet Generation Rate of Sensor Node  If the packet generation rates for all sensor nodes are constant and equal, for example, 30 packets per minute, the network system can support 6 sensor nodes with very high packet reception rate.However, in real world, the sensor events are usually random.Thus, instead of generating constant data rate, a sensor node is more likely to generate random rate of sensor data.In this paper, the normal distribution is used to simulate the packet generation rate (number of packet per minute) for a sensor node.Figure 8 depicts a Matlab simulation result for two hundred minutes.As it can be seen in Figure 8, in most of the time units (minutes), a sensor node will transmit 30 packets while in some rare situations, a sensor node will issue few (close to 0) or many (close to 60) packets per minute.
By using the approach of packet generation rate simulation for one sensor node, the total network traffic generated by all sensor nodes can be simulated.Figure 9 plots the simulation results of the entire number of packets in each minute generated by different number of sensor nodes.This experiment simulates three packet traffic scenarios where 4, 5 and 6 sensor nodes are issuing packets respectively.Each of them lasts for two thousand minutes.The red horizontal line indicates the throughput capacity of one gateway node (180 packets/minute).As shown, in the simulation of four sensor nodes generating packets, no time instance (one minute time period) exists when the total traffic exceeds gateway throughput limit.For five sensor node traffic simulation, there are few instances when the required packet rate is larger than the throughput limit.It depends on the application whether such packet lose rate is acceptable or not, e.g.status packets can be dropped while alarm messages cannot.In the last experiment where there are six sensor nodes sending packets, a packet could be lost in nearly 50% of the time.This shows that a gateway should use compression, aggregation and other bandwidth increasing techniques for it to support larger networks.

Memory Usage
The link between the sensor network and the infrastruc-ture network is the gateway node which is resource constrained in terms of energy, bandwidth and memory.Table 3     Second, enabling buffering capability for the gateway node should be considered.Although the gateway node is memory constrained, there is still free RAM available to support packet buffering inside the gateway node.As the sensor nodes' instant packet rate is always changing according to the packet generation rate simulation, packet buffering is beneficial.This means the demand of gateway up link throughput is changing as well.Therefore, the gateway node could buffer incoming 802.15.4 packets when it reaches uplink limit, and forward these packets later to decrease packet loss rate.
It is necessary to verify the simulation result for scalability investigation in another simulator, for example NS2, since it can simulate the networking performance in a more detailed way.
Finally, it is important to investigate the possibility of adopting 6LoWPAN in the proposed architecture to enable IP at WSN level as well.This approach would increase interoperability.Another way to improve interoperability would be to use a standardized way to transmit sensor data.The use of a service-oriented architecture (SOA) would be very beneficial since it can enable automatic device and service discovery, and allow sensor data to be transmitted using standardized communication protocols, e.g.SOAP and DPWS.

Conclusions
The real world performance of a heterogeneous sensor network architecture which combines features from both WSN and PAN architectures has been experimentally studied.In order to investigate the characteristics of the used 3G network, its performance was measured at different time periods during two days.The performance of the proposed sensor network architecture was evaluated by measuring the end-to-end delay when different packet size and network topologies were used.The number of sensor nodes that can be supported by a single gateway node was simulated in Matlab.The key device of the proposed architecture is the gateway node which enables reuse of existing infrastructure network access points.The derived measurement results were based on a specific hardware and network architecture, but clearly indicate that the proposed heterogeneous sensor network architecture, using a resource-constrained sensor node as gateway to the Internet is a feasible solution for smallscale wireless mobile sensor networks.
et al. implemented an IEEE 802.15.4 patch for NS2.They presented the performance comparison between IEEE 802.15.4 and IEEE 802.11, and studied the association efficiency, orphaning, collision and data transmission methods of the IEEE 802.15.4 standard [10].In [12], Rousselot et al. investigated the possibility of accurate simulation of the IEEE 802.15.4 standard in Omnet++.They compared the simulation and experiment results in terms of timeliness and transmission success rate and analyzed the attributions of the differences between the results.

Figure 4 .
Figure 4. Statistics of round trip time for the used 3G network.

Figure 5 .
Figure 5. Single-and multi-hop data collection chain.
summarizes the memory usage of the gateway node.Memory is consumed by the Bluetooth and TCP/IP stacks, and the IEEE 802.15.4 stack.The required static random access memory (RAM) is 20.6 kB.The required read only memory (ROM) is 122.3 kB.

Figure 8 .
Figure 8. Packet generation rate distribution simulation for one sensor node.

Figure 9 .
Figure 9. Packet generation rate simulation for different sensor node numbers.

Table 3 .
Memory footprint.In the experiments, the maximum payload size was set to 129 bytes.This value could be increased to several hundred bytes, allowing the gateway node to aggregate several 802.15.4 packets in one TCP/IP packet.For example, under the current settings, a TCP/IP packet can carry 25 802.15.4 packets if the 802.15.4 packet's payload length is 5 bytes, as used in the experiments.As a result, theoretically, the packet throughput could be increased by around 25 times.