Optimal Joint Expected Delay Forwarding in Delay Tolerant Networks

Multicopy forwarding schemes have been employed in delay tolerant network (DTN) to improve the delivery delay and delivery rate. Much effort has been focused on reducing the routing cost while retaining high performance. This paper aims to provide an optimal joint expected delay forwarding (OJEDF) protocol which minimizes the expected delay while satisfying a certain constant on the number of forwardings per message. We propose a comprehensive forwarding metric called joint expected delay (JED) which is a function of remaining hop-count (or ticket) and residual lifetime. We use backward induction to calculate JED by modeling forwarding as an optimal stopping rule problem. We also present an extension to allow OJEDF to run in delay constrained scenarios. We implement OJEDF as well as several other protocols and perform trace-driven simulations. Simulation results confirm that OJEDF shows superiority in delay and cost with acceptable decrease of delivery rate.


Introduction
A delay tolerant network (DTN) [1] is a sparse mobile multihop network where a complete path from the source to the destination may not exit most of the time. A DTN can be viewed as a collection of time-varying and separate node clusters. Under this condition, conventional mobile ad hoc network (MANET) or wireless sensor network (WSN) routing will become impractical because of frequent disruption and high packet loss. The Delay-Tolerant Networking Research Group (DTNRG) [2] is concerned with architecture and relevant protocols in order to provide interoperable communications with and among extreme and performancechallenged environments. In DTNs, the messages can be forwarded asynchronously using the store-carry-forward routing mechanism which relies on the contact opportunities between nodes [3].
Routing in DTN has been an active research area in recent years. Epidemic [4] exchanges the copies whenever contact occurred within pair of nodes. Epidemic can guarantee the maximized delivery rate and the minimized end-toend delay under ideal conditions. However, unrestrained flooding is impractical because of huge consumption of energy, buffer and bandwidth. Much literature has been focused on reducing the number of copies of each message while retaining a high routing performance. Previous works proposed a variety of metrics, including movement trend [5], delivery probability [6], encounter frequency [7], the distance to destination [8], end-to-end delay [9], energy [10], signal to noise ratio (SNR) [11], and comprehensive utility [12]. These metrics are used to make decision for forwarding in order to optimize certain performance such as expected delay and delivery rate.
In this paper, we propose optimal joint expected delay forwarding whose main idea is similar to that of optimal probabilistic forwarding (OPF) [13]. This means that optimal stopping rule [14] is used again. Different from maximizing the delivery rate in OPF, we concentrate on optimizing the expected delay of each message with the constraint on the maximum of copies per message. The problem is how to 2 International Journal of Distributed Sensor Networks forward the copies of each message in each discrete time slot in order to minimize the expected delay when the number of copies is given. To solve this problem, we firstly employ the comprehensive dynamic forwarding metric called joint expected delay (JED) which is a function of two important states of a message copy: remaining hop-count and residual lifetime. Then, we propose the forwarding rules to decide whether to forward the copies. Based on the hop-count constraint forwarding scheme, we propose a general ticket constraint OJEDF which can be viewed as a development of Spray and Wait [15] with special spray metric. We also extend OJEDF in delay constrained scenarios.
We performed simulations using St. Andrews encounter trace [16] and Illinois encounter trace [17] to evaluate the routing performance of OJEDF against several DTN routing protocols, including Spray and Wait, ProphetV2 [7], and Epidemic, in terms of average delay, delivery rate, and the number of forwardings for each message. Simulation results confirmed that OJEDF improves the average delay and cost with acceptable decrease of delivery rate. This paper is organized as follows. Section 2 summarizes the related works. Section 3 introduces preliminaries on optimal joint expected delay forwarding and presents the motivation and assumption of OJEDF. Section 4 proposes the calculation algorithms of our delivery metric JED and optimal forwarding rule in hop-count constraint OJEDF and ticket constraint OJEDF, respectively. Section 5 proposes an extension of OJEDF which makes it work in delay constrained scenarios. Section 6 presents our simulation methods and results. Finally, the paper is concluded in Section 7.

Related Works
The single-copy routing protocols such as MobiSpace [18] and CAR [19] are energy efficient but always have large delay. Due to uncertainty in nodal mobility and network partition, DTN routing algorithms usually spawn and keep multiple copies for the same message in different nodes. Epidemic is the first distribution routing protocol for DTN, and it gains good delivery delay. However, Epidemic wastes a lot of energy and suffers from severe contention.
Much literature has aimed to balance the delay and energy. Some restricted flooding or spray mechanisms disseminate a small number of message copies to potential relay nodes, and then each copy finds routing to the destination independently. These mechanisms can obtain better delay performance without a lot of resource consumption.
Spray and Wait is a multicopy routing without any routing knowledge. Spray and Wait combines the speed of Epidemic routing with the simplicity and thriftiness of direct transmission. This mechanism "sprays" a number of copies into the network and then "waits" till one of these nodes meets the destination. Spray and Focus [20] replaced the direct transmission in wait phase with utility-based single-copy strategy and proposed the utility transfer mechanism to disseminate the history contact information. Thrasyvoulos was absorbed in utility-based spraying [21] and proposed three potential utility functions: last-seen-first spraying, mostmobile-first spraying, and most-social-first spraying. Jindal proposed distance utility-based spray strategy [22] which utilized dynamic programming to calculate the optimal relay node. An adaptive distributed spray mechanism (AMR) was performed in [23] which determined the depth of spray tree by relay nodes. But AMR is just efficient in random waypoint model and sometimes can produce at most /2 copy redundancy ( is the number of nodes). In addition, theoretical analyses of expected delay in single-copy case and multicopy case were proposed in [24,25], respectively. Different from Spray and Wait, delegation forwarding [26] forwards the messages based on different delivery probability metrics. Delegation forwarding maintains a forwarding threshold when contact of node pairs occurred. Forwarding threshold indicates the quality of node such as cost, delivery rate, and average delay. RAPID [27] treats DTN routing as a resource allocation problem that translates the routing metric into per-packet utilities such as minimizing average delay, minimizing missed deadlines, and minimizing maximum delay.
Some routing protocols aimed to achieve the desired performance with different levels of prior knowledge about the network. OPF [28] was proposed to maximize the delivery rate of each message when mean inter-meeting times between all pairs of nodes were known. Cyclic MobiSpace [29] assumes that each node has full contact information to other nodes, and it optimizes minimum expected delay.

Hop-Count Constrained
Forwarding and Ticket Constrained Forwarding. We consider two constraint patterns related to the number of message copies: hop-count constrained forwarding and ticket constrained forwarding.
In hop-count pattern, a maximal remaining hop-count was defined in source node. The remaining hop-count decreased progressively when forwarding occurred. The calculation of the remaining hop-count for the copies of each message is independent. Figure 1 depicts the forwarding process with = 3 in node . Ticket pattern uses the total number of copies which can be redistributed in each forwarding. Figure 2 depicts the ticket constrained forwarding process with = 8 in node using binary spray [9].

Motivation.
Many routing protocols in DTNs forward the messages based on the specific routing metrics by means of comparing the direct forwarding qualities of node pairs. However, such forwarding strategy ignores the time dimensionality which is an important character in DTNs. In fact, the forwarding decision only relied on the direct forwarding qualities of node pairs and cannot stand for the optimal choice in whole time dimensionality of delivery.
In this paper, we optimize the expected delay of each message based on the comprehensive forwarding metric JED , , ,Tr for each copy in for destination with remaining hop-count (or tickets) and residual time-to-live Tr.  We use backward induction to calculate JEDs by modeling forwarding as an optimal stopping rule problem. Our contributions are as follows.
(1) We propose a comprehensive forwarding metric JED which is related to the time dimensionality and the number of copies for each message.
(2) We propose the backward induction algorithms to calculate JEDs with hop-count constraint and ticket constraint, respectively.
(3) We solve the delay restraint optimal cost forwarding by the extension of OJEDF.

Assumption.
To model our optimal forwarding problem as an optimal stopping rule problem, we have the following assumptions.
(1) Like other probabilistic forwarding protocols, nodal mobility exhibits long-term regularities and each node knows the matrix × which contains the mean inter-meeting times MIT , between all pairs of nodes { , }. This matrix can be estimated from contact history.
(2) Time dimensionality in whole forwarding process is discrete. OJEDF employs a discrete residual time-tolive Tr with time slot size for each message copy. So we can state the next time-slot as Tr −1.
(3) JEDs are exponentially distributed. Therefore, the JED of two copies with JED 1 and JED 2 can be calculated by Proof. Let JED 1 = 1/ and JED 2 = 1/ , and let the joint probability density function be ( ) = − ⋅ − . Then, the expected value is Similarly, the meeting probability of two nodes in any time slot with size can be estimated by

OJEDF with Hop-Count Constraint.
Without loss of generality, we consider a copy in node with destination , remaining hop-count ( ≥ 1), and residual time-to-live Tr. Upon meeting with , can either forward the copy to or not. Since we assume the forwarding process is discrete, the JED in the next time slot can be stated as JED , , ,Tr −1 in no forwarding case. As an alternative, the copy can be forwarded and is logically regarded as being replaced by two new copies, both of which have − 1 remaining hop-count. If the copy is forwarded, the JED of the copy in node will be JED , , −1,Tr −1 and that of the copy in node will become JED , , −1,Tr −1 . The joint JED can be calculated using formula (1).
We only consider unicast forwarding. When a node contacts with several nodes at the same time slot,   the copy is forwarded to the node which has the minimum JED , , −1,Tr −1 .
The forwarding metric JED , , ,Tr equals the sum of the weighted expected delay in forwarding case and not forwarding case. In forwarding case, node has the opportunity to meet the relay node , , . . . with probability , , , , . . ., respectively. So the joint expected delay that node meets one of the nodes , , . . . in time-slot Tr and then forwards the copy to it is If node did not meet any relay node, the copy was kept in node with joint expected delay JED , , ,Tr −1 and probability Algorithm 1 shows the calculation of JED , , ,Tr in hop-count based forwarding using the backward induction.
Specially, the copy only can be delivered to the destination when the remaining hop-count = 0, and we set JED , ,0,Tr = MIT , . For each pairs of nodes { , | = }, we let JED , , ,Tr = ( is a very small number such as 10 − 6) in order to make the formula (1) calculable.
In our OJEDF, Tr is only used to define the finite upper bound on the number of stages which is essential for modeling OJEDF as optimal stopping rule problem. So it is feasible to calculate the JEDs of the previous stage using the backward induction method even when JED , , ,Tr > Tr. This implies that the OJEDF rule depicted in formula (4) is valid when JED , , ,Tr > Tr. However, this kind of JEDs cannot be employed in delay constrained scenarios which we will introduce in Section 5 since it is impossible to deliver the message within residual time-to-live.
As an example, a matrix 8×8 including discrete mean inter-meeting times (normalized by time-slot size ) between each pair of nodes was defined in Table 1. The JEDs calculated by OJEDF using Algorithm 1 were shown in Figure 3 with = 3 (enough to forward each message to all nodes), maximum Tr = 30, = 0, = 1, and = 1. In Figure 3, the JEDs are decreased with increasing and Tr. It means that a message has smaller expected delay when more remaining hop-count and residual time can be used to accomplish delivery. As a result, we can make forwarding options based on formula (4)  that the message forwarding tends to be more cautious when the remaining hop-count declines gradually along with the forwarding process.

OJEDF with Ticket Constraint.
Another copy constraint pattern is ticket constraint which is more precise than hopcount constraint since it replaces the remaining hop-count with the number of tickets . Without loss of generality, we consider a copy in node with destination , the number of tickets ( > 1), and residual time-to-live Tr. When node meets any relay node , the forwarding option depends on JED , , ,Tr −1 and the joint JED of JED , , 1,Tr −1 and JED , , 2,Tr −1 ( 1 + 2 = ). We define , MIN = min(1/(1/JED , , 1,Tr −1 + 1/JED , , 2,Tr −1 )), and the copy is forwarded only if JED , , ,Tr −1 > , MIN .
Algorithm 2 shows the backward induction algorithm to calculate JED , , ,Tr for ticket based forwarding. Specially, the copy only can be delivered to the destination when the number of tickets = 1, and we set JED , ,1,Tr = MIT , .
We use the same matrix defined in Table 1. The JEDs calculated by OJEDF in ticket based forwarding were shown in Figure 5 with

An Extension of OJEDF for Delay Constrained Scenarios
We have solved the optimal joint expected delay problem with the constraint of remaining hop-count or ticket of messages. As a further consideration, we extend OJEDF to run in delay constrained DTN applications. For example, electronic notice in campus networks [30] and village networks [31] and shortterm weather information in large national parks [32] must be forwarded with constrained delay. On the other hand, as the access networks, configuration and forecast for QoS are also necessary in order to provide acceptable service (such as e-mail) in DTNs.
Since JED is a function of remaining hop-count (or the number of tickets) and residual lifetime, the OJEDF can also be used to estimate the minimum hop-count or the minimum tickets to meet the constrained delay. We give a simple hop-count estimation algorithm with constrained delay based on JED in Algorithm 3. We define the MAX as the maximum of hop-count which equals ⌈log 2 ⌉ ( is the number of nodes) which is large enough to spray copies to all of the nodes in hop-count based forwarding. The constrained delay (CD) and initial time-to-live TTL (TTL init ) must be defined in advance. Both of them are normalized by time-slot and satisfy TTL init ≥ CD. Similarly, Algorithm 4 shows the ticket estimation algorithm in which the maximum of tickets MAX equals . We can get the minimum of hop-count with different constrained delay for each destination node using Algorithm 3. Assuming that the message was generated in node 0 and the mean inter-meeting times matrix was in Table 1, Figure 7 showed the minimum of hop-count with TTL = 29. The ticket estimation was shown in Figure 8 with the same parameters.

Simulation
We evaluated OJEDF with hop-count constraint (OJEDF-H) and OJEDF with ticket constraint (OJEDF-T) against Spray and Wait (SNW), ProphetV2 (PRO2), and Epidemic (EP) using St. Andrews encounter trace and Illinois encounter trace. We implemented the above protocols in The ONE [33] which was developed by the Helsinki University of Technology. We obtained the performance indicators including average delay (only of the copies delivered successfully), the delivery rate, and the average number of copies for each message (no acknowledgment mechanism) from The ONE. The simulation parameters were shown in Table 2 (only the default values without any special instruction). from the participants' Facebook friend lists. Participants were asked to carry the devices whenever possible. Due to the limit of buffer and the queue of external events, we only employ the encounter data from time 0 to 5369. We optimize the original data in order to remove the reduplicative records and invalid users. The reduplicative records are the same encounter records in the same time, and the invalid users mean the users who never appeared before time 5369. As a result, we get 25 valid users and 591 external encounter events. The final encounter data must be formatted into standard external event queue data which can be read by The ONE.
We get the performance with different message overload. The intervals of message generator varied from 10 s to 50 s. Figure 9(a) showed the average delay with different intervals of message generator. The average delay increased with the increasing intervals. When the message overload was high, the multihop forwardings were difficult due to the constrained buffer size. As a result, the average delay was low. The average delay increased with the decreased message  overload since the multihop forwardings became common. There was no copy control in EP and PRO2, so the average delay of them was high compared with that of SNW and OJEDF due to the constrained buffer size set in simulation. We can see from Figure 9(a) that the average delay of PRO2 was lower than that of SNW and OJEDF when the interval of message generator was long enough. This is because the buffer size becomes sufficient in low message overload, and the average delay is low when there is no copy control in PRO2. OJEDF which optimizes the joint expected delay can be viewed as an improvement of SNW. The average delay rates of OJEDF-H and OJEDF-T were 8.89% and 14.05% lower than that of SNW, respectively. Figure 9(b) showed the delivery rate with different intervals of message generator. The delivery rate increased when the overload decreased. All of the delivery rates are lower than 50%. This is because only 244 contacts occurred between 25 users in the first 5369 s. This is a very low contact probability. OJEDF did not forward the copies to the node first encountered but forwarded based on the expected delay. So the delivery rates of OJEDF-H and OJEDF-T were 14.20% and 13.61% lower than that of SNW in average, respectively. Most of the time, the delivery rate of OJEDF is higher than that of EP and PRO2.
We also got the number of forwardings with different intervals of message generator. We can see form Figure 9(c) that the numbers of forwardings of OJEDF-H and OJEDF-T were much lower than those of EP, PRO2, and SNW. OJEDF only forwarded the messages to the optimal node, and the numbers of forwardings of OJEDF-H and OJEDF-T were 34.86% and 36.00% lower than that of SNW, respectively.
We also compared the performance with different TTLs (from 10 minutes to 90 minutes). Figure 10(a) showed the average delay. The average delay means the delivery delay of messages which were delivered successfully, so the average delay was low when TTL was low. The average delay rates of EP and PRO2 were higher than those of SNW and OJEDF. OJEDF outperformed other protocols in terms of the average delay with all TTLs. Specially, the average delay rates of OJEDF-H and OJEDF-T were 9.24% and 18% lower than that of SNW, respectively. Figure 10(b) showed the delivery rate with different TTLs. With increasing TTL, more messages were unable to be forwarded due to the restraint of buffer size. As a result, the delivery rate of EP and PRO2 trended to decrease. On the contrary, International Journal of Distributed Sensor Networks   We tested the performance with different message overload. The intervals of message generator were varied from 100 s to 500 s. Figures 11(a), 11(b), and 11(c) showed the sim ulation results using Illinois encounter trace. Like the results in St. Andrews encounter trace, EP and PRO2 still have higher average delay than other copy constrained algorithms. The average delay rates of OJEDF-H and OJEDF-T were 6.09% and 12.14% lower than that of SNW, respectively. The delivery rate increased when the overload decreased. All of the delivery rates are lower than 21%. This is because about 38.08% of the total 7017 contacts occurred between user 3 and user 4 or between user 4 and user 6. These unevenly distributed contacts led to low delivery rate when we only  employed the encounter trace in one day. Different from Saint Andrews encounter trace, we set a big buffer relative to message size, so the delivery rates of EP and PRO2 were higher than those of OJEDF-H and OJEDF-T. The delivery rates of OJEDF-H and OJEDF-T were 15.55% and 9.66% lower than that of SNW in average, respectively. OJEDF still had the least forwardings for each message. The numbers of forwardings of OJEDF-H and OJEDF-T were 10.90% and 11.86% lower than that of SNW, respectively.
We compared the performance with different TTLs. Large TTLs (from 100 minutes to 500 minutes) were used in our simulations. Firstly, the valid encounters lasted for 230301 seconds at 02-26-2010 in Illinois encounter trace. Secondly, the encounters were unevenly distributed between seven persons. As a result, many copies need more time to find useful encounters during all simulation time and to meet the destinations ultimately. Figures 12(a), 12(b), and 12(c) showed the performance of different algorithms with different TTLs using Illinois encounter trace. The average delay rates of EP and PRO2 were lower than that of SNW. This profited from flooding based diffusion and sufficient buffer. OJEDF outperformed all protocols in terms of the average delay with all TTLs. The average delay rates of OJEDF-H and OJEDF-T were 47.93% and 52.76% lower than that of SNW, respectively. The delivery rates of all protocols increased with increasing TTLs in Illinois encounter trace scenario. The delivery rates of OJEDF-H and OJEDF-T were 28.61% and 21.91% lower than that of SNW, respectively. But the numbers of forwardings of OJEDF-H and OJEDF-T were 11.65% and 12.37% lower than that of SNW, respectively.

Summary of Simulation.
Simulation results confirm that OJEDF outperforms all protocols in terms of average delivery delay and forwarding cost using both St. Andrews encounter trace and Illinois encounter trace. The delivery delay and the number of forwardings of OJEDF-H are 18% and 23% lower than those of SNW while decreasing only 15% delivery rate. OJEDF-T is more accurate than OJEDF-H in terms of copy control. The delivery delay and the number of forwardings of OJEDF-T are 24% and 25% lower than those of SNW while decreasing only 12% delivery rate.

Conclusion
In this paper, we provide an optimal forwarding protocol which minimizes the expected delay while satisfying the constant on the number of forwardings per message. We propose the optimal joint expected delay forwarding which makes optimal forwarding decisions by modeling forwarding as an optimal stopping rule problem. We firstly employ the comprehensive dynamic forwarding metric called joint expected delay (JED) which is a function of two important states of a message copy: remaining hop-count and residual lifetime. Then we propose the forwarding rules to decide whether to forward the copies. Based on the hop-count constraint forwarding scheme, we propose a general ticket constraint OJEDF which can be viewed as a development of Spray and Wait with special spray metric. We also present an extension to allow OJEDF to run in delay constrained scenarios. We implemented OJEDF as well as several other protocols and performed trace-driven simulations. Simulation results verified the efficiency of OJEDF.
In the future, we will perform simulations on the extended version of OJEDF, that is, estimating the minimum hop-count or tickets to meet the constrained delay. We will also do research on the expansibility and adaptability of OJEDF when only partial routing information is known.