A four-state Markov model for modelling bursty traffic and benchmarking of random early detection

,


Introduction
Internet communication has increased the widespread use of network devices, with routers playing a crucial role in this network infrastructure.Data packets are transmitted through the network and pass through multiple routers before reaching their destinations.Within this complex network ecosystem, routers serve as key waypoints for these packets, temporarily storing them in memory buffers as they await further processing and forwarding.The memory within the router has a finite capacity, and when the memory buffer is overloaded and cannot accommodate incoming packets, packet loss occurs due to buffer congestion.Specifically, buffer congestion arises when the demand for buffer resources exceeds capacity, eventually leading to packet loss.As such, congestion is a critical network problem because it can significantly influence network performance, causing delays and reducing overall efficiency (Jafri et al., 2022).
To effectively manage the flow of packets within router buffers and mitigate the congestion risks, routers employ two distinct but interrelated packet management mechanisms: queue management and scheduling.As discussed by Zhang et al. (2011), Queue management methods focus on how routers organize and prioritize packets within their buffers.These methods determine which packets should be forwarded next, which should be dropped, and how to allocate resources to ensure fair and efficient packet processing.The scheduling mechanisms address the temporal aspects of packet forwarding.Scheduling decides when specific packets should be transmitted from the router's buffer onto the outgoing links, considering factors like packet priorities and quality of service (QoS) requirements.Effective scheduling ensures that packets are sent out in a manner that optimizes network performance while meeting various service-level agreements (Marin et al., 2020).
Queue management is a critical packet management aspect, especially in rapidly changing traffic.With the potential for a sudden increase in network traffic and the rapid accumulation of packets in router buffers, the effectiveness of traditional scheduling mechanisms can be compromised.Accordingly, queue management, such as the Random Early Detection (RED), is pivotal in handling rapidly changing traffic.Queue management commonly uses packet dropping, which involves selectively discarding packets to alleviate congestion consequences within routers.Depending on the queue management mechanism, packet dropping as a congestion-handling technique may be used before or after packet accumulations (Almomani et al., 2019).
As such, two broad categories of congestion control methods have been proposed: active and lazy mechanisms.Lazy methods, such as the drop tail approach, detect congestion only after the router buffer reaches capacity, often resulting in packet loss.On the other hand, active queue management (AQM) methods are designed to predict congestion before occurrence and respond to changes in network traffic at an earlier stage.AQM methods aim to control congestion before it becomes severe proactively.Notable AQM methods include RED (Floyd & Jacobson, 1993) and BLUE (Feng et al., 1999), which have been extended to conduct various advanced AQM methods.Subsequent developments resulted in more sophisticated AQM techniques such as the Effective RED (Abbasov & Korukoglu, 2009), Time-window Augmented RED (Windowed-RED), and Enhanced RED (EnRED) (Abu-Shareha, 2019).
As Internet traffic is highly dynamic and diverse, it exhibits unique features that must be considered for effective network management.These features include inherent aggregation, burstiness, and correlation (Liu et al., 2008).The presence of bursts of arrival packets and an inter-packet time shorter than the average intra-packet time can be early indicators of traffic congestion, a critical issue to be addressed in network management (Khalil, 1994).Moreover, the global Internet network caters to a wide array of traffic types, which presents a significant challenge when modeling.Addressing this challenge is crucial when developing, evaluating, and benchmarking existing AQM methods (Domaski et al., 2018).Traditionally, AQM methods have relied on a modeling approach known as the Bernoulli process, known for its simplicity and ease of implementation.However, this modeling approach, while stable, often falls short in accurately capturing the complex and dynamic nature of real-world network traffic.Hence, there is a need for an innovative model that can better emulate the complexities of Internet traffic, ensuring that AQM methods can adapt effectively to the evolving demands of modern network environments (Saaidah et al., 2014).
To address the limitations of the Bernoulli process (BP), the Markov Model (MM) is employed, bringing a richer and more adaptable framework to the analysis of network behavior.The MM is a robust mathematical method extensively used to describe and predict variations and burstiness within a network's traffic patterns.This approach is fundamentally rooted in the concept of Markov chains, where the probability of an event is intricately linked to the probability of the system's current state, as vividly depicted in Figure 1.In essence, the MM is a powerful tool for calculating the probabilities of various network events, notably packet arrivals, based on the current state of an independent chain of states.This flexibility allows network analysts to model the arrival process in multiple ways.MM can be described using discrete time units, where events are meticulously tracked at specific intervals, providing insights into precisely when events occur.Alternatively, this arrival process can be represented as batches of units, signifying that events arrive in clusters or groups, highlighting scenarios where network traffic exhibits burstiness.Moreover, the MM can also represent the arrival process as a continuous quantity, indicating that events flow seamlessly without distinct time intervals, which is particularly useful for understanding smooth and continuous data streams.One of the notable advantages of the MM is its adaptability to various time scales.Accordingly, network analysts can leverage the MM to capture and analyze network behavior across different time frames, from milliseconds to hours or even days.This versatility in timescale analysis empowers researchers to understand network dynamics comprehensively, making the MM a valuable tool in network modeling and analysis (Yu-Dong et al., 2009).(Alsaaidah et al., 2016).
The MMFF models arrivals as a continuous flow.Finally, MMBP models arrivals as independent events, with the probability of an event changing according to the current state.Various approaches have used MMBP for traffic modulation.However, existing methods use only two states (MMBP-2), which may not be sufficient to model the variability of Internet traffic (Li et al., 2020).
This paper introduces a novel packet arrival model utilizing a four-state Markov-modulated Bernoulli process (MMBP-4) and conducts a comparative analysis against the MMBP-2 and basic Bernoulli Process (BP).The primary objective of this model is to replicate the same overall arrival probability as traditional models but with the added dimension of time-varying characteristics.As such, this research leverages the Random Early Detection (RED) algorithm as a case study to evaluate the effectiveness of the proposed MMBP-4 model.RED, a widely used Active Queue Management (AQM) technique, relies on accurate traffic characterization to efficiently manage congestion and ensure Quality of Service (QoS) in network communication.
The aim of applying MMBP-4 on RED is to assess its ability to capture intricate traffic characteristics, including burstiness, temporal dependencies, and fluctuations in network data arrivals.This evaluation will show whether the proposed model can enhance modeling traffic behavior and improve congestion control strategies.Accordingly, the rest of the paper is organized as follows: Section 2 provides an overview of the RED method and discusses the packet management mechanisms.The related works on AQM and MMBP are discussed in detail in Section 3. Section 4 presents the proposed work for modeling arrival traffic using MMBP-4 for RED.The results are discussed in Section 5. Finally, the conclusion is stated in Section 6.

Background on the RED Method
The RED method (Floyd & Jacobson, 1993) is a fundamental active queue management method that inspired various techniques for network congestion control.Unlike other network management techniques, RED operates without the need for explicit notification of the senders.Instead, it relies on implicit notification, primarily through the practice of packet dropping when buffer congestion occurs.RED's operation depends on the dropping probability (Dp) concept, which determines the likelihood of dropping packets in a stochastic or probabilistic manner.The key to calculating this Dp lies in monitoring the average queue length (aql) within the network router or buffer, a metric that reflects the current congestion state.
Fig. 2 provides a visual representation of the process, showing the essential role of two thresholds in determining the buffer's status.These thresholds serve as indicators of the buffer's length and congestion levels.The advantage of RED's stochastic mechanism is the ability to prevent global synchronization, a phenomenon that can detrimentally influence computer networks.RED predicts the buffer status by comparing the aql with the minimum and maximum thresholds.When the aql is below the minimum threshold, typically indicating a safe buffer state, RED refrains from dropping any packets, effectively setting Dp to zero.When the aql surpasses the minimum threshold but remains below the maximum threshold, RED implements stochastic packet dropping.Dp is set between 0 and 1, reflecting a measured response to buffer congestion without resorting to full-scale packet drops.Finally, when the aql exceeds the maximum threshold, RED takes decisive action to alleviate congestion.In this scenario, full packet dropping is implemented, signified by setting Dp to 1.This aggressive response helps maintain network stability and prevents further impacts of the congestion consequences.
RED was one of the first Active Queue Management (AQM) methods that enabled early congestion detection and addressed the global synchronization problem (Abu-Shareha, 2022).However, RED exhibits several drawbacks, summarized as follows: 1) RED depends on aql for calculating Dp, which estimates the traffic load based on the connections, ignoring the packet load status (Ahmed & Nasrelden, 2018).2) Incorrectly configured threshold values may reduce network performance in delays and packet loss (Zhu et al., 2023).3) Although the goal of calculating and using Dp is to maintain moderate queue length, RED reaction to the sudden change in the network traffic (i.e., bursty traffic) is relatively slow (Abu-Alhaj et al., 2021).Accordingly, the behavior of the RED should be analyzed under correlated and bursty traffic conditions to improve its performance.This paper proposes an analysis of the RED method under the MMBP-4 model.

Literature Review
Various methods have been proposed to implement one of the MM types, which are used to build and analyze queue management techniques.Wang et al. (2011) introduced a robust network model and analysis method to predict and manage queuing delays at a specific level.The model operates within a single buffer under multi-class traffic conditions, accommodating multiple packet sources.The model consists of three essential sub-components: the predictor, the monitor, and the processor.The predictor anticipates delay values within successive time windows and adjusts the threshold position based on its relationship with the theoretical delay.The monitor mechanism dynamically regulates the incoming packets' Variable Bit Rate (VBR).The processor computes the delay within each time slot.To evaluate and compare the results, the study employed the BP model with disturbances against the RED (Floyd & Jacobson, 1993) and PI (Hollot et al., 2001) methods.Various AQM methods were proposed and evaluated using the BP model, such as BLUE (Feng et al., 1999), EnRED (Abu-Shareha, 2019), and PI (Hollot et al., 2001).
In the MMBP-2, Guan, Awan, et al. (2004) presented a discrete stochastic queueing model for analyzing RED performance with bursty and correlated traffic.Accordingly, an MMBP-2 is used with a fixed state probability (i.e., 0.9), while the other has a variable probability in the range of (0-1).The evaluation results were used to set the threshold to its optimal value and adjust the dropping probabilities for the RED method to match the required type of service.Similarly, Guan, Woodward, et al. (2004) proposed an analytical model to improve delay by mitigating aggressive source behavior.In this model, the source refrains from sending packets once it exceeds a specific threshold, effectively treating it as implicit feedback from the queue.Furthermore, these thresholds undergo dynamic adjustments based on the network's state.Consequently, this model controls the mean queue to a predetermined value, ensuring the fulfillment of the required Quality of Service (QoS) specifications.
The authors conducted their analysis using the MMBP-2 framework.The probabilities associated with the two states were closely clustered (e.g., 0.20 and 0.25) and were lower than the departure rate (0.3).As a result, the experiments were conducted under moderate traffic status.Guan et al. (2006) proposed a new method for AQM aimed at maintaining the buffer queue at a specific level, even when dealing with delay constraints.The proposed method used a bang-bang type control strategy (Chen et al., 2023), which controls the arrival rate and dynamically adjusts a threshold.The threshold is adapted based on the difference between the mean and target delay.The arrival rate is calculated based on the queue length.Like Guan, Woodward, et al. (2004), this model was developed and analyzed using MMBP-2, with arrival probabilities of 0.20 and 0.25 and a departure rate of 0.3.Lim et al. (2011) introduced an adaptive queue management method to stabilize the delay within the queue.This approach was built based on an aggregated traffic model.The proposed method reduced the queuing time by adjusting the mobile queuing threshold that governs traffic arrival rates.This queuing threshold dynamically adapts based on calculated average delays.A discrete queuing model was developed to simulate the method's process using an aggregated traffic model with N overlapping flows, modeled using MMBP-2 arrival processes.Various settings for MMBP-2 were employed; however, the results showed consistency across these settings.Consequently, the experiments were conducted under moderate traffic load.Saaidah et al. (2014) analyzed the performance of the BLUE method to adjust its parameter settings using MMBP-2 with varied state probabilities.The results showed that the performance of BLUE, simulated and analyzed using MMBP-2 during heavy congestion, is better than that obtained using BP.The results suggested that BLUE is suitable for implementation on routers processing Internet traffic with a bursty and correlated nature.Similarly, Alsaaidah et al. (2016) analyze the performance of the BLUE and Gentle BLUE (GBLUE) methods.The performance analysis shows that Gentle BLUE outperforms BLUE.For RED, Xu et al. (2021) presented a performance analysis model using MMBP-2.
The need for the MMBP-4 models was realized recently.Accordingly, Mahawish and Hassan (2022) proposed utilizing the Markov decision process (MDP) to modify the RED algorithm with MMBP-4.In this approach, various queue length parameters were modeled using the four states in the MMBP-4; each influences the performance of the RED differently.Additionally, each transition probability between the four states in the MMBP represents a different decision to be made.
According to the obtained results, MMBP-4 improves the RED's performance.
A survey of various Markov Models (MM) was presented by Zaryadov et al. (2017), which examined different AQM methods.Table 1 summarizes the methods and aims of the developed AQMs.In essence, the modeling of arrival traffic should accurately capture the characteristics of real traffic.Internet traffic is inherently aggregated, exhibiting burstiness and correlation.To accommodate these traits and represent probabilistic traffic effectively, the Markov-Modulated Bernoulli Process (MMBP) is commonly employed for arrival traffic modeling.However, the simplicity of the implemented models has limited their ability to represent the complexity of actual traffic.Therefore, this paper introduces a more intricate MMBP-4 model instead of the commonly used BP and MMBP-2 models.By increasing the number of states in the Markov Model, we enhance our ability to model the intricacies of Internet traffic and generate more complex probabilities.

The Proposed Work
BP is a stochastic and discrete-time process with a number of independent random variables Xi.Using BP, packet arrival events stochastically occur at every time slot, represented by a variable Xi.The probability of packet arrival pk, in the time slot k, is independent of the arrival probability at other time slots, as BP is memoryless.Packet arrival in slot k follows a binomial distribution, with a probability of p for a packet arrival at Xi and a probability of failure, represented as 1−p.The gap between two successive arrivals follows a geometric distribution with parameter p. MMBP is employed to evaluate and benchmark the RED AQM method.However, BP does not account for the burstiness of traffic in the design and evaluation of AQM.This paper introduces a model that extends such traffic modeling with four states while maintaining an overall probability to allow for a comparison between MMBP-4, MMBP-2, and BP. Figure 3 illustrates the proposed framework for benchmarking RED using BP, MMPP-2, and MMBP-4.

Parameter Initialization
The parameter settings are listed in Table 2.The departure probability for all models and all experiments is set to 0.5 for all models.The packet arrival probability for each model is set to different values in each experiment, ranging from [0.18-0.93].
The overall probability for each model is identical for each experiment to create a fair comparison.Each value creates a different traffic load environment.Heavy traffic is generated with possible congestion as the arrival probability exceeds the departure probability.On the other hand, a light traffic load is generated when the arrival probability is below the departure.

Traffic Modeling
Using the discrete-time queue model, unequal time slots capture and evaluate the model performance, each accommodating one departure or arrival event or both.Departure occurs before arrival, and both events depend on the events' probabilities and a linear congruential generator.According to the MMBP, these events occur based on the probability of the model's current state.MMBP-4 uses four states (i.e., s0, s1, s2, s3), each state has a different packet arrival probability value (i.e., p0, p1, p2, p3).The transmission probability from one state to another is represented in Eq. ( 1) for MMBP-4 and Eq. ( 2) for MMBP-2.The initial state for packet arrival in MMBP-2 and MMBP-4 is s0.The packet arrival is then implemented depending on the state's probability.The next event occurs in the same state or after transferring to another, as shown in Fig. 4.

Fig. 4. RED-MMBP-4
As such, assume that the packet arrives in state s0 during time slot i.The next arrival process occurs in the same state, with a probability of p0, or the process transmitted to another state, with a probability of (1-p0)/3.The key distinction between BP, MMBP-2, and MMBP-4 lies in using multiple states instead of a single state in BP.The transmission probabilities are represented by a transition probability matrix Rm.In contrast, the arrival probabilities at different states are represented using the diagonal matrix Am, as given in Eq. (3) and Eq. ( 4), respectively.
Thus, to achieve a specific overall probability of value v, the average of these probabilities should be equal to v. To simplify the process, each of these probabilities can be set to v.As such, Ps0 BP-2 and Ps1 BP-2 are both set to v. The values of P(s0) and P(s1) are set to different values in the range of [0-1], to create different network load.Finally, the values of P0 and P1 are calculated based on the equations above.A similar technique is implemented for the four-state Markov model.

Dropping Probability Calculation
According to the RED method, Dp is calculated based on the pre-calculated aql, two predetermined thresholds, a pre-defined parameter, Dmax, and a counter.The counter is set to -1 when the packet is not dropped to increase the probability of dropping the next packet and avoid global synchronization.On the other hand, the counter's value is set to 0 when the packet is dropped to decrease the probability of dropping the next packet.In all cases, Dp is calculated as given in Eqs.(11)(12).

Performance Metrics
The performance of the RED method under different traffic generation processes is measured using the mean queue length (MQL), the queuing delay (DEL), the dropping rate (DR), and packet loss (PL).The MQL is the average number of packets accommodated in the router buffer.This measure can be classified as a cost measure.The lower the measure, the better the performance.The calculation of the MQL measure is implemented as given in Eq. ( 13).
where N is the router capacity, and i represents a specific queue length within the buffer capacity.
DEL is the packets' average waiting time, calculated using Little's law, as given in Eq. ( 14).
PL is the ratio of lost packets compared to the number of arrived packets.DR reflects the rate of dropping packets based on the AQM decision.All these measures are cost types, yet these should be balanced between PL and DR.Accordingly, the increase of DR is not always related to decreased performance.

The Experimental Results
The BP, MMBP-2, and MMBP-4 methods were simulated, and the results are discussed based on the previously mentioned performance measures.

The Simulation Environment
Java was used to simulate RED-BP, RED-MMBP-2, and RED-MMBP-4 models.As illustrated in Figure 5, the discrete-time queue is used as it is the most utilized queuing technique.The simulation environment is a single router buffer node with First Come, First Serve (FCFS) as the scheduling method.

Results
The results of the RED method using different modeling techniques based on MQL, DEL, DR, and PL are illustrated in Figs.
(6-9).Fig. 6 illustrates the performance of the RED-BP, RED-MMBP-2, and RED-MMBP-4 based on MQL.As shown, the performance of these methods varied based on the arrival probability.In the case that the arrival probability is low (i.e., α is up to 0.33), no congestion occurs.In such a case, RED-MMBP-4 provides equal MQL results with other methods.In this case, the traffic characteristics are not revealed with such a low arrival rate.The value of the MQL is varied when the arrival rate increases to create a congestion state.In such a case, the results for the MMBP-4 are worse than those of the MMBP-2 and BP, as the traffic characteristics are exposed in such a case, and MMBP-4 revealed the slow response of the RED, as discussed earlier.shaping dropping rates.However, as the arrival probability continues to rise, the dynamics shift.RED-MMBP-2 takes the lead in maintaining a more favorable dropping rate than the other methods.Concurrently, RED-BP's performance remains good but begins to drop down.In contrast, RED-MMBP-4 experiences challenges in adapting to the changing load conditions.The gradual increase in packet arrivals leads to accumulation within the network buffer, ultimately impacting network performance.While Dropping Rate (DR) is indeed a cost measure, it becomes necessary during periods of heavy traffic to prevent congestion-induced loss.In this context, RED-MMBP-4 falls short in dropping adequate packets to effectively manage the congested network, resulting in undesirable packet loss.These findings highlight the importance of selecting the right congestion control method based on the prevailing traffic conditions, as a one-size-fits-all approach may not yield optimal results.Fig. 9. Methods performance based on PL Fig. 9 illustrates the performance comparison based on PL.Notably, all three models exhibit identical PL performance when operating under light and moderate traffic conditions, where the packet arrival rate (α) remains within the range of 0 to 0.63.However, it's under conditions of heavy congestion that the distinctions become apparent.In this scenario, RED-MMBP-2 emerges as the standout performer, demonstrating superior PL performance compared to the other methods.This observation underscores the effectiveness of RED-MMBP-2 in mitigating packet loss during periods of intense network congestion.

Conclusion
In this paper, a novel four-state Markov-modulated Bernoulli process is designed to serve as a robust benchmark for evaluating the performance of the RED method.To create a meaningful comparison, we also establish two-state and one-state models by carefully solving the relevant equations, ensuring they share similar probabilities while exhibiting distinct burstiness characteristics.Using the proposed model, the research vividly highlights the limitations of the RED method, particularly when contrasted with analyses involving models featuring only one or two states.The essential traffic properties were captured through our developed model, including burstiness and correlation dynamics.The results showed that the proposed model has better performance measures for MQL, DEL, PL, and DR, especially with heavy congestion.These findings underscore the potential of our model as a valuable tool for assessing and enhancing network congestion control mechanisms.

Fig. 2 .
Fig. 2. Single Router Buffer for RED Fig. 5. RED-MMBP-4 Model Fig.7illustrates the performance of the RED-BP, RED-MMBP-2, and RED-MMBP-4 based on DEL.Again, the performance of these methods varied based on the arrival probability.In the light traffic case, RED-MMBP-4 provides equal DEL results with other methods.The results are expected as the traffic characteristics are not revealed with such a low arrival rate.With heavy traffic load, the results for the RED-BP are better than those of the MMBP-2 and RED-MMBP-4, as the traffic characteristics are exposed in such a case, and MMBP-4 is a better model to address the limitations of the RED method.

Fig. 6 .Fig. 7 .Fig. 8 .
Fig. 6.Methods performance based on MQL Flow (MMFF), and Markov-modulated Bernoulli Process (MMBP).MMPP models the arrival events following the Poisson distribution that can change based on the current state.Using MMPP, only a single event can occur instantly, whether an arrival, departure, or both.As a result, it cannot represent digitalized communication well

Table 1
Summary of the Related Work

Table 2
Parameter Settings