ANFIS-based rate adaptation scheme for adaptive streaming over HTTP

Abstract

Recently, HTTP-based video streaming traffic has continued to increase. Therefore, video service providers have been using HTTP-based adaptive streaming (HAS) technology to reduce the traffic load of the HTTP server. Accordingly, many adaptive bit rate (ABR) schemes have been proposed to provide a high quality of experience (QoE) to video service clients. In this paper, we propose a new ABR scheme using an adaptive network-based fuzzy inference system (ANFIS), which is one of the neuro-fuzzy structures. The proposed scheme learns optimal fuzzy parameters by using (1) the learning ability of ANFIS and (2) the video streaming data providing high QoE to clients. Then, the bit rate of the next segment is determined according to these trained parameters. In the simulation using NS-3, we show that the proposed scheme selects the appropriate bit rate under various wireless network conditions and provides better QoE to clients than the existing schemes.

1 Introduction

In recent years, HTTP-based video streaming traffic has consistently increased. It is expected to increase more than ever due to the expansion of streaming audio and video adoption and the emergence of state-of-the-art video technologies such as 4K, high-dynamic-range video, and virtual reality [1]. This traffic growth has led to the adoption of HTTP-based adaptive streaming (HAS) technology by video service providers to reduce the traffic load of the HTTP server. HAS technology encodes video at multiple bit rates and divides the video at each bit rate into multiple segments with a fixed playback duration. Further, data such as media representation, segment duration, and segment URLs are stored as a file in the HTTP server. The clients download the file before the video starts and sequentially request a segment at the appropriate bit rate by using the file to download and play the segment. Thus, HAS technology reduces the traffic load on the server and saves bandwidth resources by sending only a single segment for each client request rather than the entire video [2].

The dynamic streaming over HTTP (DASH) standard [3, 4] developed by MPEG is one of the most popular HAS technologies. DASH has been adopted by major video service providers such as YouTube and Netflix. As DASH is widely used, various methods have been proposed to optimize its service. In [5], a quality of experience (QoE)-aware wireless resource allocation method considering DASH clients has been proposed. The proposed method reduces rebuffering time by considering the buffer level of DASH clients. It, however, does not consider non-DASH clients. Therefore, a new method considering both DASH clients and non-DASH clients has been proposed in [6]. This method reduces the rebuffering time and guarantees fairness among non-DASH clients. The authors in [7] introduce a new transmission architecture for DASH video streaming over the LTE system. They calculate the importance of each packet and propose a packet scheduling strategy and resource allocation method based on the calculated importance of each packet. In addition, they also proposed a rate adaption scheme based on the MAC queue, the client buffer, and the estimated transmission rate.

Because DASH does not define how the client selects the bit rate of the next segment, many adaptive bit rate (ABR) algorithms have been proposed to select the next bit rate [8,9,10,11,12]. These algorithms determine the next bit rate based on parameters such as the TCP throughput estimated by the application layer [8], current playback buffer level [9, 10], or both [11, 12]. The main goal of these ABR algorithms is to provide the clients with high QoE. In order to provide high QoE, ABR algorithms must meet various conditions such as prevention of rebuffering events, minimizing video start delays, maximizing video quality, minimizing the number of video quality switching events, and smoothly changing the video quality [13,14,15,16]. However, all of these requirements cannot be met because they have trade-offs with each other [17]. Therefore, the goal of ABR algorithms is to provide clients with high QoE by appropriately meeting each condition to the extent possible.

One of the recently proposed ABR schemes, FDASH [11] is a fuzzy logic-based bit rate determination technique. The fuzzy logic controller (FLC) of FDASH receives the current playback buffer level and the differential of the client buffer level. Then, the FLC performs fuzzification, fuzzy rule-based inference, and de-fuzzification to create output f, which represents the increase/decrease factor of the segment bit rate. FDASH is better in terms of average video quality and the number of video quality switching compared to other existing ABR schemes. It, however, is not explained why the fuzzy parameters used for fuzzification and de-fuzzification in the FLC are set. It is therefore not clear whether these set fuzzy parameters are optimal.

In this paper, we propose a new ABR scheme using adaptive network-based fuzzy inference system (ANFIS) [18], one of the neuro-fuzzy system structures. The neuro-fuzzy system is a hybrid intelligence system that combines a fuzzy system and an artificial intelligence network. It can find optimal fuzzy membership functions via neural network learning using a training data set [19]. The proposed scheme uses the video streaming data (training data) to provide high QoE to DASH clients. The training data are extracted from streaming cases that provide high QoE to DASH clients and provides a basis for setting up the fuzzy parameters of ANFIS. With the trained fuzzy parameters, the proposed scheme can improve the performance of the video streaming service by selecting the appropriate bit rate of the next segment.

Finally, we show that the proposed scheme provides higher QoE performance compared to the existing schemes in the time-varying wireless network environment using NS-3 simulation. In this environment, the proposed scheme shows that the number of video quality switching is similar to FDASH, that the average segment bit rate is about 25% higher than that in FDASH, and that the QoE is increased by nearly 24% compared to FDASH.

The rest of the paper is organized as follows: Section 2 describes ANFIS structure, and Section 3 presents a new ABR scheme using ANFIS. In Section 4, we verify the performance of the proposed method through simulation, and we finally conclude the paper in Section 5.

2 ANFIS structure

The neuro-fuzzy system is a combination of a fuzzy system and a neural network. The nodes in the neuro-fuzzy system operate as FLC. It also automatically learns the appropriate fuzzy parameters according to the training data provided [19].

ANFIS [18] is one of the structures of this neuro-fuzzy system, which was designed to functionally resemble the Sugeno-type fuzzy model [20]. As shown in Fig. 1, ANFIS consists of five layers. The ANFIS shown in Fig. 1 contains the single output z with two input values x, y, and the operation is as follows. First, for the input values x and y, the first layer performs fuzzification via a fuzzy membership function. In fuzzification, the output expression varies depending on the fuzzy membership function selected. For example, the output of the first layer for the ith node is expressed by Eq. (1) if the fuzzy membership function is selected by the Gaussian membership function and by Eq. (2) if the fuzzy membership function is selected by the bell membership function.

$$ {\mu}_i(x)=\exp \left(-\frac{{\left(x-{c}_i\right)}^2}{2{\sigma}_i^2}\right),i=1,2,3,4 $$

(1)

$$ {\mu}_i(x)=\frac{1}{1+{\left|\frac{x-{c}_i}{a_i}\right|}^{2{b}_i}},i=1,2,3,4 $$

(2)

where {c_i, σ_i} in Eq. (1) and {a_i, b_i, c_i} in Eq. (2) are referred to as premise parameters and are learned using training data. As the value of these parameters changes, the outputs of the fuzzy membership functions vary accordingly.

Fig. 1

The ANFIS architecture

The nodes in the second layer correspond to fuzzy rules. The output of the second layer is the product of all the input signals connected to each node. For example, in Fig. 1, the output of the first node is expressed by Eq. (3):

$$ {w}_1={\mu}_1\times {\mu}_3 $$

(3)

The third layer performs normalization of the output of the second layer. The ith node of the third layer calculates the ratio of the ith output value of the second layer to the sum of all output values of the second layer. It is expressed by Eq. (4):

$$ {\overline{w}}_i=\frac{w_i}{\sum_{j=1}^n{w}_j},i=1,2 $$

(4)

where n is the number of nodes in the second layer.

The fourth layer performs de-fuzzification via an output fuzzy membership function. The output of the fourth layer for the ith node is determined as

$$ {\overline{w}}_l{f}_i={\overline{w}}_l\left({p}_ix+{q}_iy+{r}_i\right),i=1,2 $$

(5)

where {p_i, q_i, r_i} are referred to as consequent parameters and are learned using training data. The Sugeno-type fuzzy model uses a linear function or a constant function as an output fuzzy membership function. The output membership function of Eq. (5) is a linear function, but when p_i and q_i are zero, it becomes a constant function.

Finally, the node in the fifth layer adds all the output values of the fourth layer to generate the final output z, as shown in Eq. (6):

$$ z=\sum \limits_i{\overline{w}}_i{f}_i=\frac{\sum_i\;{w}_i{f}_i}{\sum_i\;{w}_i} $$

(6)

The hybrid learning process [18], which combines the least square method and the gradient descent method, is used to train the premise parameters and the consequent parameters. The learning process of the hybrid learning process is summarized in Table 1.

Table 1 Hybrid learning process

In the forward path, the hybrid learning process fixes the premise parameters and trains the consequent parameters by using the least square method. Conversely, in the backward path, the process fixes the consequent parameters and trains the premise parameters by using the gradient descent method. The process iterates the least square and gradient descent methods and is terminated when the error between the output value of the training data and the output value of the ANFIS meets the specified value or when the process reaches the specified iteration number.

3 Methods

In this section, we propose a new ABR scheme that provides high QoE to clients in DASH environment using the ANFIS structure described in Section 2. In the FDASH scheme mentioned in Section 1, the current playback buffer level and the differential of the client buffer level are used as the inputs of the FLC. However, the proposed method determines the next bit rate based on the current playback buffer level and the estimated download time, which is advantageous because the input values of the FLC of FDASH do not consider network throughput. A detailed explanation is as follows.

3.1 Proposed ABR scheme

The ANFIS structure of the proposed ABR scheme is shown in Fig. 2, and its corresponding properties are summarized in Table 2. The proposed structure contains two inputs, x₁ and x₂, and output y. The first input parameter x₁ is the current playback buffer level, and the second input parameter x₂ is the estimated download time. The estimated download time T_next is calculated using the size of the next segment S_next and the network throughput R as expressed in Eq. (7):

$$ {T}_{\mathrm{next}}=\frac{S_{\mathrm{next}}}{R} $$

(7)

$$ R=\frac{S_{\mathrm{last}}}{T_{\mathrm{last}}} $$

(8)

Fig. 2

The ANFIS structure of the proposed ABR scheme

Table 2 The ANFIS properties of the proposed scheme

where S_last and T_last represent the size of the last segment and the actual download time for the last segment, respectively.

Because the network throughput fluctuates significantly over time, calculating the network throughput with only the size of the last segment and the download time can lead to substantial errors. Therefore, ABR schemes using the network throughput usually use average network throughput for a period [8]. However, this causes a major error in the calculated throughput when non-stationary throughput changes (e.g., level shift) occur. Therefore, the previous throughput is ignored in this case [21]. Similarly, the proposed scheme uses average download time as the second input parameter as expressed in Eq. (9):

$$ {x}_2:{T}_i={\left\{{}_{\frac{T_i-1+{T}_{\mathrm{next}}}{2},}^{T_{\mathrm{next}}}\right.}_{\mathrm{otherwise}}^{\left|{T}_{\mathrm{next}}-{T}_i-1\right|\kern0.5em >D} $$

(9)

T_i is the predicted download time for the ith segment and is calculated as the average of the last predicted download time T_i − 1 and T_next. However, if the difference between T_i − 1 and T_next is greater than a segment playback duration D, it indicates a non-stationary throughput change, meaning the calculation with T_i − 1 can cause a significant error. Therefore, in this situation, T_i − ₁ is ignored and only T_next is used in the calculation.

The two inputs are passed to the first layer and are computed using a Gaussian membership function as shown in Table 2 and Fig. 2. In this regard, since the range of the training data may be smaller than the range of the actual input values, the trained membership function may not provide the correct value for all the input values in the form of a normal Gaussian membership function. Therefore, the Gaussian membership function corresponding to A₁, B₁, A₃, and B₃ is transformed as follows:

$$ {\mu}_i(x)=\left\{{}_{1,\kern5.3em x<c}^{\exp\;\left(-\frac{{\left(x-{c}_i\right)}^2}{2{\sigma}_i^2}\right),\kern1em x\ge c},i=1,\right. $$

(10)

$$ {\mu}_i(x)=\left\{{}_{1,\kern5.3em x>c}^{\exp\;\left(-\frac{{\left(x-{c}_i\right)}^2}{2{\sigma}_i^2}\right),\kern1em x\le c},i=3,\right. $$

(11)

The final output y is obtained after the calculation in each layer of the proposed structure. The output y represents the increase/decrease factor of the segment bit rate. Therefore, the next bit rate is the bit rate closest to Q_last + y, where Q_last represents the bit rate of last segment.

When the client requests the first segment, the client cannot calculate R, because it does not know S_last and T_last. Moreover, a prolonged download time of the first segment leads to a longer start time for the client watching the video, which has a negative effect on the client’s QoE [15]. Therefore, the proposed scheme requests the lowest bit rate segment as the first segment, allowing the client to begin watching the video as quickly as possible.

3.2 Training data

The proposed scheme requires training data for training. The data that provide high satisfaction to clients in actual streaming services can be used as training data. However, if this data cannot be obtained, arbitrary training data should be created. To create the training data, we have set arbitrary situations using NS-3. First, we have set up two types of point-to-point link network environments, as shown in Fig. 3. Then, a client that knows the actual link capacity of the network requests a segment of the bit rate suitable for the actual link capacity. In this process, the data corresponding to x₁, x₂, and y of the proposed structure are obtained.

Fig. 3

Training data extraction environment. a Long-term changes in point-to-point link network. b Periodic short-term changes in point-to-point link network

In the simulation, the data are collected from a variety of link networks with different link capacity variations, different change times of the link capacity, and different initial link capacities in order to obtain different types of data corresponding to all the fuzzy rules of the proposed structure. After collecting the training data, the training process is iterated until the error between the output value of the training data and the output value of the proposed structure converges to the specified value.

4 Results and discussion

In this section, we evaluate the proposed scheme by analyzing the NS-3 simulation results compared to those of FDASH [11] and segment aware rate adaptation algorithm (SARA) [10]. In the simulations, we used the Big Buck Bunny [22] with a segment playback duration D of 4 s and available bit rates of 45, 89, 129, 177, 218, 256, 323, 378, 509, 578, 783, 1000, 1200, 1500, 2100, 2400, 2900, 3300, 3600, and 3900 kbps. In addition, the client’s maximum buffer size was 100 s. The target buffer level of FDASH was set to 35 s, and in the SARA, I was set to 8 s, B_α was set to 30 s, and B_β was set to 30 s.

To evaluate the performance of each scheme, the following QoE matrix [12], which is a weighted combination of video quality, quality variations, rebuffering time, and startup delay, was used:

$$ \mathrm{QoE}={\sum}_{k=1}^Kq\left({R}_k\right)-\lambda {\sum}_{k=1}^{K-1}\left|q\left({R}_{k+1}\right)-q\left({R}_k\right)\right|-\mu {\sum}_{k=1}^K{T}_{\mathrm{reb}}-{\mu}_s{T}_s $$

(12)

The first term in Eq. (12) represents the total video quality. K is the number of downloaded segments, and R_k is the bit rate of the kth downloaded segment. q(·) is a function which maps R_k to the video bit rate perceived by the client. The second term represents the total magnitude of the changes in video quality. T_reb in the third term is the rebuffering time. T_s in the final term is the startup delay. λ, μ, and μ_s are non-negative weighting parameters corresponding to video quality variations, rebuffering time, and startup delay, respectively. The value of each parameter determines how much each term affects QoE.

In this simulation, we assume q(R_k) = R_k, and use the weights λ = 1, μ = μ_s = 3000 [12]. The simulation was performed in two Wi-Fi environments and two LTE environments. In each environment, all DASH clients started watching videos using FDASH, SARA, or the proposed scheme as ABR scheme at the same time. Each simulation was performed 10 times for 300 s. Since the number of segments downloaded by each scheme over the period of 300 s was different, we set K as 75, which was the result of 300 s divided by D.

4.1 Training results

The training results of the proposed scheme are shown prior to the simulation results, as illustrated in Figs. 4 and 5. To fit the QoE matrix, we created the training data under the following two conditions: (1) selection of the highest bit rate without any expected rebuffering event and (2) reducing the amount of quality variations. Figure 4 represents the fuzzy membership function based on the trained premise parameters. Figure 5 represents a comparison of the streaming results applying trained ANFIS to a few parts of the training data. In Fig. 5, the orange line indicates the link capacity set in the point-to-point link network, the green line indicates the bit rate training data, and the red line indicates the bit rate determined using the trained ANFIS. Although the red and green lines are not perfectly matched, they exhibit similar shapes and request similar bit rate values.

Fig. 4

The trained input fuzzy membership functions. a Current buffer level. b Predicted download time

Fig. 5

Comparison of training data and trained data. a Comparison under long-term change point-to-point link network. b Comparison under short-term change point-to-point link network

4.2 Simulation results

The simulation results in the two Wi-Fi environments are shown in Figs. 6 and 7 in order. The first Wi-Fi environment contains a DASH server, a single DASH client, and five background TCP traffic. The second Wi-Fi environment contains a DASH server and six DASH clients, two for each ABR scheme. In the figures, (a) shows the segment bit rate requested by each algorithm over time and (b) shows the client’s playback buffer level.

Fig. 6

Simulation results in first Wi-Fi environment. a The segment bit rate requested by client over time. b The client’s playback buffer level

Fig. 7

Simulation results in second Wi-Fi environment. a The segment bit rate requested by client over time. b The client’s playback buffer level

In those simulation results, FDASH requests many segments at a low bit rate until the target buffer level is reached. After reaching the target buffer level, the bit rate of the next segment gradually increases. In Fig. 6, the FDASH client requests the segment of 1500 kbps at 73 s for the first time, which is the 34th request, while the proposed scheme requests a segment of 1500 kbps at 18 s for the first time, which is the 7th request. In other words, the FDASH client has to wait 136 s (34 × D) to view the video at 1500 kbps. The SARA increases the client buffer level quickly by requesting many low bit rate segments and requests a high bit rate segment when there is a sufficient buffer level. However, because it requests an excessively high bit rate segment, the download time for the segment is too long and the client buffer is quickly consumed. This causes an abrupt decrease in the segment bit rate. Moreover, since the SARA method increases the bit rate step by step from the low bit rate following an abrupt reduction in the bit rate, the bit rate changes frequently. In addition, although FDASH and SARA show small variations when the bit rate increases, they generally show large variations when the bit rate decreases. On the other hand, the proposed scheme shows a small variation in both the increases and decreases in bit rate, according to the characteristics of the training data.

LTE environments contain a single base station and 50 physical resource blocks (PRBs) (10 MHz bandwidth) used as radio resources. The base station uses a proportional fair resource allocation scheme as the resource allocation scheme. In addition, the extended typical urban (ETU) fading model and log-distance propagation loss model are used.

The first LTE environment contains ten background TCP traffic and three DASH clients, one for each ABR scheme. All DASH clients are fixed at the same location and all TCP traffic move randomly at a speed of 3 m/s. The second LTE environment contains a single DASH client and 20 background TCP traffic. Both a DASH client and all TCP traffic randomly move at a speed of 3 m/s. The simulation results under each LTE environment are shown in Figs. 8 and 9 in order.

Fig. 8

Simulation results in first LTE environment. a The segment bit rate requested by client over time. b The client’s playback buffer level

Fig. 9

Simulation results in second LTE environment. a The segment bit rate requested by client over time. b The client’s playback buffer level

In the figures, FDASH shows a slow increase in bit rate and stably maintains the appropriate bit rate after reaching it. On the other hand, SARA does not maintain an appropriate bit rate even though it reaches a high bit rate quickly. The proposed scheme quickly reaches a high bit rate and maintains this bit rate.

To sum up the results, FDASH maintains the appropriate bit rate and shows a low number of bit rate changes. However, it requests a low bit rate segment until the client buffer reaches the target buffer level, so clients have to wait a long time to watch high-quality video. SARA reaches a high bit rate faster than other methods. However, it consumes client buffers quickly by requiring segments with an excessively high bit rate. The proposed scheme reaches a high bit rate faster than FDASH, and it maintains the bit rate longer than SARA after reaching the bit rate. In addition, it usually shows a small quality variation.

4.3 QoE evaluation

We compared the performances of each element in Eq. (12). As all the schemes in our simulation request the lowest bit rate segment as the first segment, we did not compare the startup delays. The average results for the elements under each environment are shown in Figs. 10, 11, 12, and 13, in order.

Fig. 10

Performance of each element of QoE in the first Wi-Fi environment

Fig. 11

Performance of each element of QoE in the second Wi-Fi environment

Fig. 12

Performance of each element of QoE in the first LTE environment

Fig. 13

Performance of each element of QoE in the second LTE environment

In Figs. 10 and 11, which are the results in the two Wi-Fi environments, FDASH shows the lowest average bit rate, although its number of bit rate changes is smallest. SARA shows a higher average bit rate than that of FDASH. However, it shows the largest number of bit rate changes, and its variation in bit rate is also the largest. The proposed scheme shows a similar number of changes as FDASH. In addition, the proposed scheme shows the smallest variation and the highest average bit rate of all the schemes evaluated. Thus, it is clearly shown that the proposed method appropriately meets each element of a given QoE matrix.

Figures 12 and 13 represent the results in LTE environments. In these figures, SARA shows the highest average bit rate. However, it also shows the highest number of bit rate changes. FDASH shows the lowest average bit rate, although it shows the lowest number of changes and the lowest average variation because of the reduced occurrence of bit rate reduction. In other words, FDASH and SARA show some advantages, but they also show clear drawbacks. In contrast to those two schemes, the proposed method shows an average bit rate similar to SARA and a number of changes similar to FDASH. Therefore, in LTE environments, the proposed method appropriately meets each element, as in the simulation of the Wi-Fi environment.

Finally, we calculated the QoE for all the simulations and normalized the QoE for the proposed scheme as shown in Fig. 14. Compared with SARA and FDASH in each environment, the proposed scheme provides approximately 22% and 24% better QoE to clients, respectively.

Fig. 14

Comparison of QoE in each simulation

Our simulation results show that the proposed scheme leads to better performance than the existing ABR scheme by learning through the training data appropriate to the QoE matrix. In addition, the application of the proposed scheme to actual service is expected to improve the performance of the scheme by directly collecting and using the real video playback data that the client has satisfactorily watched.

4.4 Computational complexity

In order to compare the computational complexity of each ABR scheme, we measured the operating time of each ABR scheme. The operating time is the time to calculate the next bit rate to request a segment. Table 3 represents measured operating time. In Table 3, SARA takes the least amount of time and FDASH takes time similar to SARA. The proposed scheme operates similarly with FDASH, but it takes more time than FDASH since the input membership functions of the proposed scheme perform more complex calculations than the functions of FDASH. However, the proposed scheme can provide better QoE to DASH clients by using the more complex membership functions.

Table 3 Operation time of ABR schemes

5 Conclusion

In this paper, we proposed a new ABR scheme using ANFIS, which is one of the neuro-fuzzy system structures. While the fuzzy membership functions of the FDASH scheme lack the basis for the function setting, the proposed scheme learns the fuzzy membership functions via the training data and provides the basis. In our simulation, we trained the fuzzy membership functions of the proposed scheme with training data optimized for the given QoE matrix and demonstrated that the proposed scheme using the trained fuzzy membership functions provided better QoE than the existing schemes. In addition, it is expected that the proposed scheme will perform better by using the data that has already satisfied the client when implemented in an actual video streaming service.