Cost-efficient enactment of stream processing topologies

The continuous increase of unbound streaming data poses several challenges to established data stream processing engines. One of the most important challenges is the cost-efficient enactment of stream processing topologies under changing data volume. These data volume pose different loads to stream processing systems whose resource provisioning needs to be continuously updated at runtime. First approaches already allow for resource provisioning on the level of virtual machines, but this only allows for coarse resource provisioning strategies. Based on current advances and benefits for containerized software systems, we have designed a cost-efficient resource provisioning approach and integrated it into the runtime of the Vienna Ecosystem for Elastic Stream Processing. Our resource provisioning approach aims to maximize the resource usage for virtual machines obtained from cloud providers. This strategy only releases processing capabilities at the end of the VM's minimal leasing duration instead of releasing them eagerly as soon as possible as it is the case for threshold-based approaches. This strategy allows us to improve the service level agreement compliance by up to 25% and a reduction for the operational cost of up to 36%. ABSTRACT The continuous increase of unbound streaming data poses several challenges to established data stream processing engines. One of the most important challenges is the cost-efﬁcient enactment of stream processing topologies under changing data volume. These data volume pose different loads to stream processing systems whose resource provisioning needs to be continuously updated at runtime. First approaches already allow for resource provisioning on the level of virtual machines, but this only allows for coarse resource provisioning strategies. Based on current advances and beneﬁts for containerized software systems, we have designed a cost-efﬁcient resource provisioning approach and integrated it into the runtime of the Vienna Ecosystem for Elastic Stream Processing. Our resource provisioning approach aims to maximize the resource usage for virtual machines obtained from cloud providers. This strategy only releases processing capabilities at the end of the VM’s minimal leasing duration instead of releasing them eagerly as soon as possible as it is the case for threshold-based approaches. This strategy allows us to improve the service level agreement compliance by up to 25% and a reduction for the operational cost of up to 36%.


22
Due to the transition towards a data-centric society, today's stream processing engines (SPEs) need to deal 23 with a continuous increase of unbound streaming data regarding volume, variety, and velocity (McAfee 24 et al., 2012). Currently, this growth in data is mainly driven by the rise of the Internet of Things (IoT) 1 . 25 Sensors, which represent a vital part of the IoT, emit a huge volume of streaming data that needs to be 26 processed to provide additional value to users or to trigger actions for IoT devices or other services, e.g., 27 to handle user notifications. Furthermore, many scenarios call for data processing in near real-time, which In the following paragraphs, we describe a data stream processing scenario from our EU H2020 project 97 Cloud-based Rapid Elastic Manufacturing (CREMA) (Schulte et al., 2014). Figure 1 shows a stream 98 processing topology, which is composed of nine different stream processing operator types (O1 -O9) that 99 process the data originating from three different sources (S1, S2, S3). Each of the operator types performs 100 a dedicated operation to transform the raw data from manufacturing machines into value-added and human-101 readable information. The information from the data sources is used to monitor three different aspects, 102 like the availability of the manufacturing machines or the machine temperature to avoid overheating of 103 the machines and assess their Overall Equipment Effectiveness (OEE). In this scenario, we have two 104 different types of data sources. The first type of data source are sensors, i.e., S1 and S3, which emit 105 machine-readable data and can be directly accessed via an API. The second type of data, e.g., S2, is 106 a video feed, which scans a display of the manufacturing machines because some information is not 107 directly accessible via an API. This information needs additional preprocessing to transform the data into 108 machine-readable data. 109 The Availability Sensor (S1) emits the current status, i.e., available, defect or planned downtime, of the 110 manufacturing machine every two seconds. This information is then filtered by the Filter Availability (O2) 111 operator, which generates warnings for each new downtime incident of a specific manufacturing machine.

112
The warning is then forwarded to the Inform User (O8) operator, which informs a human supervisor of 113 the machines.

114
The second data source is the Production Data (S2), which is obtained by a video stream, i.e., an 115 image taken every ten seconds. This image contains different production-related information, such as forwarded to the Generate Report (O9) operator, which generates a PDF-report every minute. This report 121 aggregates the information of all monitored machines and is forwarded once every minute to the Inform 122 User (O8) operator. 123 The Temperature Sensor (S3) emits the temperature twice every second. This information is processed 124 by the Monitor Temperature (O6) operator, which triggers a warning whenever the temperature exceeds a 125 predefined threshold. This warning is then also forwarded to the Inform User (O8) operator to inform the 126 human supervisor.

127
Due to the different levels of complexity of the operations, each of these operator types has different 128 computational resource requirements, e.g., CPU or memory. Some of the operators, e.g., the Parse and 129 Distribute Data operator type, require more resources for processing one data item than others, like the   Filter Availability. Besides the computational requirements, each operator type is also assigned with 131 specific Service Level Objectives (SLOs), like the maximal processing duration of one single data item.

132
These SLOs are monitored, and whenever one operator type threatens to violate the imposed SLA, the 133 system needs to provide more computational resources for data processing. To enact the stream processing topology from the motivational scenario, it is required to instantiate it on an 136 SPE. For our work at hand, we are extending the VISP ecosystem, which was introduced in our previous 137 work (Hochreiner et al., 2016b). VISP represents an SPE, which is capable of provisioning computational 138 resources on demand to adapt to the incoming load from data sources. VISP is composed of different 139 components to cover the whole lifecycle of the stream processing topology enactment. Figure 2 shows a 140 subset of these components, which are relevant for enacting the topology. For a detailed description of the 141 components, please refer to our previous work (Hochreiner et al., 2016b) or to the online documentation 142 of the VISP Ecosystem 5 , which is available under the Apache 2.0 License. 143 The primary task of the SPE, i.e., VISP Runtime, is to process data originating from data sources (on 144 the left side of the figure) to obtain value added data for users (on the right side of the figure) based on 145 the Topology Definition. The actual data processing is conducted by Operators, which are deployed on 146 computational resources, e.g., VMs, provided by an Infrastructure as a Service (IaaS) environment. Each 147 operator type is instantiated from dedicated operator images hosted on an external operator repository.

148
To instantiate a specific operator instance on any host for the first time, the operator image needs to 149 be downloaded from the registry, which takes a certain amount of time, depending on the size of the

166
During the enactment, the stream processing operators need to deal with streaming data from a varying 167 amount of manufacturing machines, as shown in Figure 3 at the bottom. This varying data volume requires 168 the SPE to adapt its processing capabilities, i.e., the number of operator instances for specific operator 169 types, which are hosted on an arbitrary amount of hosts, e.g., H1 -H4 in Figure 3, on demand to comply 170 with the SLAs. Nevertheless, the SPE aims at minimizing the needed number of hosts, since each host 171 amounts for additional cost, by using an optimal deployment.

172
The enactment of our motivational scenario is partitioned into different stages, with a varying number 173 of running manufacturing machines in each stage. At the beginning of Stage 1, each operator is deployed 174 once across the two hosts H1 -H2. Since the volume of streaming data increases after some time, the 175 SPE needs to adapt the processing capabilities by deploying replicas of the operator types O1, O2 and O6 176 in Stage 2. These operator instances are hosted on a new host H3 because the two already existing hosts 177 cannot cope with the additional operator instances. Because the amount of data increases again in Stage 3, 178 the SPE needs to replicate further operators to comply with the SLAs. Although the second replication of 179 the operator type O1 is feasible on the currently available resources, the SPE is required to lease a new 180 host for the additional operator instances of types O3, O4, O5, and O9.

181
At the end of Stage 3, H1 -H2 meet the end of their BTU. Therefore, the SPE evaluates whether 182 some of the replicated operators can be removed again without violating the SLAs. Because the amount 183 of data is decreasing after Stage 3, the system can remove (O1, O3, O4, and O5) or migrate (O2) some of 184 the operator instances to other hosts. This leads to the situation that no operator instances are running on 185 host H1 at the end of its BTU and the SPE can accordingly release H1, while host H2 needs to be leased 186 for another BTU.

188
Based on our motivational scenario, we have identified several requirements which need to be addressed 189 by the optimization approach.

SLA Compliance
The first requirement is SLA compliance in terms of maximum processing duration, 191 for data that is processed by the stream processing topology. This compliance is the overall goal that 192 needs to be met, regardless of the actual incoming data rate.

Computer Science
Optimization Efficiency The optimization efficiency requirement can be split into two different aspects.

197
The first aspect is the solution of the optimization problem presented in Section 3. Because this opti-198 mization problem is NP-hard (see Section 3.2), it is required to devise heuristics to achieve a time-and 199 resource-efficient optimization approach. The second aspect is that the optimization needs to minimize 200 the number of reconfigurations, e.g., scaling operations, for the stream processing topology because each 201 reconfiguration activity has a negative performance impact on the data processing capabilities.  image, whose identifier is identical to the name of the operator type.

222
Besides the fundamental operator type attributes for instantiating operators, there is also a set of 223 attributes, which is used to ensure the SLA compliance for data processing. Each operator type is assigned 224 with an estimated data processing duration o slo , that represents the time to process one data item and 225 pass it on to the following operator type according to the stream processing topology. The o slo value is 226 recorded in an optimal processing scenario, where no data item needs to be queued for data processing.

227
Since the SLO o slo only presents the expected processing duration, we also denote the actual processing 228 duration for each operator o d and the amount of data items o queue that are queued for a particular operator 229 type for processing.

230
In addition to the current o d , the system model also considers previous processing durations. Here we  we also denote the current incoming amount of data items as DR.  deployment, it is not feasible to rely on the solution of this problem due to its complexity. To define 250 the complex nature of this problem, we are going to provide a reduction to an unbounded knapsack 251 problem (Andonov et al., 2000), which is known to be NP-hard.

Definition of Knapsack Problem
The unbounded knapsack problem assumes a knapsack, whose 253 weight capacity is bounded by a maximum capacity of C and a set of artifacts A. Each of these artifacts a 254 is assigned with a specific weight a w > 0 as well as a specific value a v > 0 and can be placed an arbitrary 255 amount of times in the knapsack. The goal is to find a set A1 of items, where ∑ a∈A a w ≤ C and ∑ a∈A a v is 256 maximized.

NP-Hardness of the Optimization Problem
For our reduction, we assume a specific instance of our 258 optimization problem. For this specific instance, we assume that the number of hosts is fixed and that 259 each of the operators has the same memory requirements o memory . Furthermore, we define the value of a 260 specific operator by the amount of data items o queue that are queued for a specific operator type, i.e., the 261 more items need to be processed, the higher is the value for instantiating a specific operator.

267
Because a specific instance of our optimization problem can be formulated as a knapsack problem, 268 we can conclude that our optimization problem is also NP-hard. This concludes that there is no known The overall goal our optimization approach is to minimize the cost for computational resources and

301
To minimize the cost of computational resources, the optimization approach aims at using the leased 302 resources as efficient as possible. This means that the SPE uses all paid resources until the end of their 303 BTUs and evaluates shortly before, i.e., within the last 5% of the BTU, whether a host needs to be leased 304 for another BTU, i.e., the resources are still required, or if the host can be released again.

305
To release hosts, as shown in Figure 5, all operator instances running on the designated host which is 306 targeted to be shut down, need to be either released or migrated to other hosts. This releasing procedure 307 consists of three phases. The first phase is a simulation phase, where the optimization approach creates cannot be shut down, the procedure simulates whether they can be migrated to other hosts. This simulation 317 uses the upscaling procedure for operator types, as described in Section 4.1. The only difference is that 318 the host which is targeted to be shut down, is omitted as a suitable host.

319
If the simulation renders no feasible downscaling plan, the host is leased for another BTU and the 320 downscaling procedure is finished. In case there is a downscaling plan, the operators are released in phase 321 two and if any migration is required, the upscaling procedure for operator types is triggered based on 322 the simulation in phase three. When all operator instances are successfully removed (scaled down or 323 migrated), the shut down of the host is initialized. In the unlikely event that the downscaling plan could 324 not be executed, i.e., the operator instance migrations fail, the host also needs to be leased for another 325 BTU.   or potential operator types, which can be scaled down (Algorithm 3). Although these algorithms do not 335 represent the core functionality of the resource provisioning algorithm, they are still essential to identify 336 required upscaling operations and choose the optimal degree of parallelism per operator whereas the 337 overall cost-reduction and reconfiguration is represented by the downscaling procedure shown in Figure 5.

338
The remainder of this section discusses the structure and rationale of these three algorithms in detail.

339
The Upscaling Algorithm as listed in Algorithm 1 is used to evaluate whether any operator needs to be 340 scaled up. This algorithm is executed on a regular basis for each operator type o and either returns 0, if the 341 current stream processing capabilities are enough to comply with the SLAs, or 1 if the operator type needs 342 to be scaled up. Therefore, this algorithm considers, on the one hand, the current processing duration 343 of the operator (Line 2) and, on the other hand, the trend of the previous processing durations. For the 344 trend prediction, we apply a simple linear regression for the last N observations, based on the linear least 345 squares estimator (Lines 5 -9). If the current duration o d or the predicted duration is higher than the SLO 346 o slo , we consider the operator type to be scaled up (Line 10). Before we trigger the upscaling operation, 347 we apply an additional check if the upscaling operation is required.

348
The stream processing topology may retrieve short-term data volume peaks, e.g., due to short network 349 disruptions. These peaks would not require any additional computational resources, because they would 350 be dealt with after a short time with the already available processing capabilities. Nevertheless, the 351 upscaling algorithm would trigger the upscaling procedure, because it would detect the processing delay.

352
Therefore, the algorithm also considers the current load of data items o queue before scaling up by checking 353 whether the amount of queued items for processing exceeds a scalingT hreshold (Lines 13 -16).

354
Algorithm 2, i.e., the Host Selection Algorithm, is used to rank all currently leased hosts according to 355 their suitability to host a new operator instance of a particular operator type. Therefore, the algorithm   return 0 20: end function resources, i.e., h cpu * and h memory * , which can be used by the operators to cover short-term data volume 372 peaks without any reconfigurations required. This aspect is covered by dividing the di f f erence value 373 by the f easibility value to prefer those hosts which are least used (Line 9). Last, we also consider the 374 deployment time aspect for a particular operator type. Here, we prefer those hosts, which have already 375 the operator image cached. While such operator images may be rather small for SPEs which operate 376 on a process or thread level, like Apache Storm, these images can reach up to 100 MB for containerized 377 operators. This requires some time to download the operator images from an external repository. In order 378 to distinguish hosts, which have a cached copy of the operator image from those hosts that do not have a 379 cached copy of the operator image, we multiply the suitability with a constant factor CF to create two 380 different groups of hosts for the overall selection (Lines 10 -12). For this constant factor, we recommend 381 to use the value 0.01 which was also used in the remainder of our work. The value 0.01 was chosen to 382 clearly distinguish these two groups, since the actual suitability values are always in the range of 0 to 1 383 based on the structure of the algorithm. Each of these group maintains their resource-based ordering, but 384 we prioritize those hosts that provide a faster startup time due to the cached image, i.e., the group with 385 lower values. The result of this algorithm is either a negative value for a host, i.e., the host can run the 386 new operator instance, or a positive value, whereas the lowest value among several hosts shows the best 387 suitability.

388
Algorithm 3, i.e., the Operator Selection Algorithm, is used to select operator types which can be 389 scaled down without violating the SLOs. Therefore, this algorithm considers several static as well as 390 runtime aspects of the operator types. The goal of the algorithm is to obtain a value which describes the 391 suitability of a particular operator type to be scaled down. Whenever the value is negative, the operator 392 type must not be scaled down, i.e., all operator instances for this type are required to fulfill the SLO. 393 First, the algorithm ensures that there is at least one operator instance for the given operator type 394 (Lines 2 -4). Second, the function considers the amount of all currently running instances for the specific 395 operator type and normalizes it to obtain a value between 0 and 1 (Line 5). This normalization is carried 396 out based on the maximal respectively minimal amount of instances for all operator types. This value 397 represents the aspect that it is better to scale down an operator type with numerous operator instances 398 because the scale down operation removes a smaller percentage of processing power compared to an 399 operator type with fewer operator instances. 400 Furthermore, we consider the SLA compliance of the particular operator. Here, we consider the actual 401 compliance for the processing duration and multiply it with the penalty cost as a weighting factor (Line 6). Since the penalty cost for the violation of a single data item is typically lower than 1, we add 1 to the 403 penalty cost P. Whenever the processing duration o d takes longer than the SLO o slo , the delay value will 404 be less than one, but when there is any delay, the delay value can become arbitrarily high. The next value 405 for consideration is the relative amount of scaling operations (both up and down) in contrast to the entire 406 scaling operations (Line 7). Here, we penalize previous scaling operations because we want to avoid any 407 oscillating effects, i.e., multiple up-and downscaling operations for a specific operator. The last factor is 408 the queueLoad. In the course of our evaluations, we have seen that the algorithm may take a long time to 409 recover after a load peak, i.e., release obsolete operator instances as soon as the data is processed. This 410 can be observed when the SPE is confronted with a massive data spike followed by a small data volume 411 for some time. For this scenario, the heuristic discourages any downscaling operation due to the delay 412 factor, which may be high due to the delayed processing of the data spike. To resolve this shortcoming, 413 we introduce the queueLoad factor QL, which encourages the downscaling of an operator type, as soon as can exceed 1 if the data processing is delayed. Therefore, we introduce optional weights W 1,W 2,W 3, 423 and W 4 for the different aspects, whereas the default value for each of these weights is 1 to treat all 424 aspects with the same emphasis. The result is the utility value, which describes the suitability of the 425 particular operator to be scaled down, whereas a higher value suggests a better suitability (Line 13).  First, we are going to discuss the sensors which emit the data items for our topology. In this topology, we 432 consider three different sensor types, as listed in Table 1. Each of these sensor types generates a data item, 433 with a particular structure, which can be only processed by a dedicated operator type, e.g., O1 for sensor 434 type S2. Due to the different structure, the size of the data items also differs. The first and the last sensor 435 type (S1 and S3) encode the information in plain text. This results in rather small data items with a size of 436 90 to 95 Bytes. The second sensor type encodes the information with an image and is therefore much return 1 + W1 * instances + W2 * queueLoad − W3 * delay − W4 * scalings 14: end function

439
The second important implementation aspect for the topology are the operators. Each of these operator 440 types performs a specific task with specific resource requirements and specific processing durations. type. The outgoing ratio describes whether a particular operator type consumes more data items than it 462 emits, e.g., O7 combines three data items before it emits a combined one, or whether it emits more data 463 items than it receives, e.g., O1 distributes the production information to three other operator types.   (Nakajima, 1988).
Calculate Availability (O4) The Calculate Availability operator type represents the overall availability 478 of the manufacturing machine from the beginning of the production cycle, e.g., the start of the evaluation.

479
The availability is defined by the formula shown in Equation 3 (Nakajima, 1988

506
For the topology enactment, the VISP Runtime leases (and releases) an arbitrary amount of VMs, i.e.,

507
Dockerhost VMs, on the private OpenStack-based cloud at runtime. These Dockerhost VMs are used to 508 run the Operator Instances, which take care of the actual data processing as described in Section 2.2. The

509
Operator Images, which are required to run the Operator Instances, are hosted on an external service, i.e.,

510
Dockerhub 13 . Finally, the Data Provider VM is in charge of simulating the data stream from the sensors, 511 as described in Section 5.1.1.

513
For the scalingT hreshold used in Algorithm 1, we used the value 50. This value was selected to be high 514 enough to allow for minimal hardware disturbances, e.g., moving data from memory to the hard drive, but 515 low enough to react to small changes of the data volume. The concrete value was identified on a number Regarding the BTU, we make use of three different BTU durations. The first duration is 60 minutes 528 (BTU60), which is the BTU of Amazon EC2 14 . The second duration is 10 minutes (BTU10), which 529 represents the minimal BTU for the Google Compute Engine 15 and the last duration is 30 minutes 530 (BTU30), which has been selected to present a middle ground between these two commonly used BTUs.  Table 1. To speed up the evaluation, we simulate the real time data emissions shown in Table 1 561 every 480 milliseconds. This enables us on the one hand to simulate 500 real-time minutes within only 562 four minutes in the course of our evaluation and therefore also increases the load on the SPE. This also 563 results in a volume level change every four minutes.

564
The second arrival pattern has only two levels, i.e., the lowest and the highest of the first pattern,   The next set of metrics describes the SLA compliance of the stream processing application. Each 585 stream processing operator is assigned a specific processing duration which describes the processing 586 16 https://github.com/visp-streaming/dataProvider 17 https://github.com/visp-streaming/reporting duration in a constant over-provisioning scenario. Due to the changing data volume in our evaluation 587 scenarios, it is often the case that the system suffers from under-provisioning for a short time, which 588 results in longer processing durations. To assess the overall compliance of the processing durations, we 589 define three different SLA compliance levels. The first compliance level requires real-time processing 590 capabilities, and states the share of data items that are produced within the given processing duration. The 591 second level applies near-realtime requirements, which is defined by processing durations that take at 592 most twice as long as the defined processing duration, and the third level applies a relaxed strategy, which 593 means that the data items need to be processed within at most five times the stated processing duration.

594
These SLA metrics are obtained from the processing duration of the data items, which are recorded by the 595 operators. To reduce the overall monitoring overhead, we only measure the processing duration of every 596 tenth data item. Nevertheless, preliminary evaluations with other intervals, e.g., every data item or every 597 third data item have shown a similar metric reliability. This similar reliability can be explained due to the 598 fact that observing every tenth data item still yields about 20-40 performance readings/second (depending 599 on the data volume). Therefore it is save to assume that these metrics cover all effects of the SPE because 600 all other activities, e.g., spawning a new operator instance takes 5-10 seconds or leasing a new VM takes 601 about 30-60 seconds.

602
The Time To Adapt metric states the arithmetic mean duration, which is required until the delayed 603 processing for an operator type is back to real-time processing.

604
The last metrics describe the scaling operations of operator instances. Here we consider Upscaling, 605 Downscaling as well as Migration operations among different hosts.

614
The discussion of our evaluation is divided in four subsections based on the four data arrival patterns.

619
For the discussion we are going to analyze the differences between the BTU-based and the threshold-620 based approach in detail only for the stepwise data arrival pattern because this arrival pattern allows us to 621 isolate specific aspects of the BTU-based approach. Nevertheless, our evaluations shows that the overall 622 trend regarding the SLA compliance and total cost is the same for all four data arrival patterns. For the 623 other arrival patterns we only highlight specific aspects of the individual pattern and refer for all other 624 effects to the discussion of the stepwise data arrival pattern.

626
For the stepwise pattern, we can see that the overall SLA compliance is higher for the BTU-based approach 627 for all three SLA compliance scenarios as shown in Table 3. This compliance benefit ranges from 9% 628 for the BTU10 configuration in the real-time compliance scenario, up to 24% in the relaxed compliance 629 scenario for the BTU60 configuration. The SLA compliance gain can be explained due to the downscaling 630 strategy of the BTU-based approach in contrast to the on-demand one for the threshold-based approach.

631
The threshold-based approach only considers the amount of data items that are considered for processing 632 based on each operator type for the scaling decisions, which can be observed in Figure 8d. This figure   633 shows that the line for the operator instances follows the data volume very closely with a short delay 634 because the threshold-based approach can only react based on the changes of the data volume. On closer 635 inspection, one can also identify smaller increases after the downscaling phase, e.g., around minutes 40, 636 18 https://github.com/visp-streaming/PeerJ_rawData 19 https://github.com/visp-streaming/reporting  a VM coincides with a peak of the data volume, e.g., at minute 30 for the BTU30 configuration. In these 655 situations, the BTU-based approach will initialize the downscaling procedure to release a VM shortly 656 before the end of its BTU. In this specific case around minute 30 for the BTU30 scenario, the downscaling 657 procedure is successful because monitoring does not report any delays for processing based on Algorithm 3 658 and the VM is triggered to be shut down. But in the next reasoning cycle, the SPE realizes the lack of 659 processing capabilities and leases another VM to compensate the resource requirements. Although these 660 non-efficient scaling operations result in a measurable overhead as well as an SLA compliance reduction, 661 the BTU-based approach still achieves a better SLA compliance than the threshold-based approach.

662
Furthermore, it can be seen that the amount of scaling activities for the operator instances is inverse Besides the SLA compliance, we also consider the operational cost for data processing. These cost are 668 composed of the resource cost, i.e., the cost for leasing VMs and the penalty cost, which accrue for delayed 669 data processing. In Table 3, it can be seen that the resource cost for the BTU10 and BTU30 configuration  The 2-level data arrival pattern exhibits the same trend for the SLA compliance and cost for the stepwise 686 data arrival pattern as shown in Table 4. When we analyze the Figures 9a, 9b, 9c and 9d, we can also 687 see a similar scaling behavior compared to the stepwise data arrival pattern. Nevertheless, there is one 688 notable effect for the BTU60 configuration in Figure 9c. The BTU-based provisioning approach tends to 689 start more and more operator instances throughout the evaluation run. We can see that after minute 20, 690 instances over time is that the SPE is likely to have more processing capabilities than it actually needs.

695
Nevertheless, at the end of the BTU of a VM, the necessity of these processing capabilities is evaluated, 696 and for example in the BTU60 configuration, the operator instances are cut back around minute 60. After 697 a short recalibration phase between minutes 65 and 75, the SPE follows the same pattern again until 698 the resources are cut back again around minute 120. This mechanism allows the SPE to use the already 699 leased resources, i.e., no additional VMs are leased from minute 80 until 120, to achieve a high resource 700 utilization.

702
Based on the numbers of Table 5, we can see that the random walk 1 data arrival pattern follows the same 703 trend for the SLA compliance as well as total cost as the stepwise data arrival pattern. At closer inspection 704 we can see that the SLA compliance is very similar with a deviation of less than 3%. This aspect shows 705 that both the baseline as well as the BTU-based provisioning approach have similar characteristics for the 706 rather simple data arrival pattern, like the stepwise or 2-level one, as well as random ones.

707
Based on the Figures 10a, 10b, 10c and 10d, we can identify one notable difference between the 708 BTU-based and the threshold-based resource provisioning approach. While the operator instance curve 709 and the data volume curve are well aligned for the threshold-based and the BTU10 configuration, we can 710 identify a clear gap for the BTU30 in Figure 10b and especially for the BTU60 configuration (Figure 10c).

711
For the latter two configurations, the operator instance curve remains high although the data volume The numerical results in terms of the SLA compliance and total cost follow similar trends as for the 719 stepwise data arrival pattern scenario, based on the numbers in Table 6. For this data arrival pattern also 720 only the BTU10 configuration requires more cost than the threshold-based baseline for the real-time 721 scenario. All other configurations and scenarios result in lower cost than the baseline. When we analyze 722 the graphical representation of Figures 11a, 11b, 11c and 11d for the random walk 2 data arrival pattern, 723 the most prominent difference in contrast to the random walk 1 data arrival pattern is the better alignment 724 of the operator instance and data volume curves. This is due to the fact that the data volume is rising for 725 the second part of the evaluation, i.e., after minute 40, and the already paid resources can be actively used 726 for data processing instead of only serving as free backup processing capabilities.

727
Furthermore, it can be seen that the BTU-based approach requires less scaling activities between 728 minute 60 and 120 in contrast to the threshold-based approach in Figure 11d. This is again due to the lazy 729 release characteristics of the BTU-based approach, which result in a higher SLA compliance in contrast 730 to the threshold-based approach.

743
We can also observe that the compliance for real-time data processing on cloud infrastructures is 744 rather low, i.e., around 40% for the baseline and around 50%-55% for the BTU-based approach. This is 745 mainly due to the fact that cloud environments are often influenced by other services running on the same 746 physical hardware. This can result in minor data transmission or processing delays that have a severe 747 impact on the SLA compliance. Nevertheless, we can see that for the near real-time and relaxed time 748 scenarios, the SLA compliance ranges from 84% to 95% for the BTU-based approach, which meets the 749 requirements of our motivational scenario discussed in Section 2. also proposes a simple threshold-based replication mechanism. In contrast to the other already discussed 802 approaches, SEEP focuses on stateful operators and employs a dedicated fault tolerance mechanism.

803
Besides the basic replication approaches, there are also some works that optimize specific aspects 804 for the topology enactment. One of these aspects is the partitioning of data to optimize the data flow Manuscript to be reviewed Computer Science among these clouds. Furthermore, we plan to investigate the structural properties of the topology in more 838 detail, e.g., to identify critical paths or high volume operators, such as the operators O2 and O6 in our 839 topology. These insights may help us to apply different scaling priorities, especially for downscaling 840 operations to avoid oscillating effects. In addition we also plan to evaluate the impact of using the 841 individual weights W 1 −W 4 in Algorithm 3 within both private and hybrid clouds.