Distributed Stream Processing Systems (DSPS) process streaming data and produce results immediately to the stakeholders. The data that flows into the system would be processed immediately and is not stored before processing in contrast to the static systems which process stored data. Data loss or failure to process the incoming data or dropped tuples might be very hard to recover or correct. The processing has to be done without any delay or visible disturbances to the end user. The stability of the system could be disturbed due to many reasons in practical – node failures, node slow down, data fluctuations and so on. In this paper, we consider the problem of bursty input streams which causes the operators in the stream graph to slow down. The increase on input flow cannot be predicted and has to be handled very fast by introducing a minimal delay. This delay might affect the throughput and the results of the application. We propose a heuristic to handle these input fluctuations by increasing the number of copies of the operator based on an estimate of the future input rates. This partition based scaling of operator threads will improvise the system performance.
Volume 12 | 07-Special Issue
Pages: 2815-2821
DOI: 10.5373/JARDCS/V12SP7/20202422