Skip to main content
SpringerLink
Log in

Budget-deadline constrained approach for scientific workflows scheduling in a cloud environment

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

In cloud computing environments, it is a great challenge to schedule a workflow application because it is an NP-complete problem. Particularly, scheduling workflows with different Quality of Service (QoS) constraints makes the problem more complex. Several approaches have been proposed for QoS workflow scheduling, but most of them are focused on a single QoS constraint. Therefore, this paper presents a new algorithm for multi-QoS constrained workflow scheduling, cost, and time, named Budget-Deadline Constrained Workflow Scheduling (BDCWS). The algorithm builds the task optimistic available budget based on the execution cost of the task on the slowest virtual machine and the optimistic spare budget, and then builds the set of affordable virtual machines according to the task optimistic available budget to control the range of virtual machine selection, and thus effectively controls the task execution cost. Finally, a new balance factor and selection strategy are given according to the optimistic spare deadline and the optimistic spare budget, so that the execution cost and time consumption of the control task are more effective. To evaluate the proposed algorithm, we experimentally evaluated our algorithm using real-world workflow applications. The experimental results show that compared with DBWS (Deadline-Budget Workflow Scheduling) and BDAS (Budget-Deadline Aware Scheduling), the proposed algorithm has a 26.3–79.7% higher success rate. Especially when the deadline and budget are tight, the improvement is more obvious. In addition, the best cost frequency of our algorithm achieves a 98%, which is more cost-competitive than DBWS.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Chard, R., Chard, K., Bubendorfer, K., Lacinski, L., Madduri, R., Foster, I.: Cost-aware cloud provisioning. In: IEEE International Conference on E-Science 2015, pp. 136–144 (2015)

  2. Lin, W., Xu, S., He, L., Li, J.: Multi-resource scheduling and power simulation for cloud computing. Inf. Sci. 397(C), 168–186 (2017)

    Article  Google Scholar 

  3. Wu, Q., Ishikawa, F., Zhu, Q., Xia, Y., Wen, J.: Deadline-constrained cost optimization approaches for workflow scheduling in clouds. IEEE Trans. Parallel Distrib. Syst. 28(12), 3401–3412 (2017)

    Article  Google Scholar 

  4. Arabnejad, H., Barbosa, J.G.: A budget constrained scheduling algorithm for workflow applications. J. Grid Comput. 12(4), 665–679 (2014)

    Article  Google Scholar 

  5. Sakellariou, R., Zhao, H., Tsiakkouri, E., Dikaiakos, M.D.: Scheduling workflows with budget constraints. In: Integrated Research in GRID Computing. Springer, Boston, MA (2007)

  6. Zheng, W., Sakellariou, R.: Budget-deadline constrained workflow planning for admission control in market-oriented environments. In: International Workshop on Grid Economics and Business Models, pp. 105–119. Springer, Berlin (2011)

  7. Zheng, W., Sakellariou, R.: Budget-deadline constrained workflow planning for admission control. J. Grid Comput. 11(4), 633–651 (2013)

    Article  Google Scholar 

  8. Arabnejad, H., Barbosa, J.G., Prodan, R.: Low-time complexity budget–deadline constrained workflow scheduling on heterogeneous resources. Fut. Gener. Comput. Syst. 55, 29–40 (2016)

    Article  Google Scholar 

  9. Prodan, R., Wieczorek, M.: Bi-criteria scheduling of scientific grid workflows. IEEE Trans. Autom. Sci. Eng. 7(2), 364–376 (2010)

    Article  Google Scholar 

  10. Yu, J., Buyya, R., Tham, C.K.: QoS-based scheduling of workflow applications on service grids. In: Proc. of 1st IEEE International Conference on e-Science and Grid Computing 2005, pp. 5–8. IEEE CS Los Alamitos, CA (2005)

  11. Cancan, L., Weimin, Z., Zhigang, L.: Path balance based heuristics for cost optimization in workflow scheduling. J. Softw. 24(6), 1207–1221 (2013)

    Google Scholar 

  12. Chen, W., Xie, G., Li, R., Bai, Y., Fan, C., Li, K.: Efficient task scheduling for budget constrained parallel applications on heterogeneous cloud computing systems. Fut. Gener. Comput. Syst. 74(2017), 1–11 (2017)

    Google Scholar 

  13. Rodriguez, M.A., Buyya, R.: Deadline based resource provisioningand scheduling algorithm for scientific workflows on clouds. IEEE Trans. Cloud Comput. 2(2), 222–235 (2014)

    Article  Google Scholar 

  14. Arabnejad, V., Bubendorfer, K., Ng, B.: Deadline distribution strategies for scientific workflow scheduling in commercial clouds. In: IEEE ACM International Conference Utility and Cloud Computing 2016, pp. 70–78 (2016)

  15. Sahni, J., Vidyarthi, P.: A cost-effective deadline-constrained dynamic scheduling algorithm for scientific workflows in a cloud environment. IEEE Trans. Cloud Comput. 6(1), 2–18 (2018)

    Article  Google Scholar 

  16. Ghafouri, R., Movaghar, A., Mohsenzadeh, M.: A budget constrained scheduling algorithm for executing workflow application in infrastructure as a service clouds. Peer-to-Peer Netw. Appl. 12(1), 241–268 (2019)

    Article  Google Scholar 

  17. Rodriguez, M.A., Buyya, R.: Budget-driven scheduling of scientific workflows in IaaS clouds with fine-grained billing periods. Acm Trans. Auton. Adapt. Syst. 12(2), 1–22 (2017)

    Article  Google Scholar 

  18. Shen, H., Li, X.: Algorithm for the cloud service workflow schedulingwith setup time and deadline constraints. J. Commun. 36, 183–192 (2015)

    Google Scholar 

  19. Singh, V., Gupta, I., Jana, P.K.: A novel cost-efficient approach for deadline-constrained workflow scheduling by dynamic provisioning of resources. Fut. Gener. Comput. Syst. 79(2018), 95–110 (2018)

    Article  Google Scholar 

  20. Arabnejad, V., Bubendorfer, K., Ng, B.: Budget and deadline aware e-science workflow scheduling in clouds. IEEE Trans. Parallel Distrib. Syst. 30(1), 29–44 (2019)

    Article  Google Scholar 

  21. Ghasemzadeh, M., Arabnejad, H., Barbosa, J.G.: Deadline-budget constrained scheduling algorithm for scientific workflows in a cloud environment. In: international conference on principles of distributed systems 2017, pp. 1–16

  22. Wu, F., Wu, Q., Tan, Y., Li, R., Wang, W.: PCP-B 2: partial critical path budget balanced scheduling algorithms for scientific workflow applications. Fut. Gener. Comput. Syst. 60(2016), 22–34 (2016)

    Article  Google Scholar 

  23. Sun, T., Xiao, C., Xu, X.: A scheduling algorithm using sub-deadline for workflow applications under budget and deadline constrained. Cluster Comput. 22(3), 5987–5996 (2019)

    Article  Google Scholar 

  24. Wu, F., Wu, Q., Tan, Y.: Workflow scheduling in cloud: a survey. J. Supercomput. 71(9), 3373–3418 (2015)

    Article  Google Scholar 

  25. Alkhanak, E.N., Lee, S.P., Khan, S.U.R.: Cost-aware challenges for workflow scheduling approaches in cloud computing environments: taxonomy and opportunities. Fut. Gener. Comput. Syst. 50(2015), 3–21 (2015)

    Article  Google Scholar 

  26. Smanchat, S., Viriyapant, K.: Taxonomies of workflow scheduling problem and techniques in the cloud. Fut. Gener. Comput. Syst. 52(2015), 1–12 (2015)

    Google Scholar 

  27. Singh, S., Chana, I.: A survey on resource scheduling in cloud computing: issues and challenges. J. Grid Comput. 14(2), 217–264 (2016)

    Article  Google Scholar 

  28. Rodriguez, M.A., Buyya, R.: A taxonomy and survey on scheduling algorithms for scientific workflows in IaaS cloud computing environments: workflow scheduling algorithms for clouds. Concurr. Comput. Pract. Exp. 29(8), e4041 (2016)

    Article  Google Scholar 

  29. Kaur, S., Bagga, P., Hans, R., Kaur, H.: Quality of Service (QoS) Aware Workflow Scheduling (WFS) in cloud computing: a systematic review. Arab. J. Sci. Eng 44(4), 2867–2897 (2019)

    Article  Google Scholar 

  30. Ming, M., Humphrey, M.: Auto-scaling to minimize cost and meet application deadlines in cloud workflows. In: High Performance Computing, Networking, Storage & Analysis 2011, pp. 1–12

  31. Abrishami, S., Naghibzadeh, M., Epema, D.H.J.: Deadline-constrained workflow scheduling algorithms for infrastructure as a service clouds. Fut. Gener. Comput. Syst. 29(1), 158–169 (2013)

    Article  Google Scholar 

  32. Abrishami, S., Naghibzadeh, M., Epema, D.H.J.: Cost-driven scheduling of grid workflows using partial critical paths. IEEE Trans. Parallel Distrib. Syst. 23(8), 1400–1414 (2012)

    Article  Google Scholar 

  33. Calheiros, R.N., Buyya, R.: Meeting deadlines of scientific workflows in public clouds with tasks replication. IEEE Trans. Parallel Distrib. Syst. 25(7), 1787–1796 (2014)

    Article  Google Scholar 

  34. Anwar, N., Deng, H.: Elastic scheduling of scientific workflows under deadline constraints in cloud computing environments. Fut. Internet 10(1), 5 (2018)

    Article  Google Scholar 

  35. Meena, J., Kumar, M., Vardham, M.: Cost effective genetic algorithm for workflow scheduling in cloud under deadline constraint. IEEE Access 4, 5065–5082 (2016)

    Article  Google Scholar 

  36. Wu, C.Q., Lin, X., Yu, D., Xu, W., Li, L.: End-to-end delay minimization for scientific workflows in clouds under budget constraint. IEEE Trans. Cloud Comput. 3(2), 169–181 (2015)

    Article  Google Scholar 

  37. Arabnejad, V., Bubendorfer, K., Ng, B.: Budget distribution strategies for scientific workflow scheduling in commercial clouds. In: International Conference on E-science 2016, pp. 137–146

  38. Faragardi, H.R., Sedghpour, M.R.S., Fazliahmadi, S., Fahringer, T., Rasouli, N.: GRP-HEFT: A budget-constrained resource provisioning scheme for workflow scheduling in IaaS clouds. IEEE Trans. Parallel Distrib. Syst. 31(6), 1239–1254 (2019)

    Article  Google Scholar 

  39. Rizvi, N., Ramesh, D.: Fair budget constrained workflow scheduling approach for heterogeneous clouds. Cluster Comput. 1–17 (2020).

  40. Chakravarthi, K.K., Shyamala, L., Vaidehi, V.: Budget aware scheduling algorithm for workflow applications in IaaS clouds. Cluster Comput. 1–15 (2020).

  41. Su, S., Jian, L., Huang, Q., Xiao, H., Kai, S., Jie, W.: Cost-efficient task scheduling for executing large programs in the cloud. Parallel Comput. 39(4–5), 177–188 (2013)

    Article  Google Scholar 

  42. Topcuoglu, H., Hariri, S., Wu, M.: Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans. Parallel Distrib. Syst. 13(3), 260–274 (2002)

    Article  Google Scholar 

  43. Zhu, Z., Zhang, G., Li, M., Liu, X.: Evolutionary multi-objective Workflow scheduling in cloud. IEEE Trans. Parallel Distrib. Syst. 27(5), 1344–1357 (2016)

    Article  Google Scholar 

  44. Choudhary, A., Gupta, I., Singh, V., Jana, P.K.: A GSA based hybrid algorithm for bi-objective workflow scheduling in cloud computing. Fut. Gener. Comput. Syst. 83, 14–26 (2018)

    Article  Google Scholar 

  45. Malawski, M., Juve, G., Deelman, E., Nabrzyski, J.: Algorithms for cost-and deadline-constrained provisioning for scientific workflow ensembles in IaaS clouds. Fut. Gener. Comput. Syst. 48, 1–18 (2015)

    Article  Google Scholar 

  46. Verma, A., Kaushal, S.: Bi-criteria priority based particle swarm optimization workflow scheduling algorithm for cloud. In: Engineering & Computational Sciences 2014, pp. 1–6

  47. Verma, A., Kaushal, S.: Cost-time efficient scheduling plan for executing workflows in the cloud. J. Grid Comput. 13(4), 1–12 (2015)

    Article  Google Scholar 

  48. Amazon: Amazon EC2 Pricing. https://aws.amazon.com/ec2/pricing/. Accessed 5 Aug. 2019

  49. Google: Google Cloud Platform. https://cloud.google.com/compute/ (2017). Accessed 5 Aug 2019

  50. Microsoft: Microsoft Azure. https://azure.microsoft.com (2017). Accessed 5 Aug 2019

  51. Barr, J.: New-Per-Second Billing for EC2 Instances and EBS Volumes. https://aws.amazon.com/tw/blogs/aws/new-per-second-billing-for-ec2-instances-and-ebs-volumes/ (2017). Accessed 1 Feb 2019

  52. Juve, G., Chervenak, A., Deelman, E., Bharathi, S., Mehta, G., Vahi, K.: Characterizing and profiling scientific workflows. Fut. Gener. Comput. Syst. 29(3), 682–692 (2013)

    Article  Google Scholar 

  53. Palankar, M.R., Iamnitchi, A., Ripeanu, M., Garfinkel, S.: Amazon S3 for science grids: a viable solution? In: Proceedings of the 2008 International Workshop on Data-Aware Distributed Computing 2008, pp. 55–64. ACM

  54. Mao, M., Humphrey, M.: A performance study on the VM startup time in the cloud. In: International Conference on Cloud Computing 2012, pp. 423–430

  55. Juve, G.: Workflow Generator. https://confluence.pegasus.isi.edu/display/pegasus/WorkflowGenerator. Accessed 12 June 2018

Download references

Acknowledgements

This work is supported by National Natural Science Foundation of China (Grant Nos. 61772205, 61872084), Guangzhou Science and Technology Program key projects (Grant Nos. 202007040002, 201902010040 and 201907010001), Guangzhou Development Zone Science and Technology (Grant No. 2018GH17), Guangdong Major Project of Basic and Applied Basic Research (2019B030302002), and the Fundamental Research Funds for the Central Universities, SCUT (Grant No. 2019ZD26).

Author information

Authors and Affiliations

  1. Cyberspace Institute of Advanced Technology, Guangzhou University, Guangzhou, China

    Naqin Zhou

  2. School of Computer Science and Engineering, South China University of Technology, Guangzhou, China

    Weiwei Lin & Fang Shi

  3. Guangzhou Branch of Shanghai Yizhong Enterprise Management Consulting Co., Ltd, Shanghai, China

    Wei Feng

  4. School of Computer, South China Normal University, Guangzhou, China

    Xiongwen Pang

Authors
  1. Naqin Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weiwei Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wei Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fang Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiongwen Pang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Weiwei Lin or Xiongwen Pang.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interest regarding the publication of this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, N., Lin, W., Feng, W. et al. Budget-deadline constrained approach for scientific workflows scheduling in a cloud environment. Cluster Comput 26, 1737–1751 (2023). https://doi.org/10.1007/s10586-020-03176-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-020-03176-1

Keywords

Search

Navigation