Skip to main content
SpringerLink
Log in

Heuristic Cycle-Based Scheduling with Backfilling for Large-Scale Distributed Environments

  • Conference paper
Proceedings of the Ninth International Conference on Dependability and Complex Systems DepCoS-RELCOMEX. June 30 – July 4, 2014, Brunów, Poland

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 286))

  • 1319 Accesses

Abstract

The paper is devoted to comparing the results of an independent job batch scheduling in terms of a virtual organization policy and available resources usage efficiency in large distributed environments like utility Grid. A hybrid approach is proposed on the basis of a cyclic scheduling scheme and backfilling combination. Additionally the paper offers a heuristic shifting procedure which improves jobs execution alternatives selected in the cyclic scheme. The simulation results show that depending on the scheduling efficiency indicator and the level of resource availability each of the approaches is able to provide the best results. Moreover the obtained results are valid under conditions of dynamically varying state of resources and inaccurate user job runtime estimations.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Garg, S.K., Konugurthi, P., Buyya, R.: A Linear Programming-driven Genetic Algorithm for Meta-scheduling on Utility Grids. J. Par., Emergent and Distr. Systems 26, 493–517 (2011)

    Article  MathSciNet  Google Scholar 

  2. Buyya, R., Abramson, D., Giddy, J.: Economic Models for Resource Management and Scheduling in Grid Computing. J. Concurrency and Computation 14(5), 1507–1542 (2002)

    Article  MATH  Google Scholar 

  3. Ernemann, C., Hamscher, V., Yahyapour, R.: Economic Scheduling in Grid Computing. In: Feitelson, D.G., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2002. LNCS, vol. 2537, pp. 128–152. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  4. Lee, Y.C., Wang, C., Zomaya, A.Y., Zhou, B.B.: Profit-driven Scheduling for Cloud Services with Data Access Awareness. J. Par. and Distr. Computing 72(4), 591–602 (2012)

    Article  Google Scholar 

  5. Garg, S.K., Buyya, R., Siegel, H.J.: Scheduling Parallel Applications on Utility Grids: Time and Cost Trade-off Management. In: 32nd Australasian Computer Science Conference (ACSC 2009), Wellington, New Zealand, pp. 151–159 (2009)

    Google Scholar 

  6. Degabriele, J.P., Pym, D.: Economic Aspects of a Utility Computing Service, Trusted Systems Laboratory HP Laboratories Bristol HPL-2007-101. Technical Report, pp. 1-23 (July 3, 2007)

    Google Scholar 

  7. Garg, S.K., Yeo, C.S., Anandasivam, A., Buyya, R.: Environment-conscious Scheduling of HPC Applications on Distributed Cloud-oriented Data Centers. J. Parallel and Distributed Computing 71(6), 732–749 (2011)

    Article  MATH  Google Scholar 

  8. Tesauro, G., Bredin, J.L.: Strategic Sequential Bidding in Auctions Using Dynamic Programming. In: 1st International Joint Conference on Autonomous Agents and Multiagent Systems, Part 2, pp. 591–598. ACM, New York (2002)

    Chapter  Google Scholar 

  9. Thain, D., Tannenbaum, T., Livny, M.: Distributed Computing in Practice: the Condor Experience. J. Concurrency and Computation: Practice and Experience 17(2-4), 323–356 (2004)

    Article  Google Scholar 

  10. Berman, F.: High-performance Schedulers. In: Foster, I., Kesselman, C. (eds.) The Grid: Blueprint for a New Computing Infrastructure, pp. 279–309. Morgan Kaufmann, San Francisco (1999)

    Google Scholar 

  11. Yang, Y., Raadt, K., Casanova, H.: Multiround Algorithms for Scheduling Divisible Loads. IEEE Trans. Parallel and Distributed Systems 16(8), 1092–1102 (2005)

    Article  Google Scholar 

  12. Natrajan, A., Humphrey, M.A., Grimshaw, A.S.: Grid Resource Management in Legion. In: Nabrzyski, J., Schopf, J.M., Weglarz, J. (eds.) Grid Resource Management. State of the Art and Future Trends, pp. 145–160. Kluwer Academic Publishers, Boston (2003)

    Google Scholar 

  13. Beiriger, J., Johnson, W., Bivens, H.: Constructing the ASCI Grid. In: 9th IEEE Symposium on High Performance Distributed Computing, pp. 193–200. IEEE Press, New York (2000)

    Google Scholar 

  14. Frey, J., Foster, I., Livny, M.: Condor-G: a Computation Management Agent for Multi-institutional Grids. In: 10th International Symposium on High-Performance Distributed Computing, pp. 55–66. IEEE Press, New York (2001)

    Chapter  Google Scholar 

  15. Abramson, D., Giddy, J., Kotler, L.: High Performance Parametric Modeling with Nimrod/G: Killer Application for the Global Grid? In: International Parallel and Distributed Processing Symposium, pp. 520–528. IEEE Press, New York (2000)

    Google Scholar 

  16. Foster, I., Kesselman, C., Tuecke, S.: The Anatomy of the Grid: Enabling Scalable Virtual Organizations. Int. J. of High Performance Computing Applications 15(3), 200–222 (2001)

    Article  Google Scholar 

  17. Kurowski, K., Nabrzyski, J., Oleksiak, A., Weglarz, J.: Multicriteria Aspects of Grid Resource Management. In: Nabrzyski, J., Schopf, J.M., Weglarz, J. (eds.) Grid Resource Management. State of the Art and Future Trends, pp. 271–293. Kluwer Academic Publishers, Boston (2003)

    Google Scholar 

  18. Cafaro, M., Mirto, M., Aloisio, G.: Preference-Based Matchmaking of Grid Resources with CP-Nets. J. Grid Computing 11(2), 211–237 (2013)

    Article  Google Scholar 

  19. Rodero, I., Villegas, D., Bobroff, N., Liu, Y., Fong, L., Sadjadi, S.M.: Enabling Interoperability among Grid Meta-Schedulers. J. Grid Computing 11(2), 311–336 (2013)

    Article  Google Scholar 

  20. Toporkov, V.: Application-Level and Job-Flow Scheduling: an Approach for Achieving Quality of Service in Distributed Computing. In: Malyshkin, V. (ed.) PaCT 2009. LNCS, vol. 5698, pp. 350–359. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  21. Aida, K., Casanova, H.: Scheduling Mixed-parallel Applications with Advance Reservations. In: 17th IEEE Int. Symposium on HPDC, pp. 65–74. IEEE CS Press, New York (2008)

    Google Scholar 

  22. Ando, S., Aida, K.: Evaluation of Scheduling Algorithms for Advance Reservations. Information Processing Society of Japan SIG Notes HPC-113, 37–42 (2007)

    Google Scholar 

  23. Elmroth, E., Tordsson, J.: A Standards-based Grid Resource Brokering Service Supporting Advance Reservations, Coallocation and Cross-Grid Interoperability. J. of Concurrency and Computation 25(18), 2298–2335 (2009)

    Article  Google Scholar 

  24. Toporkov, V., Tselishchev, A., Yemelyanov, D., Bobchenkov, A.: Composite Scheduling Strategies in Distributed Computing with Non-dedicated Resources. Procedia Computer Science 9, 176–185 (2012)

    Article  Google Scholar 

  25. Moab Adaptive Computing Suite, http://www.adaptivecomputing.com/products/moab-adaptive-computing-suite.php

  26. Lee, S.B., Schwartzman, Y., Hardy, J., Snavely, A.: Are User Runtime Estimates Inherently Inaccurate? In: Feitelson, D.G., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2004. LNCS, vol. 3277, pp. 253–263. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  27. Toporkov, V., Toporkova, A., Tselishchev, A., Yemelyanov, D.: Slot Selection Algorithms in Distributed Computing with Non-dedicated and Heterogeneous Resources. In: Malyshkin, V. (ed.) PaCT 2013. LNCS, vol. 7979, pp. 120–134. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  28. Azzedin, F., Maheswaran, M., Arnason, N.: A Synchronous Co-allocation Mechanism for Grid Computing Systems. Cluster Computing 7, 39–49 (2004)

    Article  Google Scholar 

  29. Castillo, C., Rouskas, G.N., Harfoush, K.: Resource Co-allocation for Large-scale Distributed Environments. In: 18th ACM International Symposium on High Performance Distributed Compuing, pp. 137–150. ACM, New York (2009)

    Google Scholar 

  30. Takefusa, A., Nakada, H., Kudoh, T., Tanaka, Y.: An Advance Reservation-Based Co-allocation Algorithm for Distributed Computers and Network Bandwidth on QoS-Guaranteed Grids. In: Frachtenberg, E., Schwiegelshohn, U. (eds.) JSSPP 2010. LNCS, vol. 6253, pp. 16–34. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  31. Blanco, H., Guirado, F., Lérida, J.L., Albornoz, V.M.: MIP Model Scheduling for Multi-Clusters. In: Caragiannis, I., et al. (eds.) Euro-Par Workshops 2012. LNCS, vol. 7640, pp. 196–206. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  32. Moise, D., Moise, I., Pop, F., Cristea, V.: Resource CoAllocation for Scheduling Tasks with Dependencies, in Grid. In: The Second International Workshop on High Performance in Grid Middleware (HiPerGRID 2008), Bucharest, Romania, pp. 41–48. IEEE Romania (2008)

    Google Scholar 

  33. Olteanu, A., Pop, F., Dobre, C., Cristea, V.: A Dynamic Rescheduling Algorithm for Resource Management in Large Scale Dependable Distributed Systems. Computers and Mathematics with Applications 63(9), 1409–1423 (2012)

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. National Research University “MPEI”, ul. Krasnokazarmennaya, 14, Moscow, 111250, Russia

    Victor Toporkov, Dmitry Yemelyanov & Petr Potekhin

  2. Moscow State Institute of Electronics and Mathematics, National Research University Higher School of Economics, Bolshoy Trekhsvyatitelsky per., 1-3/12, Moscow, 109028, Russia

    Anna Toporkova

  3. European Organization for Nuclear Research (CERN), Geneva, 23, 1211, Switzerland

    Alexey Tselishchev

Authors
  1. Victor Toporkov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anna Toporkova
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alexey Tselishchev
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dmitry Yemelyanov
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Petr Potekhin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Victor Toporkov .

Editor information

Editors and Affiliations

  1. Institute of Computer Engineering Control and Robotics, Wrocław University of Technology, Wrocław, Poland

    Wojciech Zamojski

  2. Institute of Computer Engineering Control and Robotics, Wrocław University of Technology, Wrocław, Poland

    Jacek Mazurkiewicz

  3. Institute of Computer Engineering Control and Robotics, Wrocław University of Technology, Wrocław, Poland

    Jarosław Sugier

  4. Institute of Computer Engineering Control and Robotics, Wrocław University of Technology, Wrocław, Poland

    Tomasz Walkowiak

  5. Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland

    Janusz Kacprzyk

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this paper

Cite this paper

Toporkov, V., Toporkova, A., Tselishchev, A., Yemelyanov, D., Potekhin, P. (2014). Heuristic Cycle-Based Scheduling with Backfilling for Large-Scale Distributed Environments. In: Zamojski, W., Mazurkiewicz, J., Sugier, J., Walkowiak, T., Kacprzyk, J. (eds) Proceedings of the Ninth International Conference on Dependability and Complex Systems DepCoS-RELCOMEX. June 30 – July 4, 2014, Brunów, Poland. Advances in Intelligent Systems and Computing, vol 286. Springer, Cham. https://doi.org/10.1007/978-3-319-07013-1_44

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-07013-1_44

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-07012-4

  • Online ISBN: 978-3-319-07013-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation