Skip to main content
SpringerLink
Log in

Predicting the CPU availability of time‐shared Unix systems on the computational grid

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

In this paper we focus on the problem of making short and medium term forecasts of CPU availability on time‐shared Unix systems. We evaluate the accuracy with which availability can be measured using Unix load average, the Unix utility vmstat, and the Network Weather Service CPU sensor that uses both. We also examine the autocorrelation between successive CPU measurements to determine their degree of self‐similarity. While our observations show a long‐range autocorrelation dependence, we demonstrate how this dependence manifests itself in the short and medium term predictability of the CPU resources in our study.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. O. Arndt, B. Freisleben, T. Kielmann and F. Thilo, Scheduling parallel applications in networks of mixed uniprocessor/multiprocessor workstations, in: Proceedings of ISCA 11th Conference on Parallel and Distributed Computing (September 1998).

  2. F. Berman, R. Wolski, S. Figueira, J. Schopf and G. Shao, Application level scheduling on distributed heterogeneous networks, in: Proceedings of Supercomputing 1996 (1996).

  3. M. Bernan, R. Serman and M. Taqqu, Long-range dependence in variable-bit-rate video traffic, IEEE Trans. Comm. (March 1995).

  4. S. Burnett and S. Fitzgerald, Metacomputing supports large-scale distributed simulations, 1998, http://www.cacr.caltech. edu/sfexpress/sc98.html.

  5. A. Cambel, Applied Chaos Theory (Academic Press, New York, 1993).

    Google Scholar 

  6. B. Carpenter, G. Zhang, G. Fox, X. Li and Y. Wen, Hpjava: Data parallel extensions to Java, in: ACM Workshop on Java for High-Performance Network Computing (1998), http://www.cs. ucsb.edu/conferences/java98/papers/hpjava.pdf.

  7. H. Casanova and J. Dongarra, NetSolve: A network server for solving computational science problems, Internat. J. Supercomputer Appl. (1997).

  8. M. Crovella and A. Bestavros, Self-similarity in world wide web traffic: Evidence and possible causes, IEEE/ACMTrans. Networking 5 (December 1997).

  9. D. Culler, A. Arpaci-Dusseau, R. Arpaci-Dusseau, B. Chun, S. Lumetta, A. Mainwaring, R. Martin, C. Yoshikawa and F. Wong, Parallel computing on the berkeley now, in: Proc. of 9th Joint Symposium on Parallel Processing, JSPP'97 (1997), http://now. CS.Berkeley.EDU/Papers2.

  10. P. Dinda and D. O'Halloran, The statistical properties of host load, in: 4th Workshop on Languages, Compilers, and Run-time Systems for Scalable Computers (LCR98); also CMU Technical Report CMU-CS–98–143 (1998), http://reports-archive. adm.cs.cmu.edu/anon/1998/CMU-CS–98–143.ps.

  11. S. Figueira and F. Berman, Modeling the effects of contention on the performance of heterogeneous applications, in: Proc. 6th IEEE Symp. on High Performance Distributed Computing (August 1997).

  12. I. Foster and C. Kesselman, Globus: A metacomputing infrastructure toolkit, Internat. J. Supercomputer Appl. (1997).

  13. I. Foster and C. Kesselman (Eds.), The Grid-Blueprint for a New Computing Infrastructure (Morgan Kaufmann, San Mateo, CA, 1998).

    Google Scholar 

  14. B. Ganguly, G. Bryan, M. Norman and A. Chien, Exploring structured mesh refinement (samr) methods with the illinois concert system, in: Proceedings of SIAM Conference on Parallel Processing (March 1997).

  15. J. Gehrinf and A. Reinfeld, Mars-a framework for minimizing the job execution time in a metacomputing environment, Future Generation Computer Systems (1996).

  16. C. Granger and P. Newbold, Forecasting Economic Time Series (Academic Press, New York, 1986).

    Google Scholar 

  17. S. Gribble, S. Manku, D. Roselli and E. Brewer, Self-similarity in file systems, http://www.cs.berkeley.edu/~gribble.

  18. A.S. Grimshaw, W.A. Wulf, J.C. French, A.C. Weaver and P.F. Reynolds, Legion: The next logical step toward a nationwide virtual computer, Technical Report CS–94–21, University of Virginia (1994).

  19. R. Haddad and T. Parsons, Digital Signal Processing: Theory, Applications, and Hardware (Computer Science Press, 1991).

  20. W. Leland, M. Taqqu, W. Willinger and D. Wilson, On the selfsimilar nature of Ethernet traffic, IEEE/ACM Trans. Networking (February 1994).

  21. B.B. Mandelbrot and M.S. Taqqu, Robust R/S analysis of long-run serial correlation, in: Proceedings of the 42nd Session of the ISI, Vol. 48 (1979) pp. 69–99.

    Google Scholar 

  22. K. Park, G. Kim and M. Crovella, On the effect of traffic selfsimilarity on network performance, in: Proceedings of the SPIE International Conference on Performance and Control of Network Systems (November 1997).

  23. J. Schopf and F. Berman, Performance prediction in production environments, in: Proceedings of IPPS/SPDP 1998 (1998), http:// www.cs.ucsd.edu/users/jenny/TechReports/ ipps98.ps.

  24. N. Spring and R. Wolski, Application level scheduling: Gene sequence library comparison, in: Proceedings of ACM International Conference on Supercomputing 1998 (July 1998).

  25. T. Sterling, D. Becker and D. Savarese, Beowulf: A parallel workstation for scientific computation, in: Proceedings of the International Conference on Parallel Processing (1995).

  26. T. Tannenbaum and M. Litzkow, The condor distributed processing system, Dr. Dobbs Journal (February 1995).

  27. J. Weissman and A. Grimshaw, A framework for partitioning parallel computations in heterogeneous environments, Concurrency: Practice and Experience 7(5) (August 1995).

  28. W. Willinger, M. Taqqu, R. Sherman and D. Wilson, Self-similarity through high-variability: statistical analysis of ethernet lan traffic at the source level, in: SIGCOMM'95 Conference on Communication Architectures, Protocols, and Applications (1995).

  29. R. Wolski, Dynamically forecasting network performance using the network weather service, Cluster Computing (1998), http:// www.cs.utk.edu/~rich/publications/nws-tr.ps. gz.

  30. R. Wolski, N. Spring and J. Hayes, The network weather service: A distributed resource performance forecasting service for metacomputing, Future Generation Computer Systems (1999), http://www.cs.utk.edu/~rich/publications/nwsarch. ps.gz.

  31. R. Wolski, N. Spring and C. Peterson, Implementing a performance forecasting system for metacomputing: The network weather service, in: Proceedings of Supercomputing 1997 (November 1997).

Download references

Author information

Authors and Affiliations

  1. Computer Science Department, University of Tennessee, Knoxville, TN, 37996, USA

    Rich Wolski

  2. Computer Science Department, University of Washington, Seattle, WA, 98195, USA

    Neil Spring

  3. Computer Science and Engineering Department, University of California, San Diego, La Jolla, CA, 92093, USA

    Jim Hayes

Authors
  1. Rich Wolski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Neil Spring
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jim Hayes
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wolski, R., Spring, N. & Hayes, J. Predicting the CPU availability of time‐shared Unix systems on the computational grid. Cluster Computing 3, 293–301 (2000). https://doi.org/10.1023/A:1019052825453

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1019052825453

Keywords

Search

Navigation