Abstract
Policy driven Grid Computing environment is widely used in professional organisation and educational institutions when complex tasks need to be performed. Policies are designed so that the performance of the system is optimal and throughput of the Grid setup is maximum and also cater to the needs of all the users who are consumers of the Grid environment. Memory is one of the key resourcing parameter and policies involving the memory usage of a job are designed so that memory starvation is prevented. Machine Learning algorithms which finds its applications in diverse fields are used to predict the memory requirements of a job which in turn leads to better utilization of the Grid infrastructure. Two different datasets are considered in this paper from diverse fields and the memory requirements were predicted. Promising results were obtained from the proposed algorithm and this can be used in any field for identifying the memory requirement of jobs at a project/user level.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Berstis, V.: IBM RedBook; Fundamentals of Grid Computing. Technical report (2002)
Sun N1 Grid Engine 6.1 Administration Guide. The grid workloadformat, http://docs.oracle.com/cd/E19957-01/820-0698/chp8-1501/index.html (accessed: May 30, 2013)
Liu, K., Subbarayan, S., Shoults, R.R., Manry, M.T., Kwan, C., Lewis, F.L., Naccarino, J.: Comparison of very short-term load forecasting techniques. IEEE Transactions on Power Systems 11(2), 877–882 (1996)
Guo, X., Zhang, W., Wang, J., Wu, G.: Resource management using dynamical load prediction and multiprocessor cooperation. In: 2011 7th International Conference on Networked Computing and Advanced Information Management (NCM), pp. 200–205 (2011)
Fanand, S., Hyndman, R.J.: Short-term load forecasting based on a semi-parametric additive model. IEEE Transactions on Power Systems 27(1), 134–141 (2012)
Dumitrescu, C.L., Foster, I.: Gangsim: a simulator for grid scheduling studies. In: Proceedings of the Fifth IEEE International Symposium on Cluster Computing and the Grid (CCGrid 2005), vol. 2, pp. 1151–1158. IEEE Computer Society, Washington, DC (2005)
Sun Microsystems Inc. N1 Grid Engine 6 Administration Guide. Technical report (May 2005)
IBM. IBM Platform Computing Solutions. Technical report (December 2012)
Wang, D.: Meeting green computing challenges. In: International Symposium on High Density packaging and Microsystem Integration, HDP 2007, pp. 1–4 (2007)
Iosup, A., Li, H., Jan, M., Anoep, S., Dumitrescu, C., Wolters, L., Epema, D.H.J.: The grid workloads archive. Future Generation Computer Systems 24(7), 672–686 (2008)
Advanced School for Computing and Imaging, TU Delft. The grid workloads archive- DAS-2, http://gwa.ewi.tudelft.nl/pmwiki/pmwiki.php?n=Workloads.Gwa-t-1 (accessed: May 30, 2013)
Mitra, P., Member, S., Murthy, C.A., Pal, S.K.: Unsupervised feature selection using feature similarity. IEEE Transactions on Pattern Analysis and Machine Intelligence 24, 301–312 (2002)
Advanced School for Computing and Imaging, TU Delft. The grid workloadformat, http://gwa.ewi.tudelft.nl/TheGridWorkloadFormat_v001.pdf (accessed: May 30, 2013)
Møller, M.F.: A scaled conjugate gradient algorithm for fast supervised learning. Neural Networks 6(4), 525–533 (1993)
MATLAB. version 7.10.0 (R2010a). The MathWorks Inc., Natick, Massachusetts (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Dheenadayalan, K., Shah, M., Badjatya, A., Chatterjee, B. (2014). Machine Learning in a Policy Driven Grid Environment. In: Chatterjee, M., Cao, Jn., Kothapalli, K., Rajsbaum, S. (eds) Distributed Computing and Networking. ICDCN 2014. Lecture Notes in Computer Science, vol 8314. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-45249-9_39
Download citation
DOI: https://doi.org/10.1007/978-3-642-45249-9_39
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-45248-2
Online ISBN: 978-3-642-45249-9
eBook Packages: Computer ScienceComputer Science (R0)