Abstract
This paper describes a constructive approach of distributed parallel computing using by hybrid union of MAPREDUCE and MPI technologies for solving oil extracting problems. We extend a common architecture of MAPREDUCE model by organizing decomposition of computational domain at different stages of MAPREDUCE process. We describes Model Driven Architecture (MDA) models for developing formal views of high-performance computing technologies using MAPREDUCE. We made computing experiments and show on specific HPC infrastructure. All implementations of programs is realize on Java platform. This approach will possible one of the ways to do cloud computing on high performance heterogeneous systems.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Gelenbe, E., Lichnewsky, A., Staphylopatis, A.: Experience with the parallel solution of partial-differential equations on a distributed computing system. IEEE Transactions on Computers 31(12), 1157–1164 (1982)
Gropp, W., Lusk, E., Doss, N., et al.: A high-performance, portable implementation of the MPI message passing interface standard. Parallel Computing 22(6), 789–828 (1996)
Sunderam, V.S., Geist, G.A., Dongarra, J., et al.: The PVM concurrent computing system - evolution, experiences, and trends. Parallel Computing 20(4), 531–545 (1994)
Malyshkin, V.: Assembling of Parallel Programs for Large Scale Numerical Modeling, p. 1021. IGI Global, Chicago (2010)
Becker, J.C., Dagum, L.: Particle simulation on heterogeneous distributed supercomputers. Concurrency-Practice and Experience 5(4), 367–377 (1993)
Fougère, D., Gorodnichev, M., Malyshkin, N., Malyshkin, V., Merkulov, A., Roux, B.: NumGrid middleware: MPI support for computational grids. In: Malyshkin, V.E. (ed.) PaCT 2005. LNCS, vol. 3606, pp. 313–320. Springer, Heidelberg (2005)
Diaz, J., Munoz-Caro, C., Nino, A.A.: Survey of Parallel Programming Models and Tools in the Multi and Many-Core Era. IEEE Transactions on Parallel and Distributed Systems 23(8), 1369–1386 (2012)
Wang, J., Liu, Z.: Parallel Data Mining Optimal Algorithm of Virtual Cluster. In: International Conference on Fuzzy Systems and Knowledge, vol. 5, pp. 358–362 (2008)
Pandey, S., Buyya, R.: Scheduling Workflow Applications Based on Multi-source Parallel Data Retrieval in Distributed Computing Networks. Computer J. 55(11), 1288–1308 (2012)
Liu, H., Orban, D.: GridBatch: Cloud Computing for Large-Scale Data-Intensive Batch Applications. In: CCGRID 2008, vol. 1, pp. 295–305 (2008)
Valilai, O.F., Houshmand, M.: A collaborative and integrated platform to support distributed manufacturing system using a service-oriented approach based on cloud computing paradigm. Robotics and Computer-Integrated Manufacturing 29(1), 110–127 (2013)
Dean, J., Ghemawat, S.: Mapreduce: Simplified data processing on large clusters. Communications of the ACM 51(1), 107–113 (2008)
Fagg, G.E., Dongarra, J.: FT-MPI: Fault Tolerant MPI, Supporting Dynamic Applications in a Dynamic World. In: Dongarra, J., Kacsuk, P., Podhorszki, N. (eds.) PVM/MPI 2000. LNCS, vol. 1908, pp. 346–353. Springer, Heidelberg (2000)
He, Q., Wang, Q., Zhuang, F., Tan, Q., Shi, Z.: Parallel CLARANS Clustering Based on MapReduce. Energy Procedia 13, 3269–3279 (2011)
Cohen, J.: Graph twiddling in a MapReduce world. Computing in Science and Engineering 11, 29–41 (2009)
Baker, M.: MpiJava: A Java interface to MPI. University of Portsmouth, Portsmouth (2010)
Aziz, H., Sattar, A.: Mathematical modeling of reservoir systems. Nedra, Moscow (1982)
Demmel, J., Veselic, K.: Jacobis method is more accurate than QR. SIAM Journal on Matrix Analysis and Applications 13(4), 1204–1245 (1992)
Lam, C.: Hadoop In Action. Manning Publications Co., Stamford (2010)
Apache Software Foundation, HDFS (2011), http://hadoop.apache.org/common/docs/current/hdfs_design.html
Srirama, S.N., Jakovits, P., Vainikko, E.: Adapting scientific computing problems to clouds using MapReduce. Future Generation Computer Systems 28(2), 184–192 (2012)
Frankel, D.: Model Driven Architecture. Applying MDA to Enterprise Computing. Wiley Publishing, Indiana (2003)
Lugato, D.: Model-driven engineering for high-performance computing applications. In: The 19th IASTED International Conference on Modelling and Simulation, pp. 18–33. IEEE Press, New York (2008)
Klusik, U., Loogen, R., Priebe, S., Rubio, F.: Implementation skeletons in Eden: Low-effort parallel programming. In: Mohnen, M., Koopman, P. (eds.) IFL 2000. LNCS, vol. 2011, pp. 71–88. Springer, Heidelberg (2001)
Srirama, S.N., Batrashev, O., Jakovits, P., et al.: Scalability of parallel scientific applications on the cloud. Scientific Programming 19(2-3), 91–105 (2011)
Plimpton, S., Devine, K.: MapReduce in MPI for Large-scale graph algorithms. Parallel Computing 37(9), 610–632 (2011)
Ekanayake, J., Li, H., Zhang, B., Gunarathne, T., Bae, S.-H., Qiu, J., Fox, G.: Twister: a runtime for iterative MapReduce. In: HPDC 2010, pp. 810–818. ACM, New York (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Akhmed-Zaki, D., Danaev, N., Matkerim, B., Bektemessov, A. (2013). Design of Distributed Parallel Computing Using by MapReduce/MPI Technology. In: Malyshkin, V. (eds) Parallel Computing Technologies. PaCT 2013. Lecture Notes in Computer Science, vol 7979. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39958-9_12
Download citation
DOI: https://doi.org/10.1007/978-3-642-39958-9_12
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-39957-2
Online ISBN: 978-3-642-39958-9
eBook Packages: Computer ScienceComputer Science (R0)