Abstract
MapReduce is a programming framework for processing large amount of data in distribution. MapReduce implementations, such as Hadoop MapReduce, basically operate on dedicated clusters of workstations to achieve high performance. However, the dedicated clusters can be unrealistic for users who infrequently have a demand of solving large distributed problems. This paper presents an approach of applying the MapReduce framework on peer-to-peer (P2P) networks for distributed applications. This approach aims at exploiting leisure resources including storage, bandwidth and processing power on peers to perform MapReduce operations. The paper also introduces a prototyping implementation of a MapReduce P2P system, where the main functions of peers contain contributing computing resources, forming computing groups and executing the MapReduce operations. The performance evaluation of the system has been compared with the Hadoop cluster using the prevailing word count problem.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Dean, J., Ghemawat, S.: MapReduce: Simplified Data Processing on Large Clusters. Commun. ACM 51, 107–113 (2008)
Cohen, B.: Incentives Build Robustness in BitTorrent. In: Proc. 1st Workshop on Economics of Peer-to-Peer Systems (2003)
Heckmann, O., Bock, A., Mauthe, A., Steinmetz, R.: The eDonkey File-Sharing Network. In: Proc. GI Jahrestagung (2), pp. 224–228 (2004)
Berkovsky, S., Kuflik, T., Ricci, F.: P2P Case Retrieval with an Unspecified Ontology. In: Muñoz-Ávila, H., Ricci, F. (eds.) ICCBR 2005. LNCS (LNAI), vol. 3620, pp. 91–105. Springer, Heidelberg (2005)
Faroo, http://www.faroo.com/ (last access in January 2011)
Yacy, http://www.yacy.de/ (last access in January 2011)
Schlosser, M., Sintek, M., Decker, S., Nejdl, W.: A Scalable and Ontology-Based P2P Infrastructure for Semantic Web Services. In: Proc. 2nd International Conference on Peer-to-Peer Computing, P2P 2002, p. 104. IEEE Computer Society, Washington, DC (2002)
Tatarinov, I., Ives, Z., Madhavan, J., Halevy, A., Suciu, D., Dalvi, N., Dong, X., Kadiyska, Y., Miklau, G., Mork, P.: The Piazza Peer Data Management Project. SIGMOD Rec. 32(3), 47–52 (2003)
Tran, H.M., Schönwälder, J.: Heuristic Search using a Feedback Scheme in Unstructured Peer-to-Peer Networks. In: Proc. 5th International Workshop on Databases, Information Systems and Peer-to-Peer Computing. Springer (2007)
Ratnasamy, S., Francis, P., Handley, M., Karp, R., Schenker, S.: A Scalable Content Addressable Network. In: Proc. Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications, SIGCOMM 2001, pp. 161–172. ACM Press, New York (2001)
Stoica, I., Morris, R., Karger, D., Kaashoek, M.F., Balakrishnan, H.: Chord: A Scalable Peer-to-Peer Lookup Service for Internet Applications. In: Proc. Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications, SIGCOMM 2001, pp. 149–160. ACM Press, New York (2001)
Maymounkov, P., Mazières, D.: Kademlia: A Peer-to-Peer Information System Based on the XOR Metric. In: Druschel, P., Kaashoek, M.F., Rowstron, A. (eds.) IPTPS 2002. LNCS, vol. 2429, pp. 53–65. Springer, Heidelberg (2002)
Gnutella Protocol Specification version 0.4 (2001), http://rfc-gnutella.sourceforge.net/developer/stable/index.html (last access in March 2012)
Clarke, I., Sandberg, O., Wiley, B., Hong, T.W.: Freenet: A Distributed Anonymous Information Storage and Retrieval System. In: Federrath, H. (ed.) Anonymity 2000. LNCS, vol. 2009, pp. 46–66. Springer, Heidelberg (2001)
Yang, B., Garcia-Molina, H.: Designing a Super-Peer Network. In: Proc. 19th International Conference on Data Engineering, ICDE 2003, p. 49. IEEE Computer Society, Los Alamitos (2003)
Marozzo, F., Talia, D., Trunfio, P.: A Framework for Managing MapReduce Applications in Dynamic Distributed Environments. In: Proc. 19th International Euromicro Conference on Parallel, Distributed and Network-Based Processing, pp. 149–158. IEEE Computer Society, Los Alamitos (2011)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Dang, H.T., Tran, H.M., Vu, P.N., Nguyen, A.T. (2012). Applying MapReduce Framework to Peer-to-Peer Computing Applications. In: Nguyen, NT., Hoang, K., Jȩdrzejowicz, P. (eds) Computational Collective Intelligence. Technologies and Applications. ICCCI 2012. Lecture Notes in Computer Science(), vol 7654. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34707-8_8
Download citation
DOI: https://doi.org/10.1007/978-3-642-34707-8_8
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-34706-1
Online ISBN: 978-3-642-34707-8
eBook Packages: Computer ScienceComputer Science (R0)