Skip to main content
SpringerLink
Log in

Evaluation of messaging middleware for high-performance cloud computing

  • Original Article
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

Cloud computing is posing several challenges, such as security, fault tolerance, access interface singularity, and network constraints, both in terms of latency and bandwidth. In this scenario, the performance of communications depends both on the network fabric and its efficient support in virtualized environments, which ultimately determines the overall system performance. To solve the current network constraints in cloud services, their providers are deploying high-speed networks, such as 10 Gigabit Ethernet. This paper presents an evaluation of high-performance computing message-passing middleware on a cloud computing infrastructure, Amazon EC2 cluster compute instances, equipped with 10 Gigabit Ethernet. The analysis of the experimental results, confronted with a similar testbed, has shown the significant impact that virtualized environments still have on communication performance, which demands more efficient communication middleware support to get over the current cloud network limitations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. A Message Passing Interface Standard (MPI) http://www.mcs.anl.gov/research/projects/mpi/. Accessed July 2012

  2. Abramson D et al (2006) Intel virtualization technology for directed I/O. Intel Technol J 10(3):179–192

    MathSciNet  Google Scholar 

  3. Advanced Micro Devices (AMD) (2009) I/O Virtualization Technology (IOMMU). http://support.amd.com/us/Processor_TechDocs/34434-IOMMU-Rev_1.26_2-11-09.pdf. Accessed July 2012

  4. Advanced Micro Devices (AMD) Virtualization Technology (AMD-V). http://sites.amd.com/us/business/it-solutions/virtualization/Pages/amd-v.aspx. Accessed July 2012

  5. Baek SJ, Park SM, Yang SH, Song EH, Jeong YS (2010) Efficient server virtualization using grid service infrastructure. J Inf Process Syst 6(4):553–562

    Google Scholar 

  6. Bailey DH et al (1991) The NAS parallel benchmarks. Int J High Perform Comput Appl 5(3):63–73

    Article  Google Scholar 

  7. Baker M, Carpenter B (2000) MPJ: a proposed Java message passing API and environment for high performance computing. In: Proceedings of 15th IPDPS workshops on parallel and distributed processing (IPDPS’00), Cancun, LNCS, vol 1800, pp 552–559

  8. Barham P, Dragovic B, Fraser K, Hand S, Harris TL, Ho A, Neugebauer R, Pratt I, Warfield A (2003) Xen and the art of virtualization. In: Proceedings of 19th ACM symposium on operating systems principles (SOSP’03), Bolton Landing (Lake George), pp 164–177

  9. Buntinas D, Mercier G, Gropp W (2006) Design and evaluation of nemesis, a scalable, low-latency, message-passing communication subsystem. In: Proceedings of 6th IEEE international symposium on cluster computing and the grid (CCGRID’06), Singapore, pp 521–530

  10. Carpenter B, Fox G, Ko S, Lim S (2002) mpiJava 1.2: API specification. http://www.hpjava.org/reports/mpiJava-spec/mpiJava-spec/mpiJava-spec.html. Accessed July 2012

  11. Fraser K, Hand S, Neugebauer R, Pratt I, Warfield A, Williamson M (2004) Safe hardware access with the Xen virtual machine monitor. In: Proceedings of 1st workshop on operating system and architectural support for the on demand IT infrastructure (OASIS’04), Boston

  12. Gabriel E et al (2004) Open MPI: goals, concept, and design of a next generation MPI implementation. In: Proceedings of 11th European PVM/MPI users’ group meeting, Budapest, pp 97–104

  13. Huang B, Bauer M, Katchabaw M (2005) Hpcbench—a Linux-based network benchmark for high performance networks. In: Proceedings of 19th international symposium on high performance computing systems and applications (HPCS’05), Guelph, pp 65–71

  14. Huang W, Liu J, Abali B, Panda DK (2006) A case for high performance computing with virtual machines. In: Proceedings of 20th international conference on supercomputing (ICS’06), Cairns, pp 125–134

  15. Huang W, Koop MJ, Gao Q, Panda DK (2007) Virtual machine aware communication libraries for high performance computing. In: Proceedings of ACM/IEEE conference on supercomputing (SC'07), Reno, pp 1–12

  16. Intel Corporation (2006) Virtualization technology (Intel VT). http://www.intel.com/technology/virtualization/technology.htm?iid=tech_vt+tech. Accessed July 2012

  17. Jones T (2009) Linux virtualization and PCI passthrough. http://www.ibm.com/developerworks/linux/library/l-pci-passthrough/. Accessed July 2012

  18. Karna AK, Zou H (2010) Cross comparison on C compilers reliability impact. J Convergence 1(1):65–74

    Google Scholar 

  19. Liu J, Huang W, Abali B, Panda DK (2006) High performance VMM-bypass I/O in virtual machines. In: Proceedings of USENIX’06 annual technical conference, Boston, pp 29–42

  20. Liu ST, Chen YM (2011) Retrospective detection of malware attacks by cloud computing. Int J Inf Technol Commun Convergence 1(3):280–296

    Article  Google Scholar 

  21. Amazon Web Services LLC (AWS LLC) Amazon elastic compute cloud (Amazon EC2). http://aws.amazon.com/ec2. Accessed July 2012

  22. Amazon Web Services LLC (AWS LLC) High performance computing using Amazon EC2. http://aws.amazon.com/ec2/hpc-applications/. Accessed July 2012

  23. Mallón DA, Taboada GL, Touriño J, Doallo R (2009) NPB-MPJ: NAS parallel benchmarks implementation for message-passing in Java. In: Proceedings of 17th Euromicro international conference on parallel, distributed, and network-based processing (PDP’09), Weimar, pp 181–190

  24. Mansley K, Law G, Riddoch D, Barzini G, Turton N, Pope S (2007) Getting 10 Gb/s from Xen: safe and fast device access from unprivileged domains. In: Proceedings of workshop on virtualization/Xen in high-performance cluster and grid computing (VHPC’07), Rennes, pp 224–233

  25. Mateescu G, Gentzsch W, Ribbens CJ (2011) Hybrid computing-where HPC meets grid and cloud computing. Future Gener Comput Syst 27(5):440–453

    Article  Google Scholar 

  26. MPICH2 (2005) High-performance and widely portable MPI. http://www.mcs.anl.gov/research/projects/mpich2/. Accessed July 2012

  27. Nanos A, Koziris N (2009) MyriXen: message passing in Xen virtual machines over Myrinet and Ethernet. In: Proceedings of 4th workshop on virtualization in high-performance cloud computing (VHPC’09), Delft, pp 395–403

  28. Raj H, Schwan K (2007) High performance and scalable I/O virtualization via self-virtualized devices. In: Proceedings of 16th international symposium on high performance distributed computing (HPDC'07), Monterey, pp 179–188

  29. Regola N, Ducom JC (2010) Recommendations for virtualization technologies in high performance computing. In: Proceedings of 2nd international conference on cloud computing technology and science (CloudCom’10), Indianapolis, pp 409–416

  30. Taboada GL, Ramos S, Touriño J, Doallo R (2011) Design of efficient java message-passing collectives on multi-core clusters. J Supercomput 55(2):126–154

    Article  Google Scholar 

  31. Taboada GL, Ramos S, Expósito RR, Touriño J, Doallo R (2012) Java in the high performance computing arena: research, practice and experience. Sci Comput Program (in press). doi:10.1016/j.scico.2011.06.002

  32. Taboada GL, Touriño J, Doallo R (2012) F-MPJ: scalable Java message-passing communications on parallel systems. J Supercomput 60(1):117–140

    Article  Google Scholar 

  33. Walker E (2008) Benchmarking Amazon EC2 for high-performance scientific computing. LOGIN: USENIX Mag 33(5):18–23

    Google Scholar 

  34. Wang G, Ng TSE (2010) The impact of virtualization on network performance of Amazon EC2 data center. In: Proceedings of 29th conference on information communications (INFOCOM'10), San Diego, pp 1163–1171

  35. Wang X, Sang Y, Liu Y, Luo Y (2011) Considerations on security and trust measurement for virtualized enviroment. J Convergence 2(2):19–24

    Google Scholar 

  36. Whitaker A, Shaw M, Gribble SD (2002) Denali: lightweight virtual machines for distributed and networked applications. Technical Report 02-02-01, University of Washington, USA

  37. Won C, Lee B, Park K, Kim MJ (2008) Eager data transfer mechanism for reducing communication latency in user-level network protocols. J Inf Process Syst 4(4):133–144

    Google Scholar 

  38. Xen Org (2005) Xen PCI passthrough. http://wiki.xensource.com/xenwiki/XenPCIpassthrough. Accessed July 2012

  39. Ye Y, Li X, Wu B, Li Y (2011) A comparative study of feature weighting methods for document co-clustering. Int J Inf Technol Commun Convergence 1(2):206–220

    Article  Google Scholar 

Download references

Acknowledgments

This work was funded by the Ministry of Science and Innovation of Spain under Project TIN2010-16735, by the Galician Government (Consolidation of Competitive Research Groups, Xunta de Galicia ref. 2010/6), by an FPU Grant AP2010-4348 and by an Amazon Web Services (AWS) LLC research grant.

Author information

Authors and Affiliations

  1. Department of Electronics and Systems, University of A Coruña, A Coruña, Spain

    Roberto R. Expósito, Guillermo L. Taboada, Sabela Ramos, Juan Touriño & Ramón Doallo

Authors
  1. Roberto R. Expósito
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guillermo L. Taboada
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sabela Ramos
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Juan Touriño
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ramón Doallo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Roberto R. Expósito.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Expósito, R.R., Taboada, G.L., Ramos, S. et al. Evaluation of messaging middleware for high-performance cloud computing. Pers Ubiquit Comput 17, 1709–1719 (2013). https://doi.org/10.1007/s00779-012-0605-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-012-0605-3

Keywords

Search

Navigation