Skip to main content
SpringerLink
Log in

An Efficient Approach for MapReduce Result Verification

  • Conference paper
  • First Online:
Computational Intelligence, Cyber Security and Computational Models

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 412))

  • 1308 Accesses

Abstract

Hadoop follows a master-slave architecture and can process massive amount of data by using the MapReduce paradigm. The major problem associated with MapReduce is correctness of the results generated. Results can be altered and become wrong by the collaboration of malicious slave nodes. Credibility-based result verification is one of the effective methods to determine such malicious nodes and wrong results. The major limitation of the approach is that, it depends on the complete results of long-running jobs to identify malicious nodes and hence holds valuable resources. In this paper, we propose a new protocol called Intermediate Result Collection and Verification (IRCV) Protocol that prunes out unnecessary computations by collecting results for verification earlier in the execution line. In addition, unlike the previous approach, IRCV uses only a subset of nodes for the purpose. Our simulation experiments suggest that the new approach has improved performance and will lead to better utilization of resources.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 219.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 279.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    DataNodes are also termed as WorkerNodes.

  2. 2.

    https://hadoop.apache.org/docs/stable/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html #Example:_WordCount_v1.0.

References

  1. White, T.: Hadoop: the definitive guide. O’Reilly Media Inc. 1 (2004)

    Google Scholar 

  2. Wei, W., Du, J., Yu, T., Gu, X.: SecureMR: a service integrity assurance framework for MapReduce. In: Computer Security Applications Conference. ACSAC ‘09. Annual, pp. 73–82 (2009)

    Google Scholar 

  3. Samuel, T.A.; Nizar, M.A.: Credibility-based result verification for map-reduce. In: India Conference (INDICON), Annual IEEE, pp. 1–6, (2014)

    Google Scholar 

  4. HDFS Architecture Guide. http://hadoop.apache.org/docs/r1.2.1/hdfs_design.html

  5. Germain-Renaud, C., Monnier-Ragaigne, D.: Grid result checking. In: Proceedings of the 2nd Conference on Computing Frontiers (2005)

    Google Scholar 

  6. Huang, C., Zhu, S., Wu, D.: Towards trusted services: result verification schemes for MapReduce. cluster, cloud and grid computing (CCGrid). In: 12th IEEE/ACM International Symposium, pp. 41–48, (2012)

    Google Scholar 

  7. Zhao, S., Lo, V., Dickey, C.G.: Result verification and trust-based scheduling in peer-to-peer grids. In: Peer-to-Peer Computing, P2P 2005. Fifth IEEE International Conference, pp. 31–38, (2005)

    Google Scholar 

  8. Xiao, Z., Xiao, Y.: Accountable MapReduce in cloud computing. In: Computer Communications Workshops (INFOCOM WKSHPS), IEEE Conference, pp. 1082–1087, (2011)

    Google Scholar 

  9. Wang, Y., Wei, J.: Viaf: Verification-based integrity assurance framework for MapReduce. In: IEEE International Conference on Cloud Computing (CLOUD), pp. 300–307, (2011)

    Google Scholar 

  10. Grant, P.C.: Graduate School of Vanderbilt University, Masters thesis (2006)

    Google Scholar 

  11. Domingues, P., Sousa, B., Silva, L.M.: Sabotage tolerance and trust management in desktop grid computing. Future Gener. Comput. Syst. 23, 904–912 (2007)

    Google Scholar 

  12. Golle, P., Stubblebine, S.: Secure distributed computing in a commercial environment. In: Syverson, P. (ed.), Financial Cryptography 2339, 289–304 (2002)

    Google Scholar 

  13. Du, W., Jia, J., Mangal, M., Murugesan, M.: Uncheatable grid computing. In: Proceedings of the 24th International Conference on Distributed Computing Systems, pp. 4–11 (2004)

    Google Scholar 

  14. Wang, Y., Wei, J., Srivatsa, M.: Result integrity check for MapReduce computation on hybrid clouds. In: IEEE Sixth International Conference on Cloud Computing (CLOUD), pp. 847–854 (2013)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Engineering Trivandrum, Thiruvananthapuram, 695016, India

    K. Jiji & M. Abdul Nizar

Authors
  1. K. Jiji
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Abdul Nizar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to K. Jiji .

Editor information

Editors and Affiliations

  1. Applied Mathematics and Computational Sc, PSG College of Technology, Coimbatore, Tamil Nadu, India

    Muthukrishnan Senthilkumar

  2. Applied Mathematics and Computational Sc, PSG College of Technology, Coimbatore, Tamil Nadu, India

    Vijayalakshmi Ramasamy

  3. Applied Mathematics and Computational Sc, PSG College of Technology, Coimbatore, Tamil Nadu, India

    Shina Sheen

  4. Applied Mathematics and Computational Sc, PSG College of Technology, Coimbatore, Tamil Nadu, India

    C. Veeramani

  5. Department of Mathematics, Ryerson University, TORONTO, Ontario, Canada

    Anthony Bonato

  6. Fac. Science&Technology, Deakin University, Burwood, Australia

    Lynn Batten

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer Science+Business Media Singapore

About this paper

Cite this paper

Jiji, K., Abdul Nizar, M. (2016). An Efficient Approach for MapReduce Result Verification. In: Senthilkumar, M., Ramasamy, V., Sheen, S., Veeramani, C., Bonato, A., Batten, L. (eds) Computational Intelligence, Cyber Security and Computational Models. Advances in Intelligent Systems and Computing, vol 412. Springer, Singapore. https://doi.org/10.1007/978-981-10-0251-9_23

Download citation

  • DOI: https://doi.org/10.1007/978-981-10-0251-9_23

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-10-0250-2

  • Online ISBN: 978-981-10-0251-9

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation