Skip to main content
SpringerLink
Log in

Metadata Organization and Management for Globalization of Data Access with Onedata

  • Conference paper
  • First Online:
Parallel Processing and Applied Mathematics (PPAM 2015)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 9573))

Abstract

The Big Data revolution means that large amounts of data have not only to be stored, but also to be processed to unlock the potential of access to information and knowledge for scientific research. As a result, scientific communities require simple and convenient global access to data which is effective, secure and shareable. In this article we analyze how researchers use their data working in large scientific projects and show how their requirements may be satisfied with our solution called Onedata. Major technical mechanisms of metadata management and organization are described.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://onedata.org/.

  2. 2.

    http://syndicatedrive.com/.

  3. 3.

    http://storj.io/.

  4. 4.

    Filesystem in User Space, http://fuse.sourceforge.net.

References

  1. Hey, A.J.G., Tansley, S., Tolle, K.M.: The Fourth Paradigm: Data-intensive Scientific Discovery. Microsoft Research Redmond, Redmond (2009)

    Google Scholar 

  2. Mills, S., Lucas, S., Irakliotis, L., Data, D.B., et al.: A Practical Guide to Transforming the Business of Government. Technical report (2012). http://www.ibm.com/software/data/demystifying-big-data/

  3. Dutka, Ł., Słota, R., Wrzeszcz, M., Król, D., Kitowski, J.: Uniform and efficient access to data in organizationally distributed environments. In: Bubak, M., et al. (eds.) eScience on Distributed Computing Infrastructure. LNCS, vol. 8500, pp. 178–194. Springer, Heidelberg (2014)

    Chapter  Google Scholar 

  4. Oliveira, T., Mendes, R., Bessani, A.: Sharing files using cloud storage services. In: Lopes, L., et al. (eds.) Euro-Par 2014, Part II. LNCS, vol. 8806, pp. 13–25. Springer, Heidelberg (2014)

    Chapter  Google Scholar 

  5. Palankar, M.R., Iamnitchi, A., Ripeanu, M., Garfinkel, S.: Amazon S3 for science grids: a viable solution?. In: Proceedings of DADC, pp. 55–64. ACM (2008)

    Google Scholar 

  6. Thain, D., Livny, M.: Parrot: transparent user-level middleware for data-intensive computing. Scalable Comput. Pract. Experience 6(3), 9–18 (2005)

    Google Scholar 

  7. Hunich, D., Muller-Pfefferkorn, R.: Managing large datasets with iRODS - a performance analysis. In: Proceedings of IMCSIT, pp. 647–654. IEEE (2010)

    Google Scholar 

  8. Słota, R., Nikolow, D., Skalkowski, K., Kitowski, J.: Management of data access with quality of service in PL-Grid environment. Comput. Inform. 31(2), 463–479 (2012)

    Google Scholar 

  9. Nikolow, D.: Semantic-based storage QoS management methodology - case study for distributed environments. Comput. Inform. 31(6), 1345–1366 (2012)

    Google Scholar 

  10. Dutka, Ł., Wrzeszcz, M., Lichon, T., Słota, R., Zemek, K., Trzepla, K., Opioła, Ł., Słota, R., Kitowski, J.: Onedata - a step forward towards globalization of data access for computing infrastructures. Procedia Comput. Sci. 51, 2843–2847 (2015)

    Article  Google Scholar 

  11. Słota, R., Skitał, Ł., Nikolow, D., Kitowski, J.: Algorithms for automatic data replication in grid environment. In: Wyrzykowski, R., et al. (eds.) PPAM 2005. LNCS, vol. 3911, pp. 707–714. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  12. Słota, R., Nikolow, D., Skitał, Ł., Kitowski, J.: Implementation of replication methods in the grid environment. In: Sloot, P.M.A., et al. (eds.) EGC 2005. LNCS, vol. 3470, pp. 474–484. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

Download references

Acknowledgments

This research is supported partly by the European Regional Development Fund program no. POIG.02.03.00-12-137/13 as part of the PLGrid Core and AGH-UST grant no. 11.11.230.124.

Author information

Authors and Affiliations

  1. ACC Cyfronet AGH, AGH University, Kraków, Poland

    Michał Wrzeszcz, Krzysztof Trzepla, Rafał Słota, Konrad Zemek, Tomasz Lichoń, Łukasz Opioła, Darin Nikolow, Łukasz Dutka, Renata Słota & Jacek Kitowski

  2. Department of Computer Science, AGH University, Kraków, Poland

    Michał Wrzeszcz, Darin Nikolow, Renata Słota & Jacek Kitowski

Authors
  1. Michał Wrzeszcz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Krzysztof Trzepla
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rafał Słota
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Konrad Zemek
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tomasz Lichoń
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Łukasz Opioła
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Darin Nikolow
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Łukasz Dutka
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Renata Słota
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Jacek Kitowski
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Renata Słota .

Editor information

Editors and Affiliations

  1. Czestochowa University of Technolog, Czestochowa, Poland

    Roman Wyrzykowski

  2. Department of Computer Science, University of Southern California, Marina Del Rey, California, USA

    Ewa Deelman

  3. Electrical Engineering & Comput. Science, University of Tennessee, Knoxville, Tennessee, USA

    Jack Dongarra

  4. Czestochowa University of Technology, Institute of Computer & Information Sci., Czestochowa, Poland

    Konrad Karczewski

  5. Department of Computer Science, AGH University of Science and Technology, Krakow, Poland

    Jacek Kitowski

  6. Systèmes d’informations, Big Data et Rec, AGH University of Science and Technology, Krakow, Poland

    Kazimierz Wiatr

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Wrzeszcz, M. et al. (2016). Metadata Organization and Management for Globalization of Data Access with Onedata . In: Wyrzykowski, R., Deelman, E., Dongarra, J., Karczewski, K., Kitowski, J., Wiatr, K. (eds) Parallel Processing and Applied Mathematics. PPAM 2015. Lecture Notes in Computer Science(), vol 9573. Springer, Cham. https://doi.org/10.1007/978-3-319-32149-3_30

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-32149-3_30

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-32148-6

  • Online ISBN: 978-3-319-32149-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation