Skip to main content
SpringerLink
Log in

Study and analysis of big data for characterization of user association in large scale

  • Original article
  • Published:
International Journal of System Assurance Engineering and Management Aims and scope Submit manuscript

Abstract

The volume and the data detail are increasing like social media, multimedia and internet of things produced huge data flow in structured and unstructured format. The academia, government, and industry have the great attention for data generation. The cloud computing is conjoined with the data and it provides the user ability to utilize the commodity computing for queries process through the several datasets and timely return of resultant set. The several serious challenges are produced by the amount of collected data such as transfer speed and security issues. Big Data is in its initial stage and it required to be classifies the various attributes of big data such as management and quick progression rate. The results show that the one big executer configuration performance is better as compared to the six small executer configuration. It is also observed that the more executer configuration increase the utilization of the resources and the high probability is led by the resource contention so there is negative impact on the system performance. The fewer amounts of data are produced and it performs better for other applications. The data size is not always reduced by enabling data compression. The data size is reduced up-to 72% by compression and memory serialization.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Ali A, Qadir J, ur Rasool R, Sathiaseelan A, Zwitter A, Crowcroft J (2016) Big data for development: applications and techniques. Big Data Anal 1(1):1–24

    Article  Google Scholar 

  • Aslam B, Javed AR, Chakraborty C, Nebhen J, Raqib S (2021) Blockchain and ANFIS empowered IoMT application for privacy preserved contact tracing in COVID-19 pandemic. Person Ubiquitous Comput 1–17

  • Blazquez D, Domenech J (2018) Big Data sources and methods for social and economic analyses. Technol Forecast Soc Chang 130:99–113

    Article  Google Scholar 

  • Bolouri H (2014) Modeling genomic regulatory networks with big data. Trends Genet 30(5):182–191

    Article  Google Scholar 

  • Braganza A, Brooks L, Nepelski D, Ali M, Moro R (2017) Resource management in big data initiatives: processes and dynamic capabilities. J Bus Res 70:328–337

    Article  Google Scholar 

  • Castiglione A, Gribaudo M, Iacono M, Palmieri F (2014) Exploiting mean field analysis to model performances of big data architectures. Futur Gener Comput Syst 37:203–211

    Article  Google Scholar 

  • Cheng N, Lyu F, Chen J, Xu W, Zhou H, Zhang S, Shen X (2018a) Big data driven vehicular networks. IEEE Network 32(6):160–167

    Article  Google Scholar 

  • Cheng Y, Chen K, Sun H, Zhang Y, Tao F (2018b) Data and knowledge mining with big data towards smart production. J Ind Inf Integr 9:1–13

    Google Scholar 

  • Chittaranjan G, Blom J, Gatica-Perez D (2013) Mining large-scale smartphone data for personality studies. Pers Ubiquit Comput 17(3):433–450

    Article  Google Scholar 

  • Dash S, Chakraborty C, Giri SK, Pani SK, Frnda J (2021) BIFM: Big-data driven intelligent forecasting model for COVID-19. IEEE Access 9:97505–97517

    Article  Google Scholar 

  • Dhawan S, Chakraborty C, Frnda J, Gupta R, Rana AK, Pani SK (2021) SSII: secured and high-quality steganography using intelligent hybrid optimization algorithms for IoT. IEEE Access 9:87563–87578

    Article  Google Scholar 

  • Diamantoulakis PD, Kapinas VM, Karagiannidis GK (2015) Big data analytics for dynamic energy management in smart grids. Big Data Res 2(3):94–101

    Article  Google Scholar 

  • Dinov ID, Heavner B, Tang M, Glusman G, Chard K, Darcy M, Toga AW (2016) Predictive big data analytics: a study of Parkinson’s disease using large, complex, heterogeneous, incongruent, multi-source and incomplete observations. PLoS ONE 11(8):e0157077

    Article  Google Scholar 

  • Dogra J, Jain S, Sharma A, Kumar R, Sood M (2020) Brain tumor detection from MR images employing fuzzy graph cut technique. Recent Adv Comput Sci Commun (form: Rec Patents Comput Sci) 13(3):362–369

    Article  Google Scholar 

  • Gandomi A, Haider M (2015) Beyond the hype: Big data concepts, methods, and analytics. Int J Inf Manag 35(2):137–144

    Article  Google Scholar 

  • Gupta S, Kar AK, Baabdullah A, Al-Khowaiter WA (2018) Big data with cognitive computing: a review for the future. Int J Inf Manage 42:78–89

    Article  Google Scholar 

  • Hashem IAT, Yaqoob I, Anuar NB, Mokhtar S, Gani A, Khan SU (2015) The rise of “big data” on cloud computing: review and open research issues. Inf Syst 47:98–115

    Article  Google Scholar 

  • Hassani H, Huang X, Silva E (2018) Digitalisation and big data mining in banking. Big Data Cognit Comput 2(3):18

    Article  Google Scholar 

  • Hinkson IV, Davidsen TM, Klemm JD, Chandramouliswaran I, Kerlavage AR, Kibbe WA (2017) A comprehensive infrastructure for big data in cancer research: accelerating cancer research and precision medicine. Front Cell Dev Biol 5:83

    Article  Google Scholar 

  • Hu Y, Fang Z, Yang Y, Rohlsen-Neal D, Cheng F, Wang J (2018) Analyzing the genes related to nicotine addiction or schizophrenia via a pathway and network based approach. Sci Rep. https://doi.org/10.1038/s41598-018-21297-x

    Article  Google Scholar 

  • Huang T, Lan L, Fang X, An P, Min J, Wang F (2015) Promises and challenges of big data computing in health sciences. Big Data Research 2(1):2–11

    Article  Google Scholar 

  • Kalaimani S, Mala R (2016) Characterization of user inclinations for service recommender system in Big Data applications. Int J Innov Res Sci Eng Technol ISSN (Online) 2319–8753

  • Kumar D, Sharma A, Kumar R, Sharma N (2019) Restoration of the network for next generation (5G) optical communication network. In: 2019 international conference on signal processing and communication (ICSC). IEEE, pp 64–68

  • Li S, Dragicevic S, Castro FA, Sester M, Winter S, Coltekin A, Cheng T (2016) Geospatial big data handling theory and methods: a review and research challenges. ISPRS J Photogram Remote Sens 115:119–133

    Article  Google Scholar 

  • Lido C, Reid K, Osborne M (2019) Lifewide learning in the city: novel big data approaches to exploring learning with large-scale surveys, GPS, and social media. Oxf Rev Educ 45(2):279–295

    Article  Google Scholar 

  • Lyu F, Fang L, Xue G, Xue H, Li M (2019a) Large-scale full wifi coverage: deployment and management strategy based on user spatio-temporal association analytics. IEEE Internet Things J 6(6):9386–9398

    Article  Google Scholar 

  • Lyu F, Ren J, Cheng N, Yang P, Li M, Zhang Y, Shen X (2019) Big data analytics for user association characterization in large-scale wifi system. [Paper presentation]. In: 2019 IEEE international conference on communications (ICC), pp 1–6, Shanghai, China

  • Malik M, Rafatirad S, Homayoun H (2018) System and architecture level characterization of big data applications on big and little core server architectures. ACM Trans Model Perform Eval Comput Syst 3(3):1–32. https://doi.org/10.1145/3229049

    Article  Google Scholar 

  • Massaro A, Vitti V, Galiano A, Morelli A (2019) Business intelligence improved by data mining algorithms and big data systems: an overview of different tools applied in industrial research. Comput Sci Inf Technol 7(1):1–21

    Google Scholar 

  • Medeiros DS, Neto HNC, Lopez MA, Magalhães LCS, Fernandes NC, Vieira AB, Mattos DM (2020) A survey on data analysis on large-Scale wireless networks: online stream processing, trends, and challenges. J Internet Serv Appl 11(1):1–48

    Article  Google Scholar 

  • Mehta N, Pandit A (2018) Concurrence of big data analytics and healthcare: a systematic review. Int J Med Inform 114:57–65

    Article  Google Scholar 

  • Menandas JJ, Joshi JJ (2014) Data mining with parallel processing technique for complexity reduction and characterization of big data. Glob J Adv Res 1:68–80

    Google Scholar 

  • Mikalef P, Boura M, Lekakos G, Krogstie J (2019) Big data analytics and firm performance: findings from a mixed-method approach. J Bus Res 98:261–276

    Article  Google Scholar 

  • Panigrahi BK, Trivedi MC, Mishra KK, Tiwari S, Singh PK (2017) Smart innovations in communication and computational sciences. In: Proceedings of ICSICCS, 1

  • Pathak AR, Pandey M, Rautaray S (2018) Construing the big data based on taxonomy, analytics and approaches. Iran J Comput Sci 1(4):237–259

    Article  Google Scholar 

  • Poongodi M, Sharma A, Vijayakumar V, Bhardwaj V, Sharma AP, Iqbal R, Kumar R (2020) Prediction of the price of Ethereum blockchain cryptocurrency in an industrial finance system. Comput Electr Eng 81:106527

    Article  Google Scholar 

  • Raghupathi W, Raghupathi V (2014) Big data analytics in healthcare: promise and potential. Health Inf Sci Syst 2(1):1–10

    Article  Google Scholar 

  • Ratta P, Kaur A, Sharma S, Shabaz M, Dhiman G (2021) Application of Blockchain and internet of things in healthcare and medical sector: applications, challenges, and future perspectives. J Food Qual. https://doi.org/10.1155/2021/7608296

    Article  Google Scholar 

  • Sang J, Gao Y, Bao BK, Snoek C, Dai Q (2015) Recent advances in social multimedia big data mining and applications

  • Shafiq MZ, Ji L, Liu AX, Pang J, Wang J (2013) Large-scale measurement and characterization of cellular machine-to-machine traffic. IEEE/ACM Trans Netw 21(6):1960–1973

    Article  Google Scholar 

  • Sharma A, Kumar R (2017) A framework for pre-computated multi-constrained quickest QoS path algorithm. J Telecommun Electron Comput Eng (JTEC) 9(3–6):73–77

    Google Scholar 

  • Sharma A, Kumar R (2019) Service-level agreement—energy cooperative quickest ambulance routing for critical healthcare services. Arab J Sci Eng 44(4):3831–3848

    Article  Google Scholar 

  • Sharma A, Kumar R, Bajaj RK (2021) On Energy-constrained quickest path problem in green communication using intuitionistic trapezoidal fuzzy numbers. Recent Adv Comput Sci Commun (formerly: Recent Patents Comput Sci) 14(1):192–200

    Article  Google Scholar 

  • Singh PK, Noor A, Kolekar MH, Tanwar S, Bhatnagar RK, Khanna S (eds) (2021) Evolving technologies for computing, communication and smart world: proceedings of ETCCS 2020, vol 694. Springer Nature

  • Sivarajah U, Kamal MM, Irani Z, Weerakkody V (2017) Critical analysis of Big Data challenges and analytical methods. J Bus Res 70:263–286

    Article  Google Scholar 

  • Tsai CW, Lai CF, Chao HC, Vasilakos AV (2015) Big data analytics: a survey. J Big Data 2(1):1–32

    Article  Google Scholar 

  • Wang B, Yao X, Jiang Y, Sun C, Shabaz M (2021) Design of a Real-time monitoring system for smoke and dust in thermal power plants based on improved genetic algorithm. J Healthc Eng. https://doi.org/10.1155/2021/7212567

    Article  Google Scholar 

  • Wu PY, Cheng CW, Kaddi CD, Venugopalan J, Hoffman R, Wang MD (2016) Omic and electronic health record big data analytics for precision medicine. IEEE Trans Biomed Eng 64(2):263–273

    Google Scholar 

  • Wu X, Zhu X, Wu G-Q, Ding W (2014) Data mining with big data. IEEE Trans Knowl Data Eng 26(1):97–107. https://doi.org/10.1109/tkde.2013.109

    Article  Google Scholar 

Download references

Funding

The present research work is self funded.

Author information

Authors and Affiliations

  1. School of Electronic Engineering, Xian University of Posts and Telecommunication, Xi’an, China

    Wei-wei Zhang

  2. Electronics and Communication Engineering Department, National Institute of Technology, Hamirpur, India

    Jyoti Bhola

  3. Institute of Engineering and Technology, Chitkara University, Punjab, India

    Rajeev Kumar & Nitin Saluja

Authors
  1. Wei-wei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jyoti Bhola
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rajeev Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nitin Saluja
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jyoti Bhola.

Ethics declarations

Conflict of interest

The author(s) declare(s) that there is no conflict of interest regarding the publication of this paper.

Research involving human participants and/or animals

This research do not involve Human or Animal participants for any kind of testing on them.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Ww., Bhola, J., Kumar, R. et al. Study and analysis of big data for characterization of user association in large scale. Int J Syst Assur Eng Manag 13 (Suppl 1), 375–384 (2022). https://doi.org/10.1007/s13198-021-01434-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13198-021-01434-y

Keywords

Search

Navigation