Skip to main content
SpringerLink
Log in

A Novel Metaheuristic Based Method for Software Mutation Test Using the Discretized and Modified Forrest Optimization Algorithm

  • Published:
Journal of Electronic Testing Aims and scope Submit manuscript

Abstract

The number of detected bugs by software test data determines the efficacy of the test data. One of the most important topics in software engineering is software mutation testing, which is used to evaluate the efficiency of software test methods. The syntactical modifications are made to the program source code to make buggy (mutated) programs, and then the resulting mutants (buggy programs) along with the original programs are executed with the test data. Mutation testing has several drawbacks, one of which is its high computational cost. Higher execution time of mutation tests is a challenging problem in the software engineering field. The major goal of this work is to reduce the time and cost of mutation testing. Mutants are inserted in each instruction of a program using typical mutation procedures and tools. Meanwhile, in a real-world program, the likelihood of a bug occurrence in the simple and non-bug-prone sections of a program is quite low. According to the 80–20 rule, 80 percent of a program's bugs are discovered in 20% of its fault-prone code. The first stage of the suggested solution uses a discretized and modified version of the Forrest optimization algorithm to identify the program's most bug-prone paths; the second stage injects mutants just in the identified bug-prone instructions and data. In the second step, the mutation operators are only injected into the identified instructions and data that are bug-prone. Studies on standard benchmark programs have shown that the proposed method reduces about 27.63% of the created mutants when compared to existing techniques. If the number of produced mutants is decreased, the cost of mutation testing will also decrease. The proposed method is independent of the platform and testing tool. The results of the experiments confirm that the use of the proposed method in each testing tool such as Mujava, Muclipse, Jester, and Jumble makes a considerable mutant reduction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22

Similar content being viewed by others

Data Availability

The datasets generated during and the implemented code during the current study are available in the google.drive and can be freely accessed by the following link: https://drive.google.com/drive/folders/1eHkLdF2b-of6LqgJQaEpcyLjcfejz0Az?usp=sharing.

References

  1. Acree AT, Budd TA, DeMillo RA, Lipton RJ, Sayward FG (1980) Mutation Analysis. School of Information and Computer Science, Georgia Institute of Technology

  2. Aghdam ZK, Arasteh B (2017) An efficient method to generate test data for software structural testing using artificial bee colony optimization algorithm. Int J Software Eng Knowl Eng 27(6):2017

    Article  Google Scholar 

  3. Arasteh B, Hosseini SMJ (2022) Traxtor: An Automatic Software Test Suit Generation Method Inspired by Imperialist Competitive Optimization Algorithms. J Electron Test. https://doi.org/10.1007/s10836-022-05999-9

    Article  Google Scholar 

  4. Arasteh B, Imanzadeh P, Arasteh K et al (2022) A Source-code Aware Method for Software Mutation Testing Using Artificial Bee Colony Algorithm. J Electron Test 38:289–302. https://doi.org/10.1007/s10836-022-06008-9

    Article  Google Scholar 

  5. Arasteh B (2019) ReDup: A software-based method for detecting soft-error using data analysis. Comp Electrical Eng 78(September 2019):89–107

  6. Barbosa EF, Maldonado JC, Vincenzi AMR (2001) Toward the determination of sufficient mutant operators for C. Software Testing, Verification and Reliability 11(2):113–136

    Article  Google Scholar 

  7. Bouyer A, Arasteh B, Movaghar A (2007) A New Hybrid Model Using Case-Based Reasoning and Decision Tree Methods for Improving Speedup and Accuracy. IADIS International Conference of Applied Computing

  8. Budd TA (1980) Mutation Analysis of Program Test Data. Yale University

  9. Chandra SSV, Sankar SS, Anand HS (2022) Smell Detection Agent Optimization Approach to Path Generation in Automated Software Testing. J Electron Test. https://doi.org/10.1007/s10836-022-06033-8

    Article  Google Scholar 

  10. Delgado-Pérez P, Medina-Bulo I (2018) Search-based mutant selection for efficient test suite improvement: Evaluation and results. Inf Softw Technol 104(2018):130–143

    Article  Google Scholar 

  11. Deng L, Offutt J, Ammann P, Mirzaei N (2017) Mutation operators for testing Android apps. Inf Softw Technol 81(2017):154–168

    Article  Google Scholar 

  12. Dominguez-Jimenez JJ, Estero-Botaro A, Garcia-Dominguez A, Medina-Bulo I (2011) Evolutionary mutation testing. Inf Softw Technol 53(10):1108–1123

    Article  Google Scholar 

  13. Fenton NE, Ohlsson N (2000) Quantitative analysis of faults and failures in a complex software system. IEEE Transact Software Eng 26(8):797–814

  14. Ghaemi A, Arasteh B (2020) SFLA-based heuristic method to generate software structural test data. J Softw Evol Proc 32:e2228. https://doi.org/10.1002/smr.2228

  15. Gheyi R, Ribeiro M, Souza B, Guimarães M, Fernandes L, d’Amorim M, Alves V, Teixeira L, Fonseca B (2021) Identifying method-level mutation subsumption relations using Z3. Inf Softw Technol 132:106496

    Article  Google Scholar 

  16. Hosseini S, Arasteh B, Isazadeh A, Mohsenzadeh M, Mirzarezaee M (2021) An error-propagation aware method to reduce the software mutation cost using genetic algorithm. Data Technologies and Applications 55(1):118–148. https://doi.org/10.1108/DTA-03-2020-0073

    Article  Google Scholar 

  17. Howden WE (1982) Weak mutation testing and completeness of test sets. IEEE Trans Software Eng 8(4):371–379

    Article  Google Scholar 

  18. Jackson D, Woodward MR (2000) Parallel firm mutation of Java programs. Proc. First Workshop on Mutation Analysis, pp 55–61

    Google Scholar 

  19. Jia Y, Harman M (2011) An Analysis and Survey of the Development of Mutation Testing. IEEE Trans Software Eng 37(5):649–678. https://doi.org/10.1109/tse.2010.62

    Article  Google Scholar 

  20. Keshtgar A, Arasteh B (2017) Enhancing Software Reliability against Soft-Error using Minimum Redundancy on Critical Data. https://doi.org/10.5815/ijcnis.2017.05.03

  21. King KN, Offutt AJ (1991) A Fortran language system for mutation-based software testing. Software: Practice and Experience 21(7):685–718

  22. Kintis M, Papadakis M, Malevris N (2010) Evaluating mutation testing alternatives: a collateral experiment. Proceedings of the 17th Asia-Pacific Software Engineering Conference (APSEC)

  23. Kurtz B, Ammann P, Delamaro ME, Offutt J, Deng L (2014) Mutant subsumption graphs. 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation Workshops (ICSTW)

  24. Kurtz B, Ammann P, Offutt J (2015) Static analysis of mutant subsumption. IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW)

  25. Ma YS, Offutt J, Kwon YR (2006) MuJava: A Mutation System for Java. In 28th International Conference on Software Engineering (ICSE ’06)

  26. Malevris N, Yates D (2006) The collateral coverage of data flow criteria when branch testing. Inf Softw Technol 48(8):676–686

    Article  Google Scholar 

  27. Manizheh G, Mohammad-Reza F (2014) Forest Optimization Algorithm. Expert Syst Appl 41(15):6676–6687

    Article  Google Scholar 

  28. Offutt AJ, Lee A, Rothermel G, Untch RH, Zapf C (1996) An experimental determination of sufficient mutant operators. ACM Trans Softw Eng Methodol 5(2):99–118

    Article  Google Scholar 

  29. Offutt AJ, Rothermel G, Zapf C (1993) An experimental evaluation of selective mutation. Proceedings of the 15th International Conference on Software Engineering, ICSE ’93. IEEE Computer Society Press, Los Alamitos, CA

  30. Papadakis M, Malevris N (2010) An empirical evaluation of the first and second order mutation testing strategies. 2010 Third International Conference on Software Testing, Verification, and Validation Workshops (ICSTW)

  31. Sankar SS, Chandra SS (2020a) A Structural Testing Model Using SDA Algorithm. Lect Notes Comput Sci 405–412. https://doi.org/10.1007/978-3-030-53956-6_36

  32. Sankar SS, Chandra SS (2020b) An Ant Colony Optimization Algorithm Based Automated Generation of Software Test Cases. Lect Notes Comput Sci 231–239. https://doi.org/10.1007/978-3-030-53956-6_21

  33. Wei C, Yao X, Gong D, Liu H (2021) Spectral clustering based mutant reduction for mutation testing. Inf Softw Technol 132(2021):106502

  34. Wong WE (1993) On mutation and data flow. Ph.D. dissertation, Purdue University

  35. Yao X, Zhang G, Pan F, Gong D, Wei C (2020) Orderly Generation of Test Data via Sorting Mutant Branches Based on Their Dominance Degrees for Weak Mutation Testing. IEEE Trans Software Eng 48(4):1169–1184. https://doi.org/10.1109/tse.2020.3014960

    Article  Google Scholar 

  36. Zhang L, Hou SS, Hu JJ, Xie T, Mei H (2010) Is operator-based mutant selection superior to random mutant selection? Proceedings of the 32Nd ACM/IEEE International Conference on Software Engineering

  37. Zhang L, Gligoric M, Marinov D, Khurshid S (2013) Operator-based and random mutant selection: better together. Automated Software Engineering (ASE). IEEE/ACM 28th International Conference

Download references

Author information

Authors and Affiliations

  1. Department of Software Engineering, Faculty of Engineering and Natural Science, Istinye University, Istanbul, Turkey

    Bahman Arasteh

  2. Department of Computer, Urmia Branch, Islamic Azad University, Urmia, Iran

    Farhad Soleimanian Gharehchopogh

  3. Computer Education and Instructional Technology, Yıldız Technical University, Istanbul, Turkey

    Peri Gunes

  4. Computer Engineering Department, Faculty of Engineering, Fatih Sultan Mehmet Vakif University, 34445, Istanbul, Turkey

    Farzad Kiani

  5. Computer Engineering Department, Faculty of Engineering and Architecture, Nisantasi University, 34398, Istanbul, Turkey

    Mahsa Torkamanian-Afshar

Authors
  1. Bahman Arasteh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Farhad Soleimanian Gharehchopogh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peri Gunes
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Farzad Kiani
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mahsa Torkamanian-Afshar
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study's conception and design. The data collection, and analysis were performed by Bahman Arasteh. Experiments have been performed by Bahman Arasteh. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Bahman Arasteh.

Ethics declarations

Conflict of Interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Additional information

Responsible Editor: Y. K. Malaiya

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Arasteh, B., Gharehchopogh, F.S., Gunes, P. et al. A Novel Metaheuristic Based Method for Software Mutation Test Using the Discretized and Modified Forrest Optimization Algorithm. J Electron Test 39, 347–370 (2023). https://doi.org/10.1007/s10836-023-06070-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10836-023-06070-x

Keywords

Search

Navigation