Wyao Matcha
ABSTRACT
This paper aims at investigating the use of deep learning to suggest (prioritize) classes to be tested rigorously during unit testing of object-oriented systems. We relied on software unit testing information history and source code metrics. We conducted an empirical study using data collected from two Apache open-source Java software systems (POI and ANT). For each software system, we extracted the source code of five different versions. For each version, we collected various metrics from the source code of the Java classes. Then, for all software classes, we extracted testing coverage measures at instruction and method levels of granularity. We used the existing JUnit test cases developed for these systems. Based on the different datasets we collected, we trained several deep neural network models. We validated the obtained classifiers using four validation techniques: (1) CV: Cross Version validation, (2) CPV: Combined Previous Version validation, (3) CSPV: Combined System and Previous Version validation, and (4) LOSO: Leave One System Out validation. The obtained results in terms of classifiers’ performance vary between 70% and 80% of accuracy and strongly support the viability of our approach.
- B. Boehm, "A Spiral Model of Software Development and Enhancement", Proc. Int'l Workshop Software Process and Software Environments, ACM Press, 1985; also, in ACM Software Eng. Notes, Aug. 1986, pp. 22-42.Google Scholar
- B. Boehm. "Software Engineering Economics". Prentice Hall, Englewood Cliffs, NJ, ISBN-10: 0138221227, edition 01 oct 1981.Google Scholar
- S. Elbaum, A. G. Malishevsky and G. Rothermel, "Test Case Prioritization: A Family of Empirical Studies", IEEE Transactions Software Engineering, Vol. 28, No. 2, pp.159-182, 2002.Google ScholarDigital Library
- S. Elbaum, A. G. Malishevsky and G. Rothermel, "Prioritizing test cases for regression testing". Proc. ACM SIGSOFT Int. Symp. On Software Testing and Analysis (ISSTA), Portland, OR, USA, 22-25 August 2000, pp. 102-12Google ScholarDigital Library
- G. Rothermel, R.H Untch, C. Chu and M.J Harrold. "Test case prioritization: an empirical study", International Conference on Software Maintenance, Oxford, UK, pp. 179–188.,1999.Google Scholar
- J. Kim, and A. Porter, "A history-based test prioritization technique for regression testing in resource constrained environments", In Proceedings of the International Conference on Software Engineering, 2002.Google ScholarDigital Library
- F. Toure., M. Badri and L. Lamontagne, "Investigating the Prioritization of Unit Testing Effort Using Software Metrics", In Proceedings of the 12th International Conference on Evaluation of Novel Approaches to Software Engineering (ENASE’17) Volume 1: ENASE, pages 69-80, 2017.Google ScholarDigital Library
- Chidamber S.R. and Kemerer C.F., "A Metrics Suite for Object Oriented Design", IEEE Transactions on Software Engineering, vol. 20, no. 6, pp. 476–493, 1994.Google ScholarDigital Library
- M. Bruntink, and A.V. Deursen,"Predicting Class Testability using Object-Oriented Metrics", 4th Int. Workshop on Source Code Analysis and Manipulation (SCAM), IEEE, 2004.Google ScholarDigital Library
- V. Gupta, K. K. Aggarwal and Y. Singh, "A Fuzzy Approach for Integrated Measure of Object-Oriented Software Testability", Journal of Computer Science, Vol. 1, No. 2, 2005, pp. 276-282. doi:10.3844/jcssp.2005.276.282.Google ScholarCross Ref
- M. Bruntink and A. Van Deursen, "Predicting class testability using object-oriented metrics", in Proceedings of the 4th IEEE International Workshop on Source Code Analysis and Manipulation (SCAM ’04), pp. 136–145, September 2004.Google ScholarDigital Library
- M. Bruntink and A. van Deursen, "An empirical study of class testability", Journal of Systems and Software, vol. 79, no. 9, pp. 1219– 1232, 2006.Google ScholarDigital Library
- JUnit Framework, https://junit.org/junit5/. Visited in October 2020.Google Scholar
- K. Jarrett, K. Kavukcuoglu, M. A. Ranzato, and Y. LeCun. "What is the best multi-stage architecture for object recognition?" In International Conference on Computer Vision, pages 2146–2153. IEEE, 2009.Google Scholar
- Y. LeCun, K. Kavukcuoglu, and C. Farabet, Convolutional networks and applications in vision. In Circuits and Systems (ISCAS), Proceedings of 2010 IEEE International Symposium on, pages 253– 256. IEEE, 2010.Google Scholar
- Apache POI releases, https://github.com/apache/poi/releases, Visited in October 2020.Google Scholar
- Apache ANT releases, https://github.com/apache/ant/releases, Visited in October 2020.Google Scholar
- Williams, B. S. a. L. (2008). "A Survey on Code Coverage as a Stopping Criterion for Unit Testing.", Technical report (North Carolina State University. Dept. of Computer Science), TR-2008-22.Google Scholar
- S. Mirarab, A. Hassouna, and L. Tahvildar, "Using Bayesian belief networks to predict change propagation in software systems" in Proceedings of the 15th IEEE International Conference on Program Comprehension, pages 177-188, 2007.Google Scholar
- Lazić, L.: Software Testing Optimization by Advanced Quantitative Defect Management. Computer Science and Information Systems, Vol. 7, No. 3, 459-487, https://doi.org/10.2298/CSIS090923008L, (2010).Google Scholar
- G. Rothermel, M. J. Harrold, J. Ronne and C. Hong, “Empirical Studies of Test‐Suite Reduction”, In Journal of Software Testing, Verification, and Reliability, Vol. 12, No.4, 2002.Google ScholarCross Ref
- W. Wong, J. Horgan, S. London, and H. Agrawal, “A study of effective regression in practice”, Proceedings of the 8th International Symposium on Software Reliability Engineering, November, p.230–238, 1997.Google ScholarCross Ref
- R. Carlson, A. Denton, “A clustering approach to improving test case prioritization, An industrial case study”, 27th IEEE International Conference on Software Maintenance (ICSM), 2011.Google ScholarDigital Library
- H. Spieker, A. Gotlieb, D. Marijan and M. Mossige, "Reinforcement learning for automatic test case prioritization and selection in continuous integration", Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis, July 2017.Google ScholarDigital Library
- W. Matcha, F. Touré, M. Badri, L. Badri, Using Deep Learning Classifiers to Identify Candidate Classes for Unit Testing in Object-Oriented Systems. SEKE 2020: 353-358.Google Scholar
- Github Repository, https://github.com/apache/ant, Visited in October 2020.Google Scholar
- IntelliJ IDE, https://www.jetbrains.com/idea/, Visited in October 2020.Google Scholar
- Code Mr plugin, https://plugins.jetbrains.com/plugin/10811-codemr/, Visited in October 2020.Google Scholar
- Code-Coverage plugin, https://www.jetbrains.com/help/idea/codecoverage.html, Visited in October 2020.Google Scholar
- Artificial Intelligence and life in 2030, one-hundred-year study on artificial Intelligence, report of the 2015 Study panel, September 2016Google Scholar
- Y. LeCun, Y. Bengio, G, Hinton. "Deep learning”. Nature. 2015;521(7553):436-444. doi:10.1038/nature14539Google ScholarCross Ref
- F. Rosenblatt, "The perceptron: A probabilistic model for information storage and Organization in the brain", in cognitive systems. Buffalo: Cornell Aeronautical Laboratory, Inc. Rep. No. VG-1196-G-1, 1958.Google Scholar
- M. Minsky, and S. Papert, “Perceptrons: An Introduction to Computational Geometry,” MIT Press, expanded edition, ISBN-10: 0262631113, décembre 1987Google Scholar
- D. E. Rumelhart, G. Hinton, and R. J. Williams, "Learning representations by back-propagating errors", Cognitive modeling 5.3 (1988):Google Scholar
- Tensorflow: https://www.tensorflow.org/Google Scholar
- Kera: https://keras.io/Google Scholar
- Panda : https://pandas.pydata.orgGoogle Scholar
Index Terms
- Identifying Candidate Classes for Unit Testing Using Deep Learning Classifiers: An Empirical Validation
Recommendations
Smart Unit Testing Framework
ISSREW '12: Proceedings of the 2012 IEEE 23rd International Symposium on Software Reliability Engineering WorkshopsUnit testing(UT) is an important step in ensuring the quality of software. Considerable effort is spent in unit testing. There are several frameworks to help with UT. Some common frameworks are Cunit, Junit, Nunit etc. All of these have very similar ...
Investigating the Prioritization of Unit Testing Effort using Software Metrics
ENASE 2017: Proceedings of the 12th International Conference on Evaluation of Novel Approaches to Software EngineeringIn object-oriented software, unit testing is a level of software testing where each individual class is tested by a dedicated unit test class. Unfortunately, due to time and resources constraints, this phase does not cover all classes. The testing ...
Beyond Accuracy: An Empirical Study on Unit Testing in Open-source Deep Learning Projects
Deep Learning (DL) models have rapidly advanced, focusing on achieving high performance through testing model accuracy and robustness. However, it is unclear whether DL projects, as software systems, are tested thoroughly or functionally correct when ...
Comments