Raphael De Almeida
ABSTRACT
Dependencies in software are essential for efficiently structuring and modularizing the code, enabling software components to perform specific tasks, interact, and reuse functionalities. Furthermore, many applications also rely on external components, such as APIs and external services, which can difficult the creation of a testing environment. To overcome these challenges, developers can use mocks to simulate these dependencies, which becomes particularly useful when dealing with slow or hard-to-create dependencies, such as those requiring network access. Despite the use of mocks in software testing, there are few academic studies to understand the use of this technique. The main objective of this paper is to understand how mocks are used in automated tests in open-source projects, in addition to quantifying the use of support tools and assessing their impact on test creation. We discovered that Mockito, PHPUnit, Jest, and Mock tools are widely employed for Java, PHP, JavaScript, and Python programs, respectively. We also observed that the presence of mocks is consistent and follows the number of test files in each project. We noted that the external dependencies to the project were the most frequently simulated. However, we found no significant correlation between the number of mocks in the project and code coverage.
- Maurício Aniche. 2015. Java code metrics calculator (CK). Available in https://github.com/mauricioaniche/ck/.Google Scholar
- Andrea Arcuri, Gordon Fraser, and René Just. 2017. Private api access and functional mocking in automated unit test generation. In 2017 IEEE international conference on software testing, verification and validation (ICST). IEEE, 126–137.Google ScholarCross Ref
- Sebastian Bergmann. 2009. PHPLoc. Available in https://github.com/sebastianbergmann/phploc.Google Scholar
- Hudson Borges, Andre Hora, and Marco Tulio Valente. 2016. Understanding the factors that impact the popularity of GitHub repositories. In 2016 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 334–344.Google ScholarCross Ref
- Tom Preston-Werner e Scott Chacon Chris Wanstrath, J. Hyett. 2019. Github website. https://github.comGoogle Scholar
- Rômulo Martins da Silva, Cafer Cruz, Heleno de S. Campos, Leonardo GP Murta, and Vânia de Oliveira Neves. 2019. What is the adoption level of automated support for testing in open-source ecosystems?. In Proceedings of the IV Brazilian Symposium on Systematic and Automated Software Testing. 80–89.Google ScholarDigital Library
- Mattia Fazzini, Chase Choi, Juan Manuel Copia, Gabriel Lee, Yoshiki Kakehi, Alessandra Gorla, and Alessandro Orso. 2022. Use of test doubles in android testing: An in-depth investigation. In Proceedings of the 44th International Conference on Software Engineering. 2266–2278.Google ScholarDigital Library
- Martin Gruber and Gordon Fraser. 2022. A survey on how test flakiness affects developers and what support they need to address it. In 2022 IEEE Conference on Software Testing, Verification and Validation (ICST). IEEE, 82–92.Google ScholarCross Ref
- Tim Mackinnon, Steve Freeman, and Philip Craig. 2000. Endo-testing: unit testing with mock objects. Extreme programming examined (2000), 287–301.Google Scholar
- Avantika Mathur, Mingming Cao, and Andreas Dilger. 2007. ext4: the next generation of the ext3 file system. Usenix Association 32, 3 (2007), 25–30.Google Scholar
- Gerard Meszaros. 2007. xUnit Test Patterns: Refactoring Test Code. Addison-Wesley, Upper Saddle River, NJ. https://www.safaribooksonline.com/library/view/xunit-test-patterns/9780131495050/Google ScholarDigital Library
- Shaikh Mostafa and Xiaoyin Wang. 2014. An empirical study on the usage of mocking frameworks in software testing. In 2014 14th international conference on quality software. IEEE, 127–132.Google ScholarDigital Library
- Gustavo Pereira and Andre Hora. 2020. Assessing mock classes: An empirical study. In 2020 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 453–463.Google ScholarCross Ref
- Hesam Samimi, Rebecca Hicks, Ari Fogel, and Todd Millstein. 2013. Declarative mocking. In Proceedings of the 2013 International Symposium on Software Testing and Analysis. 246–256.Google ScholarDigital Library
- S Shaphiro and MBJB Wilk. 1965. An analysis of variance test for normality. Biometrika 52, 3 (1965), 591–611.Google ScholarCross Ref
- Davide Spadini, Maurício Aniche, Magiel Bruntink, and Alberto Bacchelli. 2017. To mock or not to mock? An empirical study on mocking practices. In 2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR). IEEE, 402ś412. https://doi. org/10.1109/MSR (2017).Google ScholarDigital Library
- Davide Spadini, Maurício Aniche, Magiel Bruntink, and Alberto Bacchelli. 2019. Mock objects for testing java systems: Why and how developers use them, and how they evolve. Empirical Software Engineering 24 (2019), 1461–1498.Google ScholarDigital Library
- Charles Spearman. 1904. The proof and measurement of association between two things. American journal of Psychology 15, 1 (1904), 72–101.Google ScholarCross Ref
- Terryin/lizard. 2009. Lizard - A simple code complexity analyser. Available in https://github.com/terryyin/lizard.Google Scholar
- Fabian Trautsch and Jens Grabowski. 2017. Are there any unit tests? an empirical study on unit testing in open source python projects. In 2017 IEEE International Conference on Software Testing, Verification and Validation (ICST). IEEE, 207–218.Google ScholarCross Ref
- Andy Zaidman, Bart Van Rompaey, Serge Demeyer, and Arie Van Deursen. 2008. Mining software repositories to study co-evolution of production & test code. In 2008 1st international conference on software testing, verification, and validation. IEEE, 220–229.Google ScholarDigital Library
Index Terms
- Mock Objects in Software Testing: An Analysis of Usage in Open-Source Projects
Recommendations
What is the adoption level of automated support for testing in open-source ecosystems?
SAST '19: Proceedings of the IV Brazilian Symposium on Systematic and Automated Software TestingIn the last decades, different kinds of automated support for testing have emerged in the open-source community. However, we still have limited evidence about the adoption level of such automated support in practice, considering different programming ...
An Empirical Study on the Adequacy of Testing in Open Source Projects
APSEC '14: Proceedings of the 2014 21st Asia-Pacific Software Engineering Conference - Volume 01During software maintenance, testing is a crucial activity to ensure the quality of code as it evolves over time. With the increasing size and complexity of software, adequate software testing has become increasingly important. Code coverage is an ...
Mock objects for testing java systems
When testing software artifacts that have several dependencies, one has the possibility of either instantiating these dependencies or using mock objects to simulate the dependencies' expected behavior. Even though recent quantitative studies showed that ...
Comments