Man Zhang
ABSTRACT
Nowadays, RESTful web services are widely used for building enterprise applications. In this paper, we propose an enhanced search-based method for automated system test generation for RESTful web services. This method exploits domain knowledge on the handling of HTTP resources, and it is integrated in the Many Independent Objectives (MIO) search algorithm. MIO is an evolutionary algorithm specialized for system test case generation with the aim of maximizing code coverage and fault finding. Our approach builds on top of the MIO by implementing a set of effective templates to structure test actions, based on the semantics of HTTP methods, used to manipulate the web services' resources. We propose four novel sampling strategies for the test cases that can use one or more of these test actions. The strategies are further supported with a set of new, specialized mutation operators that take into account the use of these resources in the generated test cases. We implemented our approach as an extension to the EvoMaster tool, and evaluated it on seven open-source RESTful web services. The results of our empirical study show that our novel, resource-based sampling strategies obtain a significant improvement in performance over the baseline MIO (up to +42% coverage).
- S. Ali, L.C. Briand, H. Hemmati, and R.K. Panesar-Walawege. 2010. A systematic review of the application and empirical investigation of search-based test-case generation. IEEE Transactions on Software Engineering (TSE) 36, 6 (2010), 742--762. Google Scholar
Digital Library
- Subbu Allamaraju. 2010. Restful web services cookbook: solutions for improving scalability and simplicity. " O'Reilly Media, Inc.".Google Scholar
- Mohammad Alshraideh and Leonardo Bottaci. 2006. Search-based software test data generation for string data using program-specific search operators. Software Testing, Verification, and Reliability 16, 3 (2006), 175--203. Google ScholarCross Ref
- Andrea Arcuri. 2018. EvoMaster: Evolutionary Multi-context Automated System Test Generation. In IEEE International Conference on Software Testing, Verification and Validation (ICST). IEEE.Google Scholar
- Andrea Arcuri. 2018. Test suite generation with the Many Independent Objective (MIO) algorithm. Information and Software Technology (IST) 104 (2018), 195--206.Google ScholarCross Ref
- Andrea Arcuri. 2019. RESTful API Automated Test Case Generation with Evo-Master. ACM Transactions on Software Engineering and Methodology (TOSEM) 28, 1 (2019), 3. Google ScholarDigital Library
- A. Arcuri and L. Briand. 2011. Adaptive Random Testing: An Illusion of Effectiveness?. In ACM Int. Symposium on Software Testing and Analysis (ISSTA). 265--275. Google ScholarDigital Library
- A. Arcuri and L. Briand. 2014. A Hitchhiker's Guide to Statistical Tests for Assessing Randomized Algorithms in Software Engineering. Software Testing, Verification and Reliability (STVR) 24, 3 (2014), 219--250. Google ScholarDigital Library
- Mustafa Bozkurt, Mark Harman, and Youssef Hassoun. 2013. Testing and verification in service-oriented architecture: a survey. Software Testing, Verification and Reliability (STVR) 23, 4 (2013), 261--313.Google ScholarCross Ref
- Gerardo Canfora and Massimiliano Di Penta. 2009. Service-oriented architectures testing: A survey. In Software Engineering. Springer, 78--105. Google ScholarDigital Library
- Sujit Kumar Chakrabarti and Prashant Kumar. 2009. Test-the-rest: An approach to testing restful web-services. In Future Computing, Service Computation, Cognitive, Adaptive, Content, Patterns, 2009. COMPUTATIONWORLD'09. Computation World:. IEEE, 302--308. Google ScholarDigital Library
- Sujit Kumar Chakrabarti and Reswin Rodriquez. 2010. Connectedness testing of restful web-services. In Proceedings of the 3rd India software engineering conference. ACM, 143--152. Google ScholarDigital Library
- S. Droste, T. Jansen, and I. Wegener. 1998. On the Optimization of Unimodal Functions with the (1 + 1) Evolutionary Algorithm. In Proceedings of the International Conference on Parallel Problem Solving from Nature. 13--22. Google ScholarDigital Library
- Tobias Fertig and Peter Braun. 2015. Model-driven Testing of RESTful APIs. In Proceedings of the 24th International Conference on World Wide Web. ACM, 1497--1502. Google ScholarDigital Library
- Roy Thomas Fielding. 2000. Architectural styles and the design of network-based software architectures. Ph.D. Dissertation. University of California, Irvine.Google ScholarDigital Library
- Mark Harman, S Afshin Mansouri, and Yuanyuan Zhang. 2012. Search-based software engineering: Trends, techniques and applications. ACM Computing Surveys (CSUR) 45, 1 (2012), 11. Google ScholarDigital Library
- Pablo Lamela Seijas, Huiqing Li, and Simon Thompson. 2013. Towards property-based testing of RESTful web services. In Proceedings of the twelfth ACM SIGPLAN workshop on Erlang. ACM, 77--78. Google ScholarDigital Library
- Pedro Victor Pontes Pinheiro, Andre Takeshi Endo, and Adenilso Simao. 2013. Model-Based Testing of RESTful Web Services Using UML Protocol State Machines. In Brazilian Workshop on Systematic and Automated Software Testing.Google Scholar
- José Miguel Rojas, Mattia Vivanti, Andrea Arcuri, and Gordon Fraser. 2017. A detailed investigation of the effectiveness of whole test suite generation. Empirical Software Engineering (EMSE) 22, 2 (2017), 852--893. Google ScholarDigital Library
- Sergio Segura, José A Parejo, Javier Troya, and Antonio Ruiz-Cortés. 2017. Metamorphic testing of RESTful web APIs. IEEE Transactions on Software Engineering (TSE) (2017).Google Scholar
Index Terms
- Resource-based test case generation for RESTful web services
Recommendations
Towards composition of RESTful web services
ICCCNT '15: Proceedings of the 2015 6th International Conference on Computing, Communication and Networking Technologies (ICCCNT)RESTful HTTP and SOAP/WSDL services are two architectural styles for building web services, Where SOAP/WSDL based services follows an operation centric approach and RESTful HTTP based services follows resource centric approach. Dynamic composition of ...
Automatic Timed Test Case Generation for Web Services Composition
ECOWS '08: Proceedings of the 2008 Sixth European Conference on Web ServicesIn order to specify the composition of Web services, WSBPEL was defined as an orchestrating language by an international standards consortium. In this paper, we propose a method to test composite Web service described in BPEL. As a first step, the BPEL ...
Ontology-Based Test Case Generation for Testing Web Services
ISADS '07: Proceedings of the Eighth International Symposium on Autonomous Decentralized SystemsWeb Services (WS) enables agile application development by orchestrating the existing service components. However, the dynamically constructed service-based system has to be tested dynamically and automatically at runtime without human intervention. To ...
Comments