Abstract
Testing is essential for quality assurance of database applications. Achieving high code coverage of the database applications is important in testing. In practice, there may exist a copy of live databases that can be used for database application testing. Using an existing database state is desirable since it tends to be representative of real-world objects’ characteristics, helping detect faults that could cause failures in real-world settings. However, to cover a specific program-code portion (e.g., block), appropriate program inputs also need to be generated for the given existing database state. To address this issue, in this paper, we propose a novel approach that generates program inputs for achieving high code coverage of a database application, given an existing database state. Our approach uses symbolic execution to track how program inputs are transformed before appearing in the executed SQL queries and how the constraints on query results affect the application’s execution. One significant challenge in our problem context is the gap between program-input constraints derived from the program and from the given existing database state; satisfying both types of constraints is needed to cover a specific program-code portion. Our approach includes novel query formulation to bridge this gap. We incorporate the data-instantiation component in our framework to deal with the case that no effective program input values can be attained. We determine how to generate new records and populate them in the new database state such that the code along the path can be covered. We also extend our approach of program-input generation to test database applications including canonical queries and group-by queries. Our approach is loosely integrated into Pex, a state-of-the-art white-box testing tool for .NET from Microsoft Research. Empirical evaluations on two real database applications show that our approach assists Pex to generate program inputs that achieve higher code coverage than the program inputs generated by Pex without our approach’s assistance.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ahmed, R., Lee, A.W., Witkowski, A., Das, D., Su, H., Zaït, M., Cruanes, T.: Cost-based query transformation in Oracle. In: Proceedings of International Conference on Very Large Data Bases (VLDB), pp. 1026–1036 (2006)
Binnig, C., Kossmann, D., Lo, E.: Reverse query processing. in: Proceedings of IEEE International Conference on Data Engineering (ICDE), pp. 506–515 (2007a)
Binnig, C., Kossmann, D., Lo, E., Özsu, M.T.: QAGen: generating query-aware test databases. In: Proceedings of ACM SIGMOD Conference pp. 341–352 (2007b)
Binnig, C., Kossmann, D., Lo, E.: Multi-RQP: generating test databases for the functional testing of OLTP applications. in: International Workshop on Testing Database Systems (DBTest), p. 5 (2008)
Bruno, N., Chaudhuri, S.: Flexible database generators. In: Proceedings of International Conference on Very Large Data Bases (VLDB), pp. 1097–1107 (2005)
Chaudhuri, S., Shim, K.: Including group-by in query optimization. In: Proceedings of International Conference on Very Large Data Bases (VLDB), pp. 354–366 (1994)
Chays, D.: Test data generation for relational database applications. PhD thesis, Computer and Information Science, Polytechnic University (2004)
Chays, D., Deng, Y., Frankl, P.G., Dan, S., Vokolos, F.I., Weyuker, E.J.: An AGENDA to test relational database applications. J. Softw. Test. Verif. Reliab. 14, 17–44 (2004)
Chays, D., Shahid, J., Frankl, P.G.: Query-based test generation for database applications. In: International Workshop on Testing Database Systems (DBTest), p. 6 (2008)
Dayal, U.: Of nests and trees: a unified approach to processing queries that contain nested subqueries, aggregates, and quantifiers. In: Proceedings of International Conference on Very Large Data Bases (VLDB), pp. 197–208 (1987)
de la Riva, C., Cabal, M.J.S., Tuya, J.: Constraint-based test database generation for SQL queries. In: International Workshop on Automation of Software Test (AST), pp. 67–74 (2010)
Deng, Y., Chays, D.: Testing database transactions with AGENDA. In: Proceedings of International Conference on Software Engineering (ICSE), pp. 78–87 (2005)
Emmi, M., Majumdar, R., Sen, K.: Dynamic test input generation for database applications. In: Proceedings of International Symposium on Software Testing and Analysis (ISSTA), pp. 151–162 (2007)
Ganski, R.A., Wong, H.K.T.: Optimization of nested SQL queries revisited. In: Proceedings of ACM SIGMOD Conference, pp. 23–33 (1987)
Godefroid, P., Luchaup, D.: Automatic partial loop summarization in dynamic test generation. In: Proceedings of International Symposium on Software Testing and Analysis (ISSTA), pp. 23–33 (2011)
Gupta, A., Harinarayan, V., Quass, D.: Aggregate-query processing in data warehousing environments. In: Proceedings of International Conference on Very Large Data Bases (VLDB), pp. 358–369 (1995)
Gupta, B.P., Vira, D., Sudarshan, S.: X-data: generating test data for killing SQL mutants. In: Proceedings of IEEE International Conference on Data Engineering (ICDE), pp. 876–879 (2010)
Halfond, W.G.J., Orso, A.: Command-form coverage for testing database applications. In: Proceedings of IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 69–80 (2006)
Kapfhammer, G.M., Soffa, M.L.: A family of test adequacy criteria for database-driven applications. In: Proceedings of the 9th European Software Engineering Conference held jointly with 11th ACM SIGSOFT International Symposium on Foundations of software engineering (ESEC/FSE), pp. 98–107 (2003)
Kapfhammer, G.M., Soffa, M.L.: Database-aware test coverage monitoring. In: Proceedings of the 1st India Software Engineering Conference (ISEC), pp. 77–86 (2008)
Khalek, S.A., Khurshid, S.: Automated SQL query generation for systematic testing of database engines. In: Proceedings of IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 329–332 (2010)
Khalek, S.A., Elkarablieh, B., Laleye, Y.O., Khurshid, S.: Query-aware test generation using a relational constraint solver. In: Proceedings of IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 238–247 (2008)
Kim, W.: On optimizing an SQL-like nested query. ACM Trans. Database Syst. 7(3), 443–469 (1982)
King, J.C.: Symbolic execution and program testing. Commun. ACM 19(7), 385–394 (1976)
Li, C., Csallner, C.: Dynamic symbolic database application testing. In: Proceedings of International Workshop on Testing Database Systems (DBTest), pp. 01–06 (2010)
Microsoft Research Foundation of Software Engineering Group: Pex: Dynamic Analysis and Test Generation for .NET
Pan, K., Wu, X., Xie, T.: Database state generation via dynamic symbolic execution for coverage criteria. In: Proceedings of International Workshop on Testing Database Systems (DBTest), pp. 01–06 (2011a)
Pan, K., Wu, X., Xie, T.: Generating program inputs for database application testing. In: Proceedings of IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 73–82 (2011b)
Pan, K., Wu, X., Xie, T.: Guided test generation for database applications via synthesized database interactions. In: ACM Transactions on Software Engineering and Methodology, 23(2), 12 (2014)
Sen, K., Marinov, D., Agha, G.: CUTE: a concolic unit testing engine for C. In: Proceedings of the joint meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE), pp. 263–272 (2005)
Seshadri, P., Pirahesh, H., Leung, T.Y.C.: Complex query decorrelation. In: Proceedings of the Twelfth International Conference on Data Engineering (ICDE), pp. 450–458 (1996)
Shah, S., Sudarshan, S., Kajbaje, S., Patidar, S., Gupta, B.P., Vira, D.: Generating test data for killing SQL mutants: a constraint-based approach. In: Proceedings of the Twelfth International Conference on Data Engineering (ICDE), pp. 1175–1186 (2011)
Taneja, K., Zhang, Y., Xie, T.: MODA: automated test generation for database applications via mock objects. In: Proceedings of IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 289–292 (2010)
Tuya, J., Cabal, M.J.S., de la Riva, C.: SQLMutation: a tool to generate mutants of SQL database queries. In: Proceedings of the Second Workshop on Mutation Analysis, p. 1 (2006)
Tuya, J., Cabal, M.J.S., de la Riva, C.: Mutating database queries. Inf. Softw. Technol. 49(4), 398–417 (2007)
Tuya, J., Cabal, M.J.S., de la Riva, C.: Full predicate coverage for testing SQL database queries. J. Softw. Test. Verif. Reliab. 20, 237–288 (2010)
Willmor, D., Embury, S.M.: An intensional approach to the specification of test cases for database applications. In: Proceedings of International Conference on Software Engineering (ICSE), pp. 102–111 (2006a)
Willmor, D., Embury, S.M.: Testing the implementation of business rules using intensional database tests. In: TAIC PART, pp. 115–126 (2006b)
Wu, X., Sanghvi, C., Wang, Y., Zheng, Y.: Privacy aware data generation for testing database applications. In: Proceedings of International Database Engineering and Applications Symposium (IDEAS), pp. 317–326 (2005)
Wu, X., Wang, Y., Guo, S., Zheng, Y.: Privacy preserving database generation for database application testing. Fundam. Inf. 78(4), 595–612 (2007)
Zhou, C., Frankl, P.G.: Mutation testing for java database applications. In: Proceedings of IEEE International Conference on Software Testing, Verification and Validation (ICST), pp. 396–405 (2009)
Zhou, C., Frankl, P.G.: Inferential checking for mutants modifying database states. In: Proceedings of IEEE International Conference on Software Testing, Verification and Validation (ICST), pp. 259–268 (2011a)
Zhou, C., Frankl, P.G.: JDAMA: Java database application mutation analyser. Softw. Test. Verif. Reliab. 21(3), 241–263 (2011b)
Zhou, C., Frankl, P.G.: Empirical studies on test effectiveness for database applications. In: Proceedings of IEEE International Conference on Software Testing, Verification and Validation (ICST), pp. 61–70 (2012)
Acknowledgments
This work was supported in part by U.S. National Science Foundation under CCF-0915059 for Kai Pan and Xintao Wu, and under CCF-1349666, CNS-1434582, CCF-1434596, CCF-1434590, and CNS-1439481 for Tao Xie.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Pan, K., Wu, X. & Xie, T. Program-input generation for testing database applications using existing database states. Autom Softw Eng 22, 439–473 (2015). https://doi.org/10.1007/s10515-014-0158-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10515-014-0158-y