ABSTRACT
App testing tools tend to generate thousand test inputs; they help engineers identify crashing conditions but not functional failures. Indeed, detecting functional failures requires the visual inspection of App outputs, which is infeasible for thousands of inputs. Existing App testing tools ignore that most of the Apps are frequently updated and engineers are mainly interested in testing the updated functionalities; indeed, automated regression test cases can be used otherwise. We present ATUA, an open source tool targeting Android Apps. It achieves high coverage of the updated App code with a small number of test inputs, thus alleviating the test oracle problem (less outputs to inspect). It implements a model-based approach that synthesizes App models with static analysis, integrates a dynamically-refined state abstraction function and combines complementary testing strategies, including (1) coverage of the model structure, (2) coverage of the App code, (3) random exploration, and (4) coverage of dependencies identified through information retrieval. Our empirical evaluation, conducted with nine popular Android Apps (72 versions), has shown that ATUA, compared to state-of-the-art approaches, achieves higher code coverage while producing fewer outputs to be manually inspected. A demo video is available at https://youtu.be/RqQ1z_Nkaqo.
- Android. 2022. Monkey. http://developer.android.com/tools/help/monkey.html Google Scholar
- Nataniel P. Borges Jr., Jenny Hotzkow, and Andreas Zeller. 2018. DroidMate-2: A Platform for Android Test Generation. In ASE 2018. ACM, New York, NY, USA. 916–919. isbn:978-1-4503-5937-5 https://doi.org/10.1145/3238147.3240479 Google ScholarDigital Library
- Wontae Choi, Koushik Sen, George Necula, and Wenyu Wang. 2018. DetReduce: Minimizing Android GUI Test Suites for Regression Testing. In ICSE 2018. ACM, New York, NY, USA. 445–455. isbn:9781450356381 https://doi.org/10.1145/3180155.3180173 Google ScholarDigital Library
- Daniel Domínguez-Álvarez and Alessandra Gorla. 2019. Release Practices for IOS and Android Apps. In WAMA 2019. ACM, New York, NY, USA. 15–18. isbn:9781450368582 Google Scholar
- Tianxiao Gu, Chengnian Sun, Xiaoxing Ma, Chun Cao, Chang Xu, Yuan Yao, Qirun Zhang, Jian Lu, and Zhendong Su. 2019. Practical GUI Testing of Android Applications via Model Abstraction and Refinement. In ICSE 2019. IEEE Press, 269–280. https://doi.org/10.1109/ICSE.2019.00042 Google ScholarDigital Library
- Chanh Duc Ngo, Fabrizio Pastore, and Lionel Briand. 2021. Automated, Cost-effective, and Update-driven App Testing. ACM TOSEM, Dec., https://doi.org/10.1145/3502297 arXiv: 2012.02471. Google ScholarDigital Library
- Chanh Duc Ngo, Fabrizio Pastore, and Lionel Briand. 2022. ATUA replicability package. https://doi.org/10.5281/zenodo.5734090 Google ScholarDigital Library
- Atanas Rountev and Dacong Yan. 2014. Static Reference Analysis for GUI Objects in Android Software. In CGO 2014. ACM, New York, NY, USA. 143:143–143:153. isbn:978-1-4503-2670-4 Google Scholar
- Aman Sharma and Rupesh Nasre. 2019. QADroid: Regression Event Selection for Android Applications. In ISSTA 2019. ACM, New York, NY, USA. 66–77. isbn:9781450362245 Google ScholarDigital Library
- Porfirio Tramontana, Domenico Amalfitano, Nicola Amatucci, and Anna Rita Fasolino. 2019. Automated functional testing of mobile applications: a systematic mapping study. Software Quality Journal, 27, 1 (2019), 149–201. isbn:1573-1367 Google ScholarDigital Library
- Wenyu Wang, Dengfeng Li, Wei Yang, Yurui Cao, Zhenwen Zhang, Yuetang Deng, and Tao Xie. 2018. An Empirical Study of Android Test Generation Tools in Industrial Cases. In ASE 2018. ACM, New York, NY, USA. 738–748. isbn:978-1-4503-5937-5 Google Scholar
Index Terms
- ATUA: an update-driven app testing tool
Recommendations
Automated, Cost-effective, and Update-driven App Testing
Apps’ pervasive role in our society led to the definition of test automation approaches to ensure their dependability. However, state-of-the-art approaches tend to generate large numbers of test inputs and are unlikely to achieve more than 50% method ...
A compatibility testing platform for android multimedia applications
Along with the widespread use of smartphones, Android has become one of the major platforms for multimedia applications (apps). However, due to the fast evolution of Android operating system and the fragmentation of Android devices, it becomes important ...
Preference-wise Testing of Android Apps via Test Amplification
Preferences, the setting options provided by Android, are an essential part of Android apps. Preferences allow users to change app features and behaviors dynamically, and therefore their impacts need to be considered when testing the apps. Unfortunately, ...
Comments