Skip to main content
SpringerLink
Log in

Crowdsourced test case generation for android applications via static program analysis

  • Published:
Automated Software Engineering Aims and scope Submit manuscript

Abstract

The testing of Android applications(apps) is a challenging task due to the serious fragmentation issues and diverse usage environments. To improve the testing efficiency and collect the feedbacks from real usage scenarios, crowdsourcing has been employed in the testing of Android. However, crowdsourced testing is a manual working paradigm, while the shortage of testing guidance for crowd workers who often have limited software engineering knowledge may result in many redundant or invalid test reports. To fill this gap, this paper presents an automated test case generation approach for the testing of Android apps. Our approach is built upon static program analysis and is capable of providing detailed testing steps to guide workers in performing testing. Furthermore, we use the automated testing tool for pre-testing, and crowd workers only need to test the uncovered test cases. We evaluate our approach with six widely-used apps to evaluate its effectiveness and efficiency. The experimental results show that our approach can detect 71.5% more bugs in diverse categories and achieve 21.8% higher path coverage in comparison with classic crowdsourced testing techniques. Also, in the experiment, we detect 44 unknown bugs in the six subjects, which indicates our approach is highly promising for assisting the testing of Android apps in practice.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. https://www.appbrain.com/stats. Accessed 29 August 2022.

  2. https://www.appbrain.com/stats. Accessed 2 March 2023.

  3. http://web.cse.ohio-state.edu/presto/software/gator/. Accessed 29 August 2022.

  4. www.mooctest.net. Accessed 29 August 2022.

  5. https://ibotpeaches.github.io/Apktool/.Accessed 29 August 2022.

  6. https://search.f-droid.org/. Accessed 29 August 2022.

  7. https://play.google.com/store/apps/details?id=protect.budgetwatch.play. Accessed 8 March 2023.

  8. https://f-droid.org/en/packages/home.jmstudios.calc/. Accessed 8 March 2023.

  9. https://f-droid.org/zh_Hant/packages/douzifly.list/. Accessed 8 March 2023.

  10. https://f-droid.org/zh_Hans/packages/dev.corruptedark.diditakemymeds/. Accessed 8 March 2023.

  11. https://github.com/KuoLuoC/JZ_app. Accessed 8 March 2023.

  12. https://f-droid.org/zh_Hans/packages/com.uberspot.a2048/. Accessed 8 March 2023.

  13. The data complies with the t-test normality hypothesis.

References

  • Alyahya, S.: Crowdsourced software testing: a systematic literature review. Inf. Softw. Technol. 127, 106363 (2020)

    Article  Google Scholar 

  • Amalfitano, D., Fasolino, A.R., Tramontana, P., Ta, B.D., Memon, A.M.: Mobiguitar: automated model-based testing of mobile apps. IEEE Softw. 32(5), 53–59 (2014)

    Article  Google Scholar 

  • Amalfitano, D., Amatucci, N., Fasolino, A.R., Tramontana, P.: Agrippin: a novel search based testing technique for android applications. In: Proceedings of the 3rd International Workshop on Software Development Lifecycle for Mobile: pp. 5–12 (2015)

  • Amalfitano, D., Amatucci, N., Memon, A.M., Tramontana, P., Fasolino, A.R.: A general framework for comparing automatic testing techniques of android mobile apps. J. Syst. Softw. 125, 322–343 (2017)

    Article  Google Scholar 

  • Anand, S., Naik, M., Harrold, M.J., Yang, H.: Automated concolic testing of smartphone apps. In: Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering: pp. 1–11 (2012)

  • Arzt, S., Rasthofer, S., Fritz, C., Bodden, E., Bartel, A., Klein, J., Le Traon, Y., Octeau, D., McDaniel, P.: Flowdroid: precise context, flow, field, object-sensitive and lifecycle-aware taint analysis for android apps. ACM Sigplan Notices 49(6), 259–269 (2014)

    Article  Google Scholar 

  • Au, F.T., Baker, S., Warren, I., Dobbie, G.: Automated usability testing framework. In: Proceedings of the ninth conference on Australasian user interface-Volume 76: pp. 55–64 (2008)

  • Chen, S., Fan, L., Chen, C., Su, T., Li, W., Liu, Y., Xu, L.: Storydroid: Automated generation of storyboard for android apps. In: 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE): IEEE: pp. 596–607 (2019)

  • Chen, Y., Pandey, M., Song, J.Y., Lasecki, W.S., Oney, S.: Improving crowd-supported gui testing with structural guidance. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems: pp. 1–13 (2020)

  • Cui, Q., Wang, J., Yang, G., Xie, M., Wang, Q., Li, M.: Who should be selected to perform a task in crowdsourced testing? In: 2017 IEEE 41st Annual Computer Software and Applications Conference (COMPSAC): IEEE: vol 1: pp. 75–84 (2017)

  • Dimara, E., Bezerianos, A., Dragicevic, P.: Narratives in crowdsourced evaluation of visualizations: A double-edged sword? In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems: Association for Computing Machinery, New York, NY, USA: p. 5475-5484 (2017)

  • Gadiraju, U., Yang, J., Bozzon, A.: Clarity is a worthwhile quality: On the role of task clarity in microtask crowdsourcing. In: Proceedings of the 28th ACM conference on hypertext and social media: pp. 5–14 (2017)

  • Gebrekrstos, S.W., Gereziher, T.B.: Search-based test generation framework for android apps with support for multiobjective generation: Stgfa-smog (2022)

  • Hettiachchi, D., Van Berkel, N., Kostakos, V., Goncalves, J.: Crowdcog: a cognitive skill based system for heterogeneous task assignment and recommendation in crowdsourcing. Proc. ACM Hum.-Comput. Inter. 4(CSCW2), 1–22 (2020)

    Article  Google Scholar 

  • Hettiachchi, D., Kostakos, V., Goncalves, J.: A survey on task assignment in crowdsourcing. ACM Comput. Surv. (CSUR) 55(3), 1–35 (2022)

    Article  Google Scholar 

  • Howe, J.: Why the power of the crowd is driving the future of business (2008)

  • Irani, L.C., Silberman, M.S.: Stories we tell about labor: Turkopticon and the trouble with“ design”. In: Proceedings of the 2016 CHI conference on human factors in computing systems: pp. 4573–4586 (2016)

  • Jabbarvand, R., Lin, J.W., Malek, S.: Search-based energy testing of android. In: 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE): IEEE: pp. 1119–1130 (2019)

  • Jensen, C.S., Prasad, M.R., Møller, A.: Automated testing with targeted event sequence generation. In: Proceedings of the 2013 International Symposium on Software Testing and Analysis: pp. 67–77 (2013)

  • Khan, U.H., Khan, M.N.A., Mirza, A.M., Akram, M., Fakhar, S., Hussain, S., Magsi, I.A., Wagan, R.A.: Automating test case generation for android applications using model-based testing. EMITTER International Journal of Engineering Technology pp. 63–82 (2022)

  • Kong, P., Li, L., Gao, J., Liu, K., Bissyandé, T.F., Klein, J.: Automated testing of android apps: a systematic literature review. IEEE Trans. Reliabil. 68(1), 45–66 (2018)

    Article  Google Scholar 

  • Li, L., Bissyandé, T.F., Papadakis, M., Rasthofer, S., Bartel, A., Octeau, D., Klein, J., Traon, L.: Static analysis of android apps: a systematic literature review. Inf. Softw. Technol. 88, 67–95 (2017)

    Article  Google Scholar 

  • Machiry, A., Tahiliani, R., Naik, M.: Dynodroid: an input generation system for android apps. In: Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering: pp. 224–234 (2013)

  • Mao, K., Yang, Y., Wang, Q., Jia, Y., Harman, M.: Developer recommendation for crowdsourced software development tasks. In: 2015 IEEE Symposium on Service-Oriented System Engineering: IEEE: pp. 347–356 (2015)

  • Mao, K., Capra, L., Harman, M., Jia, Y.: A survey of the use of crowdsourcing in software engineering. J. Syst. Softw. 126, 57–84 (2017)

    Article  Google Scholar 

  • Moreno, I.A.: Search-based test generation for android apps. In: 2020 IEEE/ACM 42nd International Conference on Software Engineering: Companion Proceedings (ICSE-Companion): IEEE: pp. 230–233 (2020)

  • Pan, M., Huang, A., Wang, G., Zhang, T., Li, X.: Reinforcement learning based curiosity-driven testing of android applications. In: Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis: pp. 153–164 (2020)

  • Pezzè, M., Young, M.: Software Testing and Analysis: Process, Principles, and Techniques. Wiley (2008)

  • Rountev, A., Yan, D.: Static reference analysis for gui objects in android software. In: Proceedings of Annual IEEE/ACM International Symposium on Code Generation and Optimization: Association for Computing Machinery, New York, NY, USA: p. 143-153 (2014)

  • Salman, I., Misirli, A.T., Juristo, N.: Are students representatives of professionals in software engineering experiments? In: 2015 IEEE/ACM 37th IEEE international conference on software engineering: IEEE: vol 1: pp. 666–676 (2015)

  • Su, T., Meng, G., Chen, Y., Wu, K., Yang, W., Yao, Y., Pu, G., Liu, Y., Su, Z.: Guided, stochastic model-based gui testing of android apps. In: Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering: pp. 245–256 (2017)

  • Tan, S.H., Li, Z.: Collaborative bug finding for android apps. In: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: pp. 1335–1347 (2020)

  • Usman, A., Ibrahim, N., Salihu, I.A.: Tegdroid: test case generation approach for android apps considering context and gui events. Int. J. Adv. Sci. Eng. Inf. Technol. 10(1), 16 (2020)

    Article  Google Scholar 

  • Wang, J., Jiang, Y., Xu, C., Cao, C., Ma, X., Lu, J.: Combodroid: generating high-quality test inputs for android apps via use case combinations. In: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: pp. 469–480 (2020)

  • Wang, J., Yang, Y., Wang, S., Chen, C., Wang, D., Wang, Q.: Context-aware personalized crowdtesting task recommendation. IEEE Transactions on Software Engineering (2021)

  • Wei, L., Liu, Y., Cheung, S.C., Huang, H., Lu, X., Liu, X.: Understanding and detecting fragmentation-induced compatibility issues for android apps. IEEE Trans. Softw. Eng. 46(11), 1176–1199 (2018)

    Article  Google Scholar 

  • Yang, S., Yan, D., Wu, H., Wang, Y., Rountev, A.: Static control-flow analysis of user-driven callbacks in android applications. In: 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering: IEEE: vol 1: pp. 89–99 (2015)

  • Yang, S., Wu, H., Zhang, H., Wang, Y., Swaminathan, C., Yan, D., Rountev, A.: Static window transition graphs for android. Autom. Softw. Eng. 25(4), 833–873 (2018)

    Article  Google Scholar 

  • Yang, Y., Karim, M.R., Saremi, R., Ruhe, G.: Who should take this task? dynamic decision support for crowd workers. In: Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement: pp. 1–10 (2016)

Download references

Acknowledgements

We would like to thank anonymous reviewers for their insightful and constructive comments. This research was partially funded by the National Natural Science Foundation of China under Grant Nos. 62002158, 61832009, and 61932012, and the Science, Technology and Innovation Commission of Shenzhen Municipality (No. CJGJZD20200617103001003).

Author information

Authors and Affiliations

  1. The State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, 210093, Jiangsu, China

    Yuying Li, Yang Feng, Chao Guo, Zhenyu Chen & Baowen Xu

Authors
  1. Yuying Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yang Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chao Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhenyu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Baowen Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yang Feng.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Y., Feng, Y., Guo, C. et al. Crowdsourced test case generation for android applications via static program analysis. Autom Softw Eng 30, 26 (2023). https://doi.org/10.1007/s10515-023-00394-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10515-023-00394-w

Keywords

Search

Navigation