Linda Erlenhov
ABSTRACT
Software engineering bots – automated tools that handle tedious tasks – are increasingly used by industrial and open source projects to improve developer productivity. Current research in this area is held back by a lack of consensus of what software engineering bots (DevBots) actually are, what characteristics distinguish them from other tools, and what benefits and challenges are associated with DevBot usage. In this paper we report on a mixed-method empirical study of DevBot usage in industrial practice. We report on findings from interviewing 21 and surveying a total of 111 developers. We identify three different personas among DevBot users (focusing on autonomy, chat interfaces, and “smartness”), each with different definitions of what a DevBot is, why developers use them, and what they struggle with.We conclude that future DevBot research should situate their work within our framework, to clearly identify what type of bot the work targets, and what advantages practitioners can expect. Further, we find that there currently is a lack of general purpose “smart” bots that go beyond simple automation tools or chat interfaces. This is problematic, as we have seen that such bots, if available, can have a transformative effect on the projects that use them.
Supplemental Material
- Ablett, R., Sharlin, E., Maurer, F., Denzinger, J., and Schock, C. BuildBot: Robotic Monitoring of Agile Software Development Teams. In RO-MAN 2007-The 16th IEEE International Symposium on Robot and Human Interactive Communication (Aug 2007 ), pp. 931-936.Google Scholar
- Bacchelli, A., and Bird, C. Expectations, Outcomes, and Challenges of Modern Code Review. In 2013 35th International Conference on Software Engineering (ICSE) (May 2013 ), pp. 712-721.Google Scholar
- Baysal, O., Holmes, R., and Godfrey, M. W. Revisiting Bug Triage and Resolution Practices. In 2012 First International Workshop on User Evaluation for Software Engineering Researchers (USER) ( June 2012 ), pp. 29-30.Google Scholar
- Bieliauskas, S., and Schreiber, A. A Conversational User Interface for Software Visualization. In 2017 IEEE Working Conference on Software Visualization (VISSOFT) ( Sep. 2017 ), pp. 139-143.Google ScholarCross Ref
- Bratthall, L., and Jorgensen, M. Can you Trust a Single Data Source Exploratory Software Engineering Case Study? Empirical Software Engineering ( 2002 ).Google Scholar
- Brown, C., and Parnin, C. Sorry to Bother You: Designing Bots for Efective Recommendations. In 2019 IEEE/ACM 1st International Workshop on Bots in Software Engineering (BotSE) (May 2019 ), pp. 54-58.Google Scholar
- Canfora, G., Di Penta, M., Oliveto, R., and Panichella, S. Who is Going to Mentor Newcomers in Open Source Projects? In Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering (New York, NY, USA, 2012 ), FSE'12, Association for Computing Machinery.Google Scholar
- Damasceno Costa, D. E., Bezemer, C., Leitner, P., and Andrzejak, A. What's Wrong With My Benchmark Results? Studying Bad Practices in JMH Benchmarks. IEEE Transactions on Software Engineering ( 2019 ), 1-1.Google Scholar
- Erlenhov, L., de Oliveira Neto, F. G., and Leitner, P. Replication package to An Empirical Study of Bots in Software Development-Characteristics and Challenges from a Practitioner's Perspective, Mar. 2020. https://doi.org/10.5281/zenodo.4022892. Google ScholarCross Ref
- Erlenhov, L., de Oliveira Neto, F. G., Scandariato, R., and Leitner, P. Current and Future Bots in Software Development. In First Workshop on Bots in Software Engineering, (BotSE ICSE ) ( 2019 ).Google Scholar
- Feitelson, D. G., Frachtenberg, E., and Beck, K. L. Development and deployment at facebook. IEEE Internet Computing 17, 4 ( July 2013 ), 8-17.Google ScholarDigital Library
- Ferrara, E., Varol, O., Davis, C., Menczer, F., and Flammini, A. The Rise of Social Bots. Communications of the ACM 59, 7 ( June 2016 ), 96-104.Google ScholarDigital Library
- Hoda, R., Noble, J., and Marshall, S. Developing a Grounded Theory to Explain the Practices of Self-Organizing Agile Teams. Empirical Software Engineering ( 2012 ).Google Scholar
- Kim, Y., and Sundar, S. S. Anthropomorphism of computers: Is it mindful or mindless? Computers in Human Behavior 28, 1 ( 2012 ), 241-250.Google ScholarDigital Library
- Lebeuf, C., Storey, M.-A., and Zagalsky, A. Software Bots. IEEE Software 35, 1 ( 2018 ), 18-23.Google ScholarCross Ref
- Lebeuf, C., Zagalsky, A., Foucault, M., and Storey, M.-A. Defining and Classifying Software Bots: A Faceted Taxonomy. In Proceedings of the 1st International Workshop on Bots in Software Engineering ( 2019 ), BotSE'19, IEEE Press, pp. 1-6.Google Scholar
- Lee, M., Ackermans, S., van As, N., Chang, H., Lucas, E., and IJsselsteijn, W. Caring for Vincent: A Chatbot for Self-Compassion. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (New York, NY, USA, 2019 ), CHI'19, Association for Computing Machinery.Google ScholarDigital Library
- Matthies, C., Dobrigkeit, F., and Hesse, G. An Additional Set of (Automated) Eyes: Chatbots for Agile Retrospectives. In Proceedings of the 1st International Workshop on Bots in Software Engineering ( 2019 ), BotSE'19, IEEE Press, pp. 34-37.Google Scholar
- Menzies, T., Williams, L., and Zimmermann, T. Perspectives on Data Science for Software Engineering, 1 ed. Morgan Kaufmann, 7 2016.Google Scholar
- Meyer, A. N., Fritz, T., Murphy, G. C., and Zimmermann, T. Software Developers' Perceptions of Productivity. In FSE '14: Proceedings of the 22nd ACM SIGSOFT International Symposium on the Foundations of Software Engineering (November 2014 ), ACM-Association for Computing Machinery.Google Scholar
- Müller-Birn, C., Dobusch, L., and Herbsleb, J. D. Work-to-Rule: The Emergence of Algorithmic Governance in Wikipedia. In Proceedings of the 6th International Conference on Communities and Technologies (New York, NY, USA, 2013 ), Association for Computing Machinery, pp. 80-89.Google ScholarDigital Library
- Murgia, A., Janssens, D., Demeyer, S., and Vasilescu, B. Among the Machines: Human-Bot Interaction on Social Q&A Websites. In CHI Conference on Human Factors in Computing Systems ( 2016 ), CHI Extended Abstracts, ACM, pp. 1272-1279.Google Scholar
- Paikari, E., and van der Hoek, A. A Framework for Understanding Chatbots and Their Future. In Proceedings of the 11th International Workshop on Cooperative and Human Aspects of Software Engineering (New York, NY, USA, 2018 ), CHASE'18, Association for Computing Machinery, pp. 13-16.Google Scholar
- Pruitt, J., and Grudin, J. Personas: Practice and Theory. In Proceedings of the 2003 Conference on Designing for User Experiences (New York, NY, USA, 2003 ), DUX'03, Association for Computing Machinery, pp. 1-15.Google Scholar
- Stol, K.-J., Ralph, P., and Fitzgerald, B. Grounded Theory in Software Engineering Research: A Critical Review and Guidelines. In 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE) (May 2016 ), pp. 120-131.Google Scholar
- Storey, M.-A., and Zagalsky, A. Disrupting Developer Productivity One Bot at a Time. In Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering (New York, NY, USA, 2016 ), FSE 2016, Association for Computing Machinery, pp. 928-931.Google Scholar
- Strauss, A., Corbin, J., and Corbin, J. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. SAGE Publications, 1998.Google Scholar
- SUNDAR, S. S., and NASS, C. Source orientation in human-computer interaction: Programmer, networker, or independent social actor. Communication Research 27, 6 ( 2000 ), 683-703.Google Scholar
- Urli, S., Yu, Z., Seinturier, L., and Monperrus, M. How to Design a Program Repair Bot? Insights from the Repairnator Project. In 2018 IEEE/ACM 40th International Conference on Software Engineering: Software Engineering in Practice Track (ICSE-SEIP) (May 2018 ), pp. 95-104.Google Scholar
- van Tonder, R., and Le Goues, C. Towards s/Engineer/Bot: Principles for Program Repair Bots. In Proceedings of the 1st International Workshop on Bots in Software Engineering ( 2019 ), BotSE'19, IEEE Press, pp. 43-47.Google Scholar
- Wessel, M., de Souza, B. M., Steinmacher, I., Wiese, I. S., Polato, I., Chaves, A. P., and Gerosa, M. A. The Power of Bots: Characterizing and Understanding Bots in OSS Projects. Proc. ACM Hum.-Comput. Interact. 2, CSCW (Nov. 2018 ).Google Scholar
- Wessel, M., Steinmacher, I., Wiese, I., and Gerosa, M. A. Should I Stale or Should I Close? An Analysis of a Bot That Closes Abandoned Issues and Pull Requests. In Proceedings of the 1st International Workshop on Bots in Software Engineering ( 2019 ), BotSE'19, IEEE Press, pp. 38-42.Google Scholar
- Wyrich, M., and Bogner, J. Towards an Autonomous Bot for Automatic Source Code Refactoring. In Proceedings of the 1st International Workshop on Bots in Software Engineering ( 2019 ), BotSE'19, IEEE Press, pp. 24-28.Google Scholar
- Zhu, L., Bass, L., and Champlin-Scharff, G. DevOps and Its Practices. IEEE Software 33, 3 (May 2016 ), 32-34.Google Scholar
Index Terms
- An empirical study of bots in software development: characteristics and challenges from a practitioner’s perspective
Recommendations
"Leagile" software development
In recent years there has been a noticeable shift in attention from those who use agile software development toward lean software development, often labelled as a shift "from agile to lean". However, the reality may not be as simple or linear as this ...
Cleanroom Software Development: An Empirical Evaluation
The Cleanroom software development approach is intended to produce highly reliable software by integrating formal methods for specification and design, nonexecution-based program development, and statistically based independent testing. In an empirical ...
Software development: what it is, what it should be, and how to get there
Developing large software systems is notoriously difficult and unpredictable. Software projects are often canceled, finish late and over budget, or yield low quality results --- setting software engineering apart from established engineering ...
Comments