Panos Louridas
Georgios Gousios
Abstract
As any empirical science, Software Engineering research should strive towards better research practices. Replication is regrettably not a priority for Software Engineering researchers and, moreover, not afforded by many published studies. Here we report our experience from our encounter with a recent paper in a agship Software Engineering conference. Our experience shows that current publication requirements do not guarantee replicability.
- A. Brooks, M. Roper, M. Wood, J. Daly, and J. Miller. Replication's role in software engineering. In Forrest Shull, Janice Singer, and Dag I. K. Sjøberg, editors, Guide to Advanced Empirical Software Engineering, pages 365--379. Springer London, 2008. 10.1007/978-1-84800-044-5 14.Google Scholar
- H.L. Dreyfus and S.E. Dreyfus. Mind over machine. Free Press, 1988.Google Scholar
- Marius Eriksen. Effective Sscala. http://twitter.github.com/effectivescala, 2012.Google Scholar
- Myles Hollander and Douglas A. Wolfe. Nonparametric Statistical Methods. John Wiley & Sons, Inc., 2nd edition, 1999.Google Scholar
- Cay S. Horstmann. Scala for the Impatient. Addison-Wesley, 2012. Google ScholarDigital Library
- Darrel C. Ince, Leslie Hatton, and John Graham-Cumming. The case for open computer programs. Nature, 482:485--488, 23 February 2012.Google ScholarCross Ref
- Barbara R. Jasny, Gilbert Chin, Lisa Chong, and Sacha Vignieri. Again, and again, and again.... Science, 334:1225, 2 December 2011.Google ScholarCross Ref
- Peter Norvig. Teach yourself programming in ten years. http://norvig.com/21-days.html, 2001.Google Scholar
- Martin Odersky. Scala levels: beginner to expert, application programmer to library designer. http://www.scala-lang.org/node/8610, 2011.Google Scholar
- Martin Odersky. The Sscala language specification version 2.9. Technical report, Programming Methods Laboratory, EPFL, Switzerland, May 24 2011.Google Scholar
- Victor Pankratius, Felix Schmidt, and Gilda Garretón. Combining functional and imperative programming for multicore software: An empirical study evaluating Scala and Java. In Proceedings of te 34th International Conference on Software Engineering, Zurich, June 6-9 2012. Google ScholarDigital Library
- Roger D. Peng. Reproducible research in computational science. Science, 334:1226--1227, 2 December 2011.Google ScholarCross Ref
- Victor R.Basili, Forrest Shull, and Filippo Lanubile. Building knowledge through families of experiments. IEEE Transactions on Software Engineering, 25(04):456--473, 1999. Google ScholarDigital Library
- M. Shaw. Writing good software engineering research papers. In Proceedings of the 25th International Conference on Software Engineering, 2003., pages 726--736, May 2003. Google ScholarDigital Library
- Forrest Shull, Jeffrey Carver, Sira Vegas, and Natalia Juristo. The role of replications in empirical software engineering. Empirical Software Engineering, 13:211--218, 2008. 10.1007/s10664-008-9060-1. Google ScholarDigital Library
- Dag I.K Sjøberg, J.E. Hannay, O. Hansen, V.B. Kampenes, A. Karahasanovic, N.-K. Liborg, and A.C. Rekdal. A survey of controlled experiments in software engineering. IEEE Transactions on Software Engineering, 31(9):733--753, Sept. 2005. Google ScholarDigital Library
- Bjarne Stroustrup. Posting to comp.lang.c++. http://www2.research.att.com/~bs/blast.html, December 1994.Google Scholar
- Dean Wampler and Alex Payne. Programming Scala. O'Reilly, 2009.Google Scholar
- Leon E. Winslow. Programming pedagogy a psychological overview. SIGCSE Bull., 28(3):17--22, September 1996. Google ScholarDigital Library
- Carmen Zannier, Grigori Melnik, and Frank Maurer. On the success of empirical studies in the International Conference on Software Engineering. In ICSE '06: Proceedings of the 28th international conference on Software engineering, pages 341--350, New York, NY, USA, 2006. ACM. Google ScholarDigital Library
Index Terms
- A note on rigour and replicability
Recommendations
Engaging the net generation with evidence-based software engineering through a community-driven web database
Software engineering faculty face the challenge of educating future researchers and industry practitioners regarding the generation of empirical software engineering studies and their use in evidence-based software engineering. In order to engage the ...
Communication Metrics for Software Development
We present empirical evidence that metrics on communication artifacts generated by groupware tools can be used to gain significant insight into the development process that produced them. We describe a test-bed for developing and testing communication ...
Evidence-Based Software Engineering for Practitioners
Software engineers might make incorrect decisions about adopting new techniques if they donýt consider scientific evidence about the techniquesý efficacy. Procedures used for evidence-based medicine can also apply to software engineering. Such evidence-...
Comments