Skip to main content
SpringerLink
Log in

JWalk: a tool for lazy, systematic testing of java classes by design introspection and user interaction

  • Published:
Automated Software Engineering Aims and scope Submit manuscript

Abstract

Popular software testing tools, such as JUnit, allow frequent retesting of modified code; yet the manually created test scripts are often seriously incomplete. A unit-testing tool called JWalk has therefore been developed to address the need for systematic unit testing within the context of agile methods. The tool operates directly on the compiled code for Java classes and uses a new lazy method for inducing the changing design of a class on the fly. This is achieved partly through introspection, using Java’s reflection capability, and partly through interaction with the user, constructing and saving test oracles on the fly. Predictive rules reduce the number of oracle values that must be confirmed by the tester. Without human intervention, JWalk performs bounded exhaustive exploration of the class’s method protocols and may be directed to explore the space of algebraic constructions, or the intended design state-space of the tested class. With some human interaction, JWalk performs up to the equivalent of fully automated state-based testing, from a specification that was acquired incrementally.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Agitar Software: Agitator. http://www.agitar.com/products/20040518-agitator.html. Accessed 12 February 2007

  • Ball, T., Larus, J.R.: Using paths to measure, explain and enhance program behavior. IEEE Comput. 33(7), 57–65 (2000)

    Google Scholar 

  • Beck, K.: Extreme Programming Explained: Embrace Change, 1st edn. Addison–Wesley, New York (2000)

    Google Scholar 

  • Beck, K.: The JUnit Pocket Guide, 1st edn. O’Reilly, Beijing (2004)

    Google Scholar 

  • Beck, K.: Extreme Programming Explained: Embrace Change, 2nd edn. Addison–Wesley, New York (2005)

    Google Scholar 

  • Bernot, B., Gaudel, M.-C., Marre, B.: Software testing based on formal specifications: a theory and a tool. Softw. Eng. J. 6(6), 387–405 (1991)

    Article  Google Scholar 

  • Bezier, B.: Software Testing Techniques. International Thomson Computer Press (1990)

  • Binder, R.V.: Testing object-oriented software: a survey, Softw. Test. Verif. Reliab. 6(3/4), 125–252 (1996)

    Article  Google Scholar 

  • Binder, R.V.: TOOTSIE, a high-end OO development environment. http://www.rbsc.com/pages/tootsie.html. Accessed 28 April 2006

  • Boshernitsan, M., Doong, R., Savoia, A.: From Daikon to Agitator: Lessons and challenges in building a commercial tool for developer testing. In: Proc. 5th ACM Sigsoft Int. Symp. on Softw. Testing and Analysis, Portland, ME, pp. 169–180 (2006)

  • Boyapati, C., Khurshid, S., Marinov, D.: Korat: automated testing based on Java predicates. In: Proc. ACM Sigsoft 3rd Int. Symp. on Softw. Test. and Analysis (ISSTA ’02), Rome, Italy, pp. 123–133 (2002)

  • Buy, U., Orso, A., Pezzè, M.: Automated testing of classes. In: Proc. 2nd ACM Sigsoft Int. Symp. on Softw. Testing and Analysis, Portland, OR, pp. 39–48 (2000)

  • Chen, H.Y., Tse, T.H., Chan, F.T., Chen, T.Y.: In black and white: an integrated approach to class-level testing of object-oriented programs. ACM Trans. Softw. Eng. Methods 7(3), 250–295 (1998)

    Article  Google Scholar 

  • Chen, H.Y., Tse, T.H., Chen, T.Y.: TACCLE: a methodology for object-oriented software testing at the class and cluster levels. ACM Trans. Softw. Eng. Methods 10(1), 56–109 (2001)

    Article  Google Scholar 

  • Cheon, Y., Leavens, G.T.: A simple and practical approach to unit testing: the JML and JUnit way. In: Proc. 16th European Conf. Obj.-Oriented Progr., Malaga, Spain, Lecture Notes in Computer Science, vol. 2374, pp. 231–255. Springer, Berlin (2002)

    Google Scholar 

  • Chow, T.: Testing software design modeled by finite state machines. IEEE Trans. Softw. Eng. 4(3), 178–187 (1978)

    Article  Google Scholar 

  • Csallner, C., Smaragdakis, Y.: JCrasher: An automatic robustness tester for Java. Softw. Pract. Exp. 34(11), 1025–1050 (2004)

    Article  Google Scholar 

  • Csallner, C., Smaragdakis, Y.: DSD-Crasher: a hybrid analysis tool for bug finding. In: Proc. 5th ACM Sigsoft Int. Symp. on Softw. Testing and Analysis, Portland, MN, pp. 245–254 (2006a)

  • Csallner, C., Smaragdakis, Y.: Dynamically discovering likely interface specifications. In: Proc. 28th Int. Conf. on Softw. Eng. (ICSE ’06), Shanghai, pp. 861–864 (2006b)

  • De Scheemaecker, M.: NanoXML 2.2.1. Sourceforge. http://nanoxml.sourceforge.net/orig/. Accessed 23 February 2007

  • Doong, R.K., Frankl, P.: The ASTOOT approach to testing object-oriented programs. ACM Trans. Softw. Eng. Methods 3(4), 101–130 (1994)

    Article  Google Scholar 

  • Doong, R.K., Frankl, P.: Case studies on testing object-oriented programs. In: Proc. 4th Symp. Softw. Testing, Analysis and Verif., pp. 165–177. ACM Press, New York (1991)

    Google Scholar 

  • Dranidis, D., Tigka, K., Kefalas, P.: Formal modelling of use cases with X-machines. In: Proc. 1st South East European Workshop on Formal Methods, Thessaloniki, Greece, pp. 72–83 (2004)

  • D’Souza, D.F., Wills, A.C.: Objects, Components and Frameworks with UML: the Catalysis Approach. Addison–Wesley, Reading (1999)

    Google Scholar 

  • Eclipse Foundation: Eclipse: an open development platform. http://www.eclipse.org/. Accessed 14 February 2007

  • Ernst, M.D.: Dynamically discovering likely program invariants. PhD thesis, University of Washington Department of Computer Science and Engineering, Seattle, Washington (2000)

  • Ernst, M.D., Cockrell, J., Griswold, W.G., Notkin, D.: Dynamically discovering likely program invariants to support program evolution. IEEE Trans. Softw. Eng. 27(2), 99–123 (2001)

    Article  Google Scholar 

  • Ernst, M.D., Perkins, J.H., Guo, P.J., McCamant, S., Pacheco, C., Tschantz, M.S., Xiao, C.: The Daikon system for dynamic detection of likely invariants. Sci. Comput. Program. (2007, in press)

  • Gamma, E., Helm, R., Johnson, R., Vlissides, J.: Design Patterns: Elements of Reusable Object-Oriented Software. Addison–Wesley, Reading (1995)

    Google Scholar 

  • Goguen, J., Winkler, T., Meseguer, J., Futatsugi, K., Jouannaud, J.-P.: Introducing OBJ. Technical Report, Oxford Programming Research Group and SRI International, Menlo Park, CA (1993)

  • Goguen, J., Malcolm, G.: Algebraic Semantics of Imperative Programs. MIT Press, Cambridge (1997)

    Google Scholar 

  • Grieskamp, W., Gurevich, Y., Schulte, W., Veanes, M.: Generating finite state machines from abstract state machines. In: Proc. 3rd ACM Sigsoft Int. Symp. on Softw. Testing and Analysis (ISSTA ’02), Rome, Italy, pp. 112–122 (2002)

  • Henkel, J., Diwan, A.: Discovering algebraic specifications from Java classes. In: Proc. 17th. European Conf. Obj.-Oriented Progr., Darmstadt, Germany. Lecture Notes in Computer Science, vol. 2743, pp. 431–456. Springer, Berlin (2003)

    Google Scholar 

  • Henkel, J., Diwan, A.: A tool for writing and debugging algebraic specifications. In: Proc. 26th Int. Conf. Softw. Eng., pp. 449–458. IEEE Computer Society, Los Alamitos (2004)

    Chapter  Google Scholar 

  • Holcombe, W.M.L.: Where do unit tests come from? In: Proc. 4th Int. Conf. on Extreme Progr. and Flexible Proc. in Softw. Eng., Genova, Italy, Lecture Notes in Computer Science, vol. 2675, pp. 161–169. Springer, Berlin (2003)

    Google Scholar 

  • Holcombe, M., Bogdanov, K., Gheorghe, M.: Functional test generation for eXtreme Programming. In: Proc. 2nd Int. Conf. on Extreme Progr. and Flexible Proc. in Softw. Eng., Sardinia, Italy, pp. 109–113 (2001)

  • Holcombe, W.M.L., Ipate, F.: Correct Systems: Building a Business Process Solution. Applied Computing Series. Springer, Berlin (1998)

    Google Scholar 

  • Ipate, F., Holcombe, W.M.L.: An integration testing method that is proved to find all faults. Int. J. Comput. Math. 63, 159–178 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  • IPL (Information Processing, Ltd., UK): Cantata++ for testing C, C++ and Java. http://www.ipl.com/. Accessed 12 February 2007

  • Irvine, A., Offutt, A.: The effectiveness of category partition testing of object-oriented software. ISSE Department, George Mason University, Fairfax, VA (1995)

  • Jard, C., von Bochmann, G.: An approach to testing specifications. J. Syst. Softw. 3(4), 315–323 (1983)

    Article  Google Scholar 

  • JUnit, The JUnit project website. http://www.junit.org/. Accessed 12 February 2007

  • Kiczales, G., des Rivieres, J., Bobrow, D.G.: The Art of the Metaobject Protocol. MIT Press, Cambridge (1991)

    Google Scholar 

  • Lerda, F., Visser, W.: Addressing dynamic issues of program model checking, In: Proc. 8th Int. SPIN Workshop (SPIN ’01), Toronto, pp. 80–102 (2001)

  • Lorenzoli, D., Mariani, L., Pezzè, M.: Inferring state-based behavior models. In: Workshop on Dynamic Analysis (WODA ’06), Shanghai, China, pp. 25–32 (2006)

  • Marinov, D., Khurshid, S.: TestEra: A novel framework for testing Java programs. In: Proc. 16th IEEE Conf. Automated Softw. Eng. (ASE ’01), San Diego, CA, pp. 22–31 (2001)

  • McGregor, J.D.: Constructing functional test cases using incrementally-derived state machines. In: Proc. 11th Int. Conf. on Testing Computer Software, USPDI, Washington (1994)

  • Parasoft: Parasoft Jtest \(\textcircled{R}\) product description. http://www.parasoft.com/, Parasoft, Monrovia, CA. Accessed 22 August 2007

  • Pacheco, C., Lahiri, S. K., Ernst, M. D., Ball, T.: Feedback-directed random test generation. In: Proc. 29th Int. Conf. Softw. Eng., Minneapolis, MN, USA, pp. 75–84. IEEE Computer Society, Los Alamitos (2007)

    Google Scholar 

  • Pacheco, C., Ernst, M.D.: Eclat: Automatic generation and classification of test inputs. In: Proc. 19th European Conf. Obj.-Oriented Prog., pp. 504–527 (2005)

  • Perkins, J.H., Ernst, M.D.: Efficient incremental algorithms for dynamic detection of likely invariants. In: Proc. ACM Sigsoft 12th Symp. Found. Softw. Eng. (FSE ’04), Newport, CA, pp. 23–32 (2004)

  • Petrenko, A., Boroday, S., Groz, R.: Confirming configurations in EFSM testing. IEEE Trans. Softw. Eng. 30(1), 29–42 (2004)

    Article  Google Scholar 

  • Robillard, P.N., Kruchten, P.: Software Engineering Process with the UPEDU. Addison–Wesley, Reading (2002)

    Google Scholar 

  • Simons, A.J.H.: Testing with guarantees and the failure of regression testing in eXtreme Programming. In: Proc. 6th Int. Conf. on Extreme Progr. and Flexible Proc. in Soft. Eng. Lecture Notes in Computer Science, vol. 3556, pp. 118–126. Springer, Sheffield (2005)

    Google Scholar 

  • Simons, A.J.H.: A theory of regression testing for behaviourally compatible object types. Softw. Test. Verif. Reliab. 16(3), 133–156 (2006)

    Article  Google Scholar 

  • Simons, A.J.H.: JWalk: Lazy systematic class unit testing. http://www.dcs.shef.ac.uk/~ajhs/jwalk/. Accessed 12 February 2007

  • Simons, A.J.H., Thomson, C.D.: Lazy systematic testing: JWalk versus JUnit. In: Proc. 2nd Testing in Academia and Industry Conference—Practice and Research Techniques (TaicPart ’07), Windsor Great Park, London, p. 138. IEEE Computer Society, Los Alamitos (2007)

    Google Scholar 

  • SIR: Software-artifact infrastructure repository: the nanoxml project. http://sir.unl.edu/content/sir.html. Accessed 22 February 2007

  • Tillmann, N., Schulte, W.: Parameterized unit tests. In: Proc. 5th European Softw. Eng. Conf. and ACM Sigsoft Symp. on Found. Softw. Eng. (ESEC/FSE ’05), pp. 253–262 (2005a)

  • Tillmann, N., Schulte, W.: Parameterized unit tests with Unit Meister. In: Proc. 5th European Softw. Eng. Conf. and ACM Sigsoft Symp. on Found. Softw. Eng. (ESEC/FSE ’05), pp. 241–244 (2005b)

  • Ural, H., Saleh, K., Williams, A.W.: Test generation based on control and data dependencies within system specifications in SDL. Comput. Commun. 23(7), 609–627 (2000)

    Article  Google Scholar 

  • Visser, W., Havelund, K., Brat, G., Park, S., Lerda, F.: Model checking programs. Autom. Softw. Eng. J. 10(2), 203–232 (2003)

    Article  Google Scholar 

  • Visser, W., Pasareaunu, C.S., Pelánek, R.: Test input generation for Java containers using state matching. In: Proc. 5th ACM Sigsoft Int. Symp. Softw. Testing and Analysis (ISSTA ’06), Portland, MN, pp. 37–48 (2006)

  • Xie, T., Marinov, D., Notkin, D.: Rostra: a framework for detecting redundant object-oriented unit tests. In: Proc. 19th IEEE Conf. Automated Softw. Eng. Washington DC, pp. 196–205 (2004)

  • Xie, T., Marinov, D., Schulte, W., Notkin, D.: Symstra: a framework for generating object-oriented unit tests using symbolic execution. In: Proc. Tools and Algorithms for the Construction and Analysis of Systems (TACAS/ETAPS ‘05), Edinburgh, pp. 365–381 (2005)

  • Xie, T., Notkin, D.: Tool-assisted unit test selection based on operational violations. In: Proc. 18th IEEE Int. Conf. Automated Softw. Eng. (ASE ’03), Montreal, Canada, pp. 40–48 (2003)

  • Yuan, H., Xie, T.: Automatic extraction of abstract-object-state machines based on branch coverage. In: Proc. 1st Int. Workshop on Reverse Engineering to Requirements (RETR ’05), Pittsburgh, PA, pp. 5–11 (2005)

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Sheffield, Sheffield, UK

    Anthony J. H. Simons

Authors
  1. Anthony J. H. Simons
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anthony J. H. Simons.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Simons, A.J.H. JWalk: a tool for lazy, systematic testing of java classes by design introspection and user interaction. Autom Softw Eng 14, 369–418 (2007). https://doi.org/10.1007/s10515-007-0015-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10515-007-0015-3

Keywords

Search

Navigation