Skip to main content
SpringerLink
Log in

SPARK 2014 and GNATprove

A competition report from builders of an industrial-strength verifying compiler

  • VerifyThis 2012
  • Published:
International Journal on Software Tools for Technology Transfer Aims and scope Submit manuscript

Abstract

Extensive and expensive testing is the method most widely used for gaining confidence in safety-critical software. With a few exceptions, such as SPARK, formal verification is rarely used in industry due to its high cost and level of skill required. The grand challenge of building a verifying compiler for static formal verification of programs aims at bringing formal verification to non-expert users of powerful programming languages. This challenge has nurtured competition and collaboration among verification tool builders; an example is the VerifyThis competition Huisman et al. (http://digbib.ubka.uni-karlsruhe.de/volltexte/1000034373, 2013). In this paper, we describe our approach to popularising formal verification in the design of the SPARK 2014 language and the associated formal verification tool GNATprove. In particular, we present our solution to combining tests and proofs, which provides a cost-competitive way to develop software to standards such as do-178. At the heart of our technique are executable contracts, and the ability to both test and prove those. We use running examples from the VerifyThis 2012 competition and discuss the results of using our tools on those problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Barnes, J.: Ada 2012 rationale (2012). http://www.adacore.com/knowledge/technical-papers/ada-2012-rationale/

  2. Barnes, J.: SPARK: The Proven Approach to High Integrity Software. Altran Praxis. Third Edition of the SPARK Book (2012)

  3. Burdy, L., Cheon, Y., Cok, D.R., Ernst, M.D., Kiniry, J.R., Leavens, G.T., Leino, K.R.M., Poll, E.: An overview of JML tools and applications. Int. J. Softw. Tools Technol. Transf. 7, 212–232 (2005)

  4. Bobot, F., Conchon, S., Contejean, E., Lescuyer, S.: Implementing polymorphism in SMT solvers. In: SMT’08, vol. 367 of ACM ICPS, pp. 1–5 (2008)

  5. Barnett, M., Fähndrich, M., Leino, K.R.M., Müller, P., Schulte, W., Venter, H.: Specification and verification: the Spec# experience. Commun. ACM 54(6), 81–91 (2011)

  6. Bobot, F., Filliâtre, J.-C., Paskevich, A., Marché, C.: Why3: shepherd your herd of provers. In: Proceedings of the First International Workshop on Intermediate Verification Languages, Boogie (2011)

  7. Beckert, B., Gladisch, C., Tyszberowicz, S.S., Yehudai, A.: KeYGenU: combining verification-based and capture and replay techniques for regression unit testing. Int. J. Syst. Assur. Eng. Manag. 2(2), 97–113 (2011)

  8. Barrett, C., Tinelli, C.: CVC3. In: Damm, W., Hermanns, H. (eds.) Proceedings of the 19th International Conference on Computer Aided Verification (CAV ’07), vol. 4590 of Lecture Notes in Computer Science, pp. 298–302. Springer, Berlin (2007)

  9. Cohen, E., Dahlweid, M., Hillebrand, M., Leinenbach, D., Moskal, M., Santen, T., Schulte, W., Tobies, T.: VCC: a practical system for verifying concurrent C. In Proceedings of the 22nd International Conference on Theorem Proving in Higher Order Logics, TPHOLs ’09, pp. 23–42. Springer, Berlin, Heidelberg (2009)

  10. Cadar, C., Ganesh, V., Pawlowski, P.M., Dill, D.L., Engler, D.R..: EXE: automatically generating inputs of death. In: Proceedings of the 13th ACM Conference on Computer and Communications Security, CCS ’06, pp. 322–335. ACM, New York (2006)

  11. Couchot, J.-F., Giorgetti, A., Stouls, N.: Graph Based Reduction of Program Verification Conditions. In: Saïdi, H., Shankar, N. (eds.) AFM’09, Automated Formal Methods (co-located with CAV’09), pp. 40–47, Grenoble, France. ACM Press (2009)

  12. Chalin, P.: JML support for primitive arbitrary precision numeric types: definition and semantics. J. Object Technol. 3(6), 57–79 (2004)

  13. Chalin, P.: Engineering a sound assertion semantics for the verifying compiler. IEEE Trans. Softw. Eng. 36(2), 275–287 (2010)

  14. Claessen, K., Johansson, M., Rosen, D., Smallbone, N.: HipSpec: automating inductive proofs of program properties. In: Fleuriot, J., Höfner, P., McIver, A., Smaill, A. (eds.) ATx’12/WInG’12, vol. 17 of EPiC Series, pp. 16–25. EasyChair (2013)

  15. Comar, C., Kanig, J., Moy, Y.: Integrating formal program verification with testing. In: Proceedings of Embedded Real Time Software and Systems, Toulouse (2012)

  16. Claessen, K., Smallbone, N., Hughes, J.: QuickSpec: guessing formal specifications using testing. In: Proceedings of the 4th International Conference on Tests and Proofs, TAP’10, pp. 6–21. Springer, Berlin, Heidelberg (2010)

  17. Dross, C., Conchon, S., Kanig, J., Paskevich, A.: Reasoning with triggers. In: Fontaine, P., Goel, A. (eds.) SMT 2012, vol. 20 of EPiC Series, pp. 22–31. EasyChair (2013)

  18. Dross, C., Efstathopoulos, P., Lesens, D., Mentré, D., Moy, Y: Rail, space, security: three case studies for spark 2014. In Proceedings of ERTS (2014)

  19. Dross, C., Filliâtre, J.-C., Moy Y.: Correct code containing containers. In: 5th International Conference on Tests and Proofs (TAP’11), Zurich (2011)

  20. de Moura, L.M., Bjørner, N.: Proofs and refutations, and Z3. In: Rudnicki, P., Sutcliffe, G., Konev, B., Schmidt, R.A., Schulz, S. (eds.) LPAR Workshops, vol. 418 of CEUR Workshop Proceedings. CEUR-WS.org (2008)

  21. Ernst, M.D., Perkins, J.H., Guo, P.J., McCamant, S., Pacheco, C., Tschantz, M.S., Xiao, C.: The Daikon system for dynamic detection of likely invariants. Sci. Computer Progr. 69(1–3), 35–45 (2007)

  22. Guitton, J., Kanig, J., Moy, Y.: Why Hi-Lite Ada? In: Proceedings of Boogie 2011, the 1st International Workshop on Intermediate Language Verification (2011)

  23. Godefroid, P., Klarlund, N., Sen, K.: DART: directed automated random testing. In: Proceedings of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’05, pp. 213–223. ACM, New York (2005)

  24. Hi-Lite: Simplifying the use of formal methods. http://www.open-do.org/projects/hi-lite/ (2013)

  25. Huisman, M., Klebanov, V., Monahan, R.: VerifyThis verification competition 2012—organizer’s report. Technical Report 2013–01, Department of Informatics, Karlsruhe Institute of Technology. http://digbib.ubka.uni-karlsruhe.de/volltexte/1000034373 (2013)

  26. Hatcliff, J., Leavens, G.T., Leino, K.R.M., Müller, P., Parkinson, M.J.: Behavioral interface specification languages. ACM Comput. Surv. 44(3), 16 (2012)

  27. Hoare, C.A.R.: An axiomatic basis for computer programming. Commun. ACM 12, 576–580 (October 1969)

  28. Hoare, T.: The verifying compiler: a grand challenge for computing research. J. ACM 50, 2003 (2003)

  29. Horwitz, S., Reps, T., Binkley, D.: Interprocedural slicing using dependence graphs. In: Proceedings of the ACM SIGPLAN 1988 conference on Programming Language design and Implementation, PLDI ’88, pp. 35–46. ACM, New York (1988)

  30. Kanig, J., Schonberg, E., Dross, C.: Hi-Lite: the convergence of compiler technology and program verification. In: Proceedings of the 2012 ACM conference on High integrity language technology, HILT ’12, pp. 27–34. ACM, New York (2012)

  31. Kühlwein, D., van Laarhoven, T., Tsivtsivadze, E., Urban, J., Heskes, T. Overview and evaluation of premise selection techniques for large theory mathematics. In: Proceedings of the 6th international joint conference on Automated Reasoning, IJCAR’12, pp. 378–392. Springer, Berlin, Heidelberg (2012)

  32. Leino, K.R.M.: Efficient weakest preconditions. Inf. Process. Lett. 93(6), 281–288 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  33. Meyer, B.: Object-Oriented Software Construction, 1st edn. Prentice-Hall Inc, Upper Saddle River (1988)

  34. O’Neill, I.: SPARK—a language and tool-set for high-integrity software development. In: Boulanger, J.-L. (ed.) Industrial Use of Formal Methods: Formal Verification. Wiley, New York (2012)

  35. RTCA. DO-178B: software considerations in airborne systems and equipment certification (1992)

  36. RTCA. DO-178C: software considerations in airborne systems and equipment certification (2011)

  37. Schanda, F., Brain, B: Using answer set programming in the development of verified software. In: Dovier, A., Santos Costa, V. (eds.) Technical Communications of the 28th International Conference on Logic Programming (ICLP’12), vol. 17 of Leibniz International Proceedings in Informatics (LIPIcs), pp. 72–85. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, Dagstuhl (2012)

  38. Sen, K., Marinov, D., Agha, G: CUTE: a concolic unit testing engine for c. In: Proceedings of the 10th European Software Engineering Conference Held Jointly with 13th ACM SIGSOFT International Symposium on Foundations of Software Engineering, ESEC/FSE-13, pp. 263–272. ACM, New York (2005)

  39. Tschannen, J., Furia, C.A., Nordio, M., Meyer, B.: Usable verification of object-oriented programs by combining static and dynamic techniques. In: Proceedings of the 9th International Conference on Software Engineering and Formal Methods, SEFM’11, pp. 382–398. Springer, Berlin, Heidelberg (2011)

  40. Woodcock, J., Larsen, P.G., Bicarregui, J., Fitzgerald, J.: Formal methods: practice and experience. ACM Comput. Surv 41(4), 19:1–19:36 (2009)

Download references

Acknowledgments

The authors would like to thank the anonymous reviewers for their valuable comments and suggestions to improve the quality of this report.

Author information

Authors and Affiliations

  1. Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland

    Duc Hoang

  2. AdaCore, Paris, France

    Yannick Moy

  3. Altran UK, Bath, UK

    Angela Wallenburg & Roderick Chapman

Authors
  1. Duc Hoang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yannick Moy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Angela Wallenburg
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Roderick Chapman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Angela Wallenburg.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hoang, D., Moy, Y., Wallenburg, A. et al. SPARK 2014 and GNATprove. Int J Softw Tools Technol Transfer 17, 695–707 (2015). https://doi.org/10.1007/s10009-014-0322-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10009-014-0322-5

Keywords

Search

Navigation