Abstract
It has been argued that the Duhem problem is renewed with computational models since model assumptions having a representational aim and computational assumptions cannot be tested in isolation. In particular, while the Verification and Validation methodology is supposed to prevent such holism, Winsberg (Philos Compass 4:835–845, 2009; Science in the age of computer simulation, University of Chicago Press, Chicago, 2010) argues that verification and validation cannot be separated in practice. Morrison (Reconstructing reality: models, mathematics, and simulations, Oxford University Press, Oxford, 2015) replies that Winsberg overstates the entanglement between the steps. The paper aims at arbitrating these two positions, by stressing their respective validity in relation to domains of application. It importantly argues for an increasing use of formal methods in verification, that makes disentanglement possible.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
We can also distinguish ‘‘truncation errors’’, which are created by the discretization of equations (when one transforms the differential equations into approximate algebraic equations).
In other numerical methods, such as cellular automata or agent-based models, there are no discretization errors. We will focus on discretization-based numerical methods in this paper.
As pointed out by one of the anonymous reviewers, it is not clear that past errors are actually considered by Winsberg as reasons for weakness in V&V. That said, the following objections of Morrison hold insofar as Winsberg does argue that the usual given mathematical arguments are weak, and that strategies for sanctioning a simulation aim at providing grounds for belief that the simulation is reliable.
There is a discussion on a priori arguments and rigorous error analyses of computational methods in Fillion (2017).
For an overview of the scientific community of formal methods, see http://formal.epfl.ch/.
http://www.astree.ens.fr/, section ‘Industrial Applications’. See also Bozzano et al. (2017) for a discussion on formal methods for aerospace systems.
References
Baugh, J., & Altuntas, A. (2018). Formal methods and finite element analysis of hurricane storm surge: A case study in software verification. Science of Computer Programming, 158(15), 100–121.
Borälv, A., & Stalmarck, G. (1999). Formal verification in railways. In M. G. Hinchey & J. P. Bowe (Eds.), Industrial-strength formal methods in practice. Berlin: Springer.
Bozzano, M., Bruintjes, H., Cimatti, A., Katoen, J.-P., Noll, T., & Tonetta, S. (2017). Formal methods for aerospace systems—Achievements and challenges. In S. Nakajima, J.-P. Talpin, M. Toyoshima, & H. Yu (Eds.), Cyber-Physical System Design from an Architecture Analysis Viewpoint (pp. 133–159). Berlin: Springer.
Butler, R. W. (2001). What is formal methods? Last Updated: April 10, 2016. Consulted in January 2018. https://shemesh.larc.nasa.gov/fm/fm-what.html.
Clarke, E. (2008). The birth of model checking, 25 years of model checking., Lecture notes in computer science Berlin: Springer. https://doi.org/10.1007/2F978-3-540-69850-0_1.
Dowson, M. (1997). The ARIANE 5 software failure. Software Engineering Notes, 22(2), 84.
Fillion, N. (2017). The vindication of computer simulations. In M. Carrier & J. Lenhard (Eds.), Mathematics as a tool (pp. 137–156). Boston: Boston Studies in the Philosophy of Science.
Frigg, R., & Reiss, J. (2009). The philosophy of simulation: Hot new issues or same old stew? Synthese, 169(3), 593–613.
Jackson, D. (2012). Software abstractions: Logic, language, and analysis. London: MIT Press.
Jebeile, J., & Barberousse, A. (2016). Empirical agreement in model validation. Studies in History and Philosophy of Science Part A, 56, 168–174.
Karna, A. K., Chen, Y., Yu, H., Zhong, H., & Zhao, J. (2018). The role of model checking in software engineering. Frontiers of Computer Science, 12(4), 642–668.
Lenhard, J. (2018). Holism, or the erosion of modularity: a methodological challenge for validation. Philosophy of Science, 85(5) 832–844.
Lenhard, J., & Winsberg, E. (2010). Holism, entrenchment, and the future of climate model pluralism. Studies in History and Philosophy of Science Part B, 41(3), 253–262.
Lions, J. L. (1996). Ariane 5 Flight 501 Failure. Ariane 501 Inquiry Board Report (p. 4).
Morrison, M. (2014). Values and uncertainty in simulation models. Erkenntnis, 79(S5), 939–959.
Morrison, M. (2015). Reconstructing reality: Models, mathematics, and simulations. Oxford: Oxford University Press.
Oberkampf, W. L., & Trucano, T. G. (2002). Verification and validation in computational fluid dynamics. Rapport Sandia, SAND2002-0529.
Oberkampf, W. L., Trucano, T. G., & Hirsch, C. (2002). Verification, validation and predictive capacity in computational engineering and physics. Applied Mechanics Review, 57(5), 345.
Oreskes, N., Shrader-Frechette, K., & Belitz, K. (1994). Verification, validation, and confirmation of numerical models in the earth sciences. Science, 263(5147), 641–646.
Padilla, J. J., Diallo, S. Y., Lynch, C. J., & Gore, R. (2017). Observations on the practice and profession of modeling and simulation: A survey approach. Simulation, 94(6), 493–506.
Rushby, J. (1995). Formal methods and their role in the certification of critical systems. Technical Report CSL-95-1, March 1995.
Rushby, J. (2007). Automated formal methods enter the mainstream. Communications of the Computer Society of India, Formal Methods Theme Issue, 31(2), 28–32 (Archived in Journal of Universal Computer Science vol. 13, No. 5, pp. 650–660).
Skeel, R. (1992). SIAM News (Vol. 25, No. 4). https://w3.ual.es/~plopez/docencia/itis/patriot.htm. Accessed 20 Nov 2018
Trefethen, L. N. (1994). Finite difference and spectral methods for ordinary and partial differential equations, unpublished text. Available at http://people.maths.ox.ac.uk/trefethen/pdetext.html.
Wiels, V., Delmas, R., Doose, D., Garoche, P. L., & Cazin, J. (2012). Formal verification of critical aerospace software. AerospaceLab, 4, 1.
Winsberg, E. (2009). Computer simulation and the philosophy of science. Philosophy Compass, 4, 835–845.
Winsberg, E. (2010). Science in the age of computer simulation. Chicago: University of Chicago Press.
Winsberg, E. (2018). Philosophy and climate science. Cambridge: Cambridge University Press.
Woodcock, J., Larsen, P. G., Bicarregui, J., & Fitzgerald, J. (2009). Formal methods: Practice and experience. ACM Computing Surveys, 41(4), 1–36.
Acknowledgements
We thank the guest editors Andreas Kaminski and Michael Resch, as well as to the two anonymous referees for their helpful comments. The paper has also benefited from conversations with audience members at the SPSP Conference in Ghent, and notably with Johannes Lenhard and Nic Fillion.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This work was supported by “MOVE-IN Louvain” Incoming Post-doctoral Fellowship, cofunded by the Marie Curie Actions of the European Commission.
Rights and permissions
About this article
Cite this article
Jebeile, J., Ardourel, V. Verification and Validation of Simulations Against Holism. Minds & Machines 29, 149–168 (2019). https://doi.org/10.1007/s11023-019-09493-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11023-019-09493-8