Abstract
In formal verification, we verify that a system is correct with respect to a specification. When verification succeeds and the system is proven to be correct, there is still a question of how complete the specification is, and whether it really covers all the behaviors of the system. In this paper we study coverage metrics for model checking from a practical point of view. Coverage metrics are based on modifications we apply to the system in order to check which parts of it were actually relevant for the verification process to succeed. We suggest several definitions of coverage, suitable for specifications given in linear temporal logic or by automata on infinite words. We describe two algorithms for computing the parts of the system that are not covered by the specification. The first algorithm is built on top of automata-based model-checking algorithms. The second algorithm reduces the coverage problem to the model-checking problem. Both algorithms can be implemented on top of existing model checking tools.
Chapter PDF
Similar content being viewed by others
References
B. Banieqbal and H. Barringer. Temporal logic with fixed points. In Temporal Logic in Specification, LNCS 398, pp. 62–74, 1987.
D. Beaty and R. Bryant. Formally verifying a microprocessor using a simulation methodology. In Proc. 31st DAC, pp. 596–602. IEEE Computer Society, 1994.
I. Beer, S. Ben-David, C. Eisner, and Y. Rodeh. Efficient detection of vacuity in ACTL formulas. In Proc. 9th CAV, LNCS 1254, pp. 279–290, 1997.
G. Bhat and R. Cleaveland. Efficient local model-checking for fragments of the modal µ-calculus. In Proc. TACAS, LNCS 1055, 1996.
R. Bloem, H.N. Gabow, and F. Somenzi. An algorithm for strongly connected component analysis in n log n symbolic steps. In FMCAD, LNCS, 2000.
J.P. Bergmann and M.A. Horowitz. Improving coverage analysis and test generation for large designs. In Proc 11th CAD, pp. 580–584, November 1999.
R. Bloem, K. Ravi, and F. Somenzi. Efficient decision procedures for model checking of linear time logic properties. In Proc. 11th CAV, LNCS 1633, pp. 222–235, 1999.
J.R. Büchi. On a decision method in restricted second order arithmetic. In Proc. Internat. Congr. Logic, Method. and Philos. Sci. 1960, pp. 1–12, Stanford, 1962. Stanford University Press.
E.M. Clarke and E.A. Emerson. Design and synthesis of synchronization skeletons using branching time temporal logic. In Proc. Workshop on Logic of Programs, LNCS 131, pp. 52–71, 1981.
E.M. Clarke, O. Grumberg, K.L. McMillan, and X. Zhao. Efficient generation of counterexamples and witnesses in symbolic model checking. In Proc. 32nd DAC, pp. 427–432. IEEE Computer Society, 1995.
E.M. Clarke, O. Grumberg, and D. Peled. Model Checking. MIT Press, 1999.
H. Chockler, O. Kupferman, and M.Y. Vardi. Coverage metrics for temporal logic model checking. In TACAS, LNCS 2031, pp. 528–542, 2001.
R. Cleaveland and B. Steffen. A linear-time model-checking algorithm for the alternation-free modal µ-calculus. In Proc. 3rd CAD, LNCS 575, pp. 48–58, 1991.
S. Devadas, A. Ghosh, and K. Keutzer. An observability-based code coverage metric for functional simulation. In Proc. 8th CAD, pp. 418–425, 1996.
E.A. Emerson and C.-L. Lei. Efficient model checking in fragments of the propositional µ-calculus. In Proc. 1st LICS, pp. 267–278, Cambridge, June 1986.
F. Fallah, P. Ashar, and S. Devadas. Simulation vector generation from HDL descriptions for observability enhanced-statement coverage. In Proc. of the 36th DAC, pp. 666–671, June 1999.
F. Fallah, S. Devadas, and K. Keutzer. OCCOM: efficient computation of observability-based code coverage metrics for functional simulation. In Proc. of the 35th DAC, pp. 152–157, June 1998.
R.C. Ho and M.A. Horowitz. Validation coverage analysis for complex digital designs. In Proc 8th CAD, pp. 146–151, November 1996.
R.H. Hardin, Z. Har'el, and R.P. Kurshan. COSPAN. In Proc. 8th CAV LNCS 1102, pp. 423–427, 1996.
Y. Hoskote, T. Kam, P.-H Ho, and X. Zhao. Coverage estimation for symbolic model checking. In Proc. 36th DAC, pp. 300–305, 1999.
T.A. Henzinger, O. Kupferman, and S. Qadeer. From pre-historic to post-modern symbolic model checking. In Proc 10th CAV, LNCS 1427, 1998.
Y. Hoskote, D. Moundanos, and J. Abraham. Automatic extraction of the control flow machine and application to evaluating coverage of verification vectors. In Proc. of ICDD, pp. 532–537, October 1995.
R. Ho, C. Yang, M. Horowitz, and D. Dill. Architecture validation for processors. In Proc. of the 22nd Annual Symp. on Comp. Arch., pp. 404–413, June 1995.
[KGG99]-S. Katz, D. Geist, and O. Grumberg. “Have I written enough properties ?” a method of comparison between specification and implementation. In 10th CHARME, LNCS 1703, pp. 280–297, 1999.
M. Kantrowitz and L. Noack. I'm done simulating: Now what? verification coverage analysis and correctness checking of the DEC chip 21164 alpha microprocessor. In Proc. 33th DAC, pp. 325–330, June 1996.
D. Kozen. Results on the propositional µ-calculus. Theoretical Computer Science, 27:333–354, 1983.
O. Kupferman and A. Pnueli. Once and for all. In Proc. 10th IEEE Symp. on Logic in Comp. Sci., pp. 25–35, San Diego, June 1995.
R.P. Kurshan. FormalCheck User’s Manual. Cadence Design, Inc., 1998.
O. Kupferman and M.Y. Vardi. Relating linear and branching model checking. In IFIP Work. Conf. on Programming Concepts and Methods, pp. 304–326, New York, June 1998. Chapman & Hall.
O. Kupferman and M.Y. Vardi. Vacuity detection in temporal model checking. In 10th CHARME, LNCS 1703, pp. 82–96, 1999.
O. Kupferman, M.Y. Vardi, and P. Wolper. An automata-theoretic approach to branching-time model checking. Journal of the ACM, 47(2):312–360, March 2000.
O. Lichtenstein and A. Pnueli. Checking that finite state concurrent programs satisfy their linear specification. In Proc. 12th POPL, pp. 97–107, 1985.
D. Moumdanos, J.A. Abraham, and Y.V. Hoskote. Abstraction techniques for validation coverage analysis and test generation. IEEE Trans. on Computers, 1998.
J.P. Queille and J. Sifakis. Specification and verification of concurrent systems in Cesar. In Proc. 5th Int. Symp. on Programming, LNCS 137, pp. 337–351, 1981.
A.P. Sistla. Satefy, liveness and fairness in temporal logic. Formal Aspects of Computing, 6:495–511, 1994.
M.Y. Vardi and P. Wolper. Reasoning about infinite computations. Information and Computation, 115(1):1–37, November 1994.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Chockler, H., Kupferman, O., Kurshan, R.P., Vardi, M.Y. (2001). A Practical Approach to Coverage in Model Checking. In: Berry, G., Comon, H., Finkel, A. (eds) Computer Aided Verification. CAV 2001. Lecture Notes in Computer Science, vol 2102. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44585-4_7
Download citation
DOI: https://doi.org/10.1007/3-540-44585-4_7
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-42345-4
Online ISBN: 978-3-540-44585-2
eBook Packages: Springer Book Archive