Elsevier

Computer Communications

Volume 26, Issue 16, 15 October 2003, Pages 1839-1850
Computer Communications

Towards a framework for evaluating certificate status information mechanisms

https://doi.org/10.1016/S0140-3664(03)00079-3Get rights and content

Abstract

A wide spectrum of certificate revocation mechanisms is currently in use. A number of them have been proposed by standardisation bodies, while some others have originated from academic or private institutions. What is still missing is a systematic and robust framework for the sound evaluation of these mechanisms. We present a mechanism-neutral framework for the evaluation of certificate status information (CSI) mechanisms. These mechanisms collect, process and distribute CSI. A detailed demonstration of its exploitation is also provided. The demonstration is mainly based on the evaluation of Certificate Revocation Lists, as well as of the Online Certificate Status Protocol. Other well-known CSI mechanisms are also mentioned for completeness.

Introduction

The deployment of Public Key Infrastructures, as an e-commerce and e-business enabling technology has been extensively studied. PKI applications could now be used on a large scale to enable electronic services, such as e-government, B2B transactions and e-commerce. It has rapidly become clear that these services would become successful only if PKI users can trust digital certificates and the respective digital signatures. In this context, ‘trusting a digital signature’ means that the user who validates a digital signature has obtained enough evidence to believe that the signature was computed by the intended producer of the digital signature as stated in the certificate that corresponds to that digital signature, and that the complete certificate chain is still valid. Consequently, a digital signature is not to be trusted if one of those criteria is not met.

In this paper, we present an evaluation framework for mechanisms that collect, process and distribute information pertinent to the validity of digital signatures and certificates. This framework deals with revocation of the issued certificates, and the way revocation information can be made available to certificate users, using Certificate Revocation Lists (CRLs), CRLs variants, OCSP, etc. An early version of this paper has been published at the ISSE2000 conference in Barcelona, Spain [1].

The evaluation methods described in this paper strive to avoid unnecessary biases. The framework comprises of a set of qualitative and quantitative evaluation criteria, which can be applied to any mechanism that updates information on the status of certificates (Certificate Status Information—CSI). We use this model as a tool to identify potential problems in the mechanisms in a methodical way. Our evaluation framework splits the evaluation process into three main domains, namely management, performance and security.

Management of revocation mechanisms includes the way these mechanisms operate, the way information processing is being performed, the participating entities, and the respective timeframe.

Performance of revocation mechanisms refers to the efficiency characteristics of those mechanisms. These characteristics include the timeliness of the mechanism, the freshness of information it delivers, the scalability and adjustability of the mechanism, and the capability to immediately generate information on the status of a certificate (emergency certificate status information).

The security aspect of revocation mechanisms covers issues related to protecting the operation of the mechanisms themselves, as well as of the information they produce. Certificate status information has to be protected while generated, communicated, and stored.

While designing the evaluation framework, we took into consideration the requirements and restrictions imposed on the use of these mechanisms by the modus operandi required by the European Directive on a Community Framework for Electronic Signatures [2], the EESSI Expert Team Report [3] and the NIST PKI study [4], [5]. The framework can be used to evaluate mechanisms that are operated by a Certification Service Provider (CSP) that issues ‘qualified certificates’ [2], [3]. Certificates are considered to be ‘qualified’ [2] if they meet the requirements set forth in Annex I of the Directive and are provided by a CSP meeting the requirements laid out in Annex II of the Directive. We have also considered the draft or final requirements and recommendations contained in Refs. [6], [7], [8], [9], [10], [11], [12]. Most European countries have rewritten their legislation so that digital signatures produced with a secure signature creation device must be considered as a handwritten signature if that digital signature comes with a qualified certificate.

We demonstrate the application of our framework by evaluating in detail the two most frequently used mechanisms: CRLs [6], [9], and the Online Certificate Status Protocol [11]. Other techniques that are less frequently used, but are also of interest, include the Certificate Revocation Status mechanism [13], Suicide Notes [14], Revocation Authority [15], Authenticated Dictionaries [16] and Certificate Revocation Trees [17]. These techniques are discussed in less detail.

This paper is organised as follows: In Section 2, we present the aforementioned CSI mechanisms, while Section 3 deals with the evaluation framework for CSI mechanisms. In Section 4, we describe two representative CSI mechanisms and evaluate them based on the framework presented in Section 3. In Section 5, we provide a discussion, based on the evaluation presented in Section 4 and the evaluation framework presented in Section 3, while Section 6 contains our concluding remarks.

Section snippets

Classification of CSI-retrieval mechanisms

In this section, we provide an overview of CSI mechanisms. There are three distinct mechanism categories: mechanisms that provide negative CSI, mechanisms that provide positive CSI, and those that provide complete certificate status information. Each of these categories is discussed in the following subsections.

Evaluation framework

The Electronic Signature Directive (Annex II) requires ‘the operation of a prompt and secure directory and a secure and immediate revocation service’. Furthermore, the Directive requires that the authenticity and validity of the certificate required at the time of signature verification are reliably verified, and that the verification result and the signatory's identity are correctly displayed. The EU Directive on a Community Framework for Electronic Signatures also requires that the date and

Evaluation of CSI-retrieval mechanisms

In this section, we apply our evaluation framework to CRLs [6], [9] and the OCSP [11].

Certain evaluation criteria are accompanied by quantitative metrics that can, in turn, be used in order to investigate possible enhancements to the CSI mechanisms we examine. The notation that will be used for providing quantitative metrics is explained in Table 1.

Discussion

We evaluated CRL variants and OCSP, using our evaluation framework, and we defined qualitative and quantitative metrics for estimating the timeliness, freshness and scalability of these mechanisms. The criterion of adjustability is not met by any of the mechanisms; therefore, freshness and other metrics are not adjustable by the entities that take the risk of trusting CSI.

Furthermore, none of the mechanisms we presented meets the feedback criterion, which we believe is crucial for the efficient

Conclusion

The evaluation framework we presented can be used by the research community for further research on CSI mechanisms, either for improving the existing ones or for developing new ones. The industry can also make use of this framework; until now, high-level PKI requirements were used on specific CSI mechanisms, based on empirical methods or ad hoc research. Our framework can be used by PKI implementers and policy makers to select the CSI mechanism or mechanisms that will be used in a PKI,

Acknowledgements

This work was partially funded by the European Commission (Directorate General III, contract #ETD/99/502536: ‘Study on the Scalability of Certificate Revocation and Certificate Suspension and Proposals for Enhancements on the Respective Mechanisms’). This work was also partially supported by the Concerted Research Action (GOA) Mefisto-666-2000/06 of the Flemish Government.

John Iliadis holds a BSc in Information Systems Engineering from the Department of Informatics, Technological Educational Institute of Athens, Greece. He also holds an MSc in Information Security from the Department of Mathematics, Royal Holloway College, University of London, UK. He is currently pursuing a PhD in Information and Communication Systems Security under the supervision of Prof. S. Gritzalis, at the Department of Information and Communication Systems Engineering, University of the

References (23)

  • I.S. Iliadis, D. Spinellis, S. Katsikas, B. Preneel, A Taxonomy of Certificate Status Information Mechanisms,...
  • Directive 1999/93/EC of the European Parliament and of the Council on a Community framework for electronic signatures,...
  • H. Nilsson, P. Van Eecke, M. Medina, D. Pinkas, N. Pope, European Electronic Signature Standardization Initiative,...
  • S. Berkovits, S. Chokhani, J.A. Furlong, J.A, Geiter, J.C. Guild, Public Key Infrastructure Study: Final Report....
  • US National Institute of Standards and Technology, A Public Key Infrastructure for US Government unclassified but...
  • ISO/IEC 9594-8, Open Systems Interconnection—The Directory: Authentication Framework,...
  • S. Chokhani, W.Ford, Internet X.509 Public Key Infrastructure, Certificate Policy and Certification Practices...
  • C. Adams, S. Farrell, Internet X.509 Public Key Infrastructure Certificate Management Protocols, Request for Comments...
  • R. Housley, W. Ford, W. Polk, D. Solo, Internet X.509 Public Key Infrastructure, Certificate and CRL Profile, IETF PKIX...
  • S. Santesson, W. Polk, P. Barzin, M. Nystroms, IETF PKIX Working Group, Internet X.509 Public Key Infrastructure,...
  • M. Myers, R. Ankney, A. Malpani, G. Galperin, C. Adams, Internet X.509 Public Key Infrastructure, Online Certificate...
  • Cited by (29)

    • PKI-based trust management in inter-domain scenarios

      2010, Computers and Security
      Citation Excerpt :

      This work makes use of the forward direction, as suggested by Zhao (2005). Lots of works have analyzed the performance of revocation mechanisms in PKI infrastructures (Aarnes et al., 2000; Iliadis et al., 2003), but mainly focused on timing for specific mechanisms, like CRLs, delta CRLs, or OCSP rather than the definition of complex certification scenarios and designing of building and validation algorithms. In Zhao (2005) the author proposes a framework able to model PKI protocols and services in network environments.

    • Long-term verifiability of the electronic healthcare records' authenticity

      2007, International Journal of Medical Informatics
    • Implementing regular cash with blind fixed-value electronic coins

      2007, Computer Standards and Interfaces
      Citation Excerpt :

      Such a case should be as unlikely as the compromise of the infrastructure of a central bank issuing regular money. But even if the unlikely happens, an efficient certificate revocation mechanism [22] would reestablish order in a short period with minimal consequences. Controlling also the version of the circulating e-coins is an additional mechanism against forgery.

    • A PKI approach for deploying modern secure distributed e-learning and m-learning environments

      2007, Computers and Education
      Citation Excerpt :

      Finally, we note that ACs can have an extended life, e.g. four months (one semester), or can be temporary (transaction oriented; take part in a different virtual class meeting). One of the advantages of these temporary certificates, having a short life, is that they do not usually need to be revoked and will therefore need not be included in any certificate revocation list (CRL) (Iliadis et al., 2003). If they are issued in respect of a pre-paid service, they certainly not require any revocation at all.

    • Evaluation of certificate validation mechanisms

      2006, Computer Communications
      Citation Excerpt :

      The performance evaluation of the variety of the proposed PKI mechanisms has been recently an area of active research. Iliadis et al. [26,27] described a mechanism-neutral framework for the evaluation of certificate status information. Muñoz et al. [28] presented a performance comparison of OCSP and over-issued CRLs.

    View all citing articles on Scopus

    John Iliadis holds a BSc in Information Systems Engineering from the Department of Informatics, Technological Educational Institute of Athens, Greece. He also holds an MSc in Information Security from the Department of Mathematics, Royal Holloway College, University of London, UK. He is currently pursuing a PhD in Information and Communication Systems Security under the supervision of Prof. S. Gritzalis, at the Department of Information and Communication Systems Engineering, University of the Aegean, Greece. Mr Iliadis is currently working as a Research Associate with the De Facto Joint Research Group on Information and Communication Systems Security, at the University of the Aegean. He has been involved in national and EU funded R&D projects in the areas of Information and Communication Systems Security. These research programmes include: CRL Study (DG Enterprise), COSACC (DG XIII), EUROMED-ETS (DG XIII), and national programmes concerning PKI, Digital Signatures and Risk Analysis. His published scientific work includes more than fifteen (15) journal and international conference papers. The focus of these publications is on Information and Communication Systems Security and Distributed Systems. He is a Member of the Greek Computer Society.

    Stefanos Gritzalis was born in Greece in 1961. He holds a BSc in Physics, an MSc in Electronic Automation, and a PhD in Informatics all from the University of Athens, Greece. Currently he is an Assistant Professor at the Department of Information and Communication Systems Engineering, University of the Aegean, Greece. His professional experience includes senior consulting and researcher positions in a number of private and public institutions. He has been involved in several national and CEC funded R&D projects in the areas of Information and Communication Systems. These research programs include eVOTE (Information Society DG), CRL Study (DG Enterprise), KEYSTONE (DG XIII), COSACC (DG XIII), EUROMEDETS (DG XIII), ERMIS (DG XVI), ISHTAR (DG XIII), PD4/5 (DG XIII), etc. His published scientific work includes three (3) books (in Greek) on Information and Communication Technologies topics, and more than forty (40) journal and national and international conference papers. The focus of these publications is on Information and Communication Systems Security, Applied Cryptography, and Distributed Systems. He has served on program and organising committees of national and international conferences on Informatics and is a reviewer for several scientific journals. He was a Member of the Board (Secretary General, Treasurer) of the Greek Computer Society. He is a member of the ACM and IEEE Computer Society. He is listed in ‘Who's Who in the World’ and in ‘International Who's Who of Information Technology’.

    Diomidis Spinellis is an Assistant Professor at the Department of Management Science and Technology at the Athens University of Economics and Business, Greece. He has contributed software to the BSD Unix distribution, the X-Windows system, and is the author of a number of open-source software packages, libraries, and tools. His research interests include Information Security, Software Engineering, and Ubiquitous Computing. Dr Spinellis is a member of the ACM, the IEEE, the Greek Computer Society, the Technical Chamber of Greece, and a founding member of the Greek Internet User's Society. He is a co-recipient of the Usenix Association 1993 Lifetime Achievement Award.

    Danny De Cock received the Masters degree in Computer Sciences (Licentiaat Informatica) in 1996 from the Katholieke Universiteit Leuven. Immediately after his studies, he started working as a full time researcher at the K.U. Leuven's Department of Electrical Engineering group COSIC, headed by professor Bart Preneel and professor Joos Vandewalle. His research has mainly focused on the following topics: electronic banking systems, internet voting systems, biometric authentication, pseudo-randomness, computer system administration, design and deployment of public-key infrastructures in mobile and e-government applications.

    Bart Preneel received the Electrical Engineering degree and the Doctorate in Applied Sciences in 1987 and 1993, respectively, both from the Katholieke Universiteit Leuven (Belgium). He is a professor at the Electrical Engineering Department of the Katholieke Universiteit Leuven. Together with Prof. J. Vandewalle, he is heading the research group COSIC at the K.U. Leuven, which currently has 25 members. He has held visiting professor positions at the Ruhr-Univ. Bochum (Germany), at the Univ. of Bergen (Norway), at the T.U. Graz (Austria) and at the Univ. of Ghent (Belgium). He has also been a research fellow at the EECS Department of the University of California at Berkeley. His main research interests are cryptology and information security. He has authored and co-authored more than 100 articles in international journals and conference proceedings and is editor of seven books. He is Vice President of the International Association of Cryptologic Research (www.iacr.org), Chairman of the Leuven Security Excellence Consortium (www.lsec.be) and project manager of the European IST projects NESSIE and STORK. He is a member of the Editorial Board of the Journal of Cryptology and of the ACM Transactions on Information Security.

    Dimitris Gritzalis is an Assistant Professor of Information and System Security, with the Dept. of Informatics of the Athens University of Economics and Business, and an Associate Commissioner of the Greek Data Protection Commission. He holds a BSc (Mathematics, Univ. of Patras), an MSc (Computer Science, City Univ. of New York), and a PhD (Information Systems Security, Univ. of the Aegean). Dr Gritzalis has participated in many R&D projects on information and system security and privacy, and is the author of four books and more than fifty refereed papers. He is the national representative of Greece to IFIP TC11 (Security and Protection in Information Processing Systems), and a former President of the Greek Computer Society.

    View full text