Approaches to quality management and accreditation in a genetic testing laboratory

Abstract

Medical laboratories, and specifically genetic testing laboratories, provide vital medical services to different clients: clinicians requesting a test, patients from whom the sample was collected, public health and medical-legal instances, referral laboratories and authoritative bodies. All expect results that are accurate and obtained in an efficient and effective manner, within a suitable time frame and at acceptable cost. There are different ways of achieving the end results, but compliance with International Organization for Standardization (ISO) 15189, the international standard for the accreditation of medical laboratories, is becoming progressively accepted as the optimal approach to assuring quality in medical testing. We present recommendations and strategies designed to aid genetic testing laboratories with the implementation of a quality management system, including key aspects such as document control, external quality assessment, internal quality control, internal audit, management review, validation, as well as managing the human side of change. The focus is on pragmatic approaches to attain the levels of quality management and quality assurance required for accreditation according to ISO 15189, within the context of genetic testing. Attention is also given to implementing efficient and effective quality improvement.

INTRODUCTION

Quality requirements are particularly high in genetic testing, as tests are typically performed only once in a patient's lifetime, increasing the potential harm of an error, and the results potentially have major implications not only for the tested individuals but also for their relatives. Many organizations, including Cystic Fibrosis Network, European Molecular Genetics Quality Network and European Science and Technology Observatory, have highlighted the need for improving quality and harmonization in genetic testing services within Europe.^{1, 2, 3, 4} In addition, the Organization for Economic Cooperation and Development (OECD) Guidelines for Quality Assurance in Molecular Genetic Testing aim to improve quality by recommending that ‘Laboratories reporting molecular genetic testing results for clinical care purposes should be accredited or hold an equivalent recognition’.⁵ As a result of these elements, the accreditation of genetic testing laboratories is shifting from being a ‘recommendation’ to becoming a ‘requirement’ in many countries throughout Europe.

To support laboratories working towards accreditation and to improve the understanding of quality assurance (QAu), interactive workshops were developed, within the framework of the EuroGentest Network of Excellence (EUGT NoE, FP6-512148, http://www.eurogentest.org).⁶ A successful format was achieved by integrating expertise in both laboratory and quality management, and in learning and change management. Topics include accreditation, QAu, internal audit, diagnostic validation, internal quality control (IQC), external quality assessment (EQA), management review (MR), software support for quality systems (QSs) and change management. The workshops provide a unique forum for laboratories to share experiences and to learn about developing and improving a quality management system (QMS). Furthermore, the provision of international workshops contributes to the harmonization of QMS and approaches to accreditation throughout Europe.

This report aims to combine the outcomes of the different workshops and is intended to serve as guidance and introduction to QMSs for genetic testing laboratories. Therefore, first insight is given on the setup, participants and methodology of the workshops. Second, the key aspects of a QS are tackled and recommendations formulated, based on the principles of International Organization for Standardization (ISO) 15189, with specific emphasis on practical implementation and real-life examples.

QUALITY MANAGEMENT AND ACCREDITATION FOR GENETIC TESTING SERVICES

Where to find the right and understandable information?

The EuroGentest workshops on accreditation and quality management.

EuroGentest is an EU-funded Network of Excellence that intends to structure, harmonize and improve the overall quality of genetic testing services.⁷ The project consists of five units addressing all aspects of genetic testing: quality management, information databases, public health, new technologies and education. Among the key outcomes of the past 5 years are the development of a programme of interactive workshops on quality management for people working in genetic testing laboratories; the expansion of EQA schemes; a QAu database of validated data on EQA participation and accreditation, available in the Orphanet database; and a series of information leaflets in different languages for patients and families about genetics and genetic testing.

Participation in the EuroGentest workshops on accreditation and quality management is open to laboratory directors, scientists, secretaries, technicians and quality managers, from cytogenetic, biochemical and molecular genetic testing laboratories across Europe. However, the content and topics of the workshops are mostly applicable to all types of medical laboratories. It was decided from the beginning to bring together laboratories that were already accredited, working towards accreditation, and in the early stages of developing a QMS. To ensure active involvement, the number of participants is kept to a maximum of 30 people, with lower numbers for the most interactive workshops on internal audit and the human side of change. The workshop language is generally English, although regional workshops have also been provided in French and Dutch.

Six different types of workshops were developed over the years. The first type had broad aims and addressed implementing and living with QSs, comparing the different standards for accreditation in Europe. This workshop revealed a need for and an interest in information technology (IT) support for QMSs in medical laboratories, which logically became the topic of the second workshop. Another crucial, but potentially difficult topic, internal auditing was the third theme. A fourth series was organized on ‘change management’, in which a specialist in the human and behavioural sides of change processes had the central role. Finally, two separate types of workshops were organized on more advanced elements of a QS, such as IQC, EQA, quality indicators, MR and diagnostic validation.

These workshops are successful and unique because of specific teaching concepts and methodology, which include presentations, casework in small groups, open discussion and conclusions. Structured questionnaires, audiovisual material, role-play and hands-on work with software increase interactivity and improve the outcome. Feedback sheets ensure continual improvement of the workshops’ format and a response to needs and suggestions of participants. They revealed a number of recurrent remarks and outcomes:

• The workshops offer a unique platform for accredited and non-accredited laboratories to meet their peers and to learn, share and discuss common experiences.

• Participants are encouraged to discover that they generally have many elements of a QMS in place in their laboratories, even though these may not be formalized: ‘we are closer than we thought’.

• Attendees are further encouraged to learn that they are confronted by common problems and questions in working with their QS.

• Sharing problems and solutions in groups is efficient and encourages harmonization between laboratories.

• Structured training in the more technical aspects of quality management such as validation, auditing and change processes is perceived as being more time- and cost-effective than self-training.

Who participates in the workshops?

From 2005 until 2009, 19 workshops have been organized. They have been attended by 326 different people, from 168 institutes in 125 cities in 41 countries all over the world (Europe: 32, Asia: 3, North America: 2, Africa: 2, South America: 1, Australia: 1); 80 participants (25%) have attended more than one workshop (Table 1). The workshop organizers are Belgium (Leuven) and Switzerland (Geneva), which led to the organization of a joint Belgian–Swiss workshop on accreditation. A further workshop was organized in Dutch, for participants from The Netherlands and Belgium, and one in French, at the request of the French Agence de Biomédecine, with participants from France and Switzerland. This explains the relatively large number of Belgian, French, Swiss and Dutch attendees. Countries with laboratories involved in the EuroGentest NoE tend to have a higher participation rate, such as the United Kingdom, Germany, Ireland and the Czech Republic (marked in italics in Table 1). Furthermore, countries hosting the workshops have more participants, for example Spain, Italy, Greece and Austria (marked in bold in Table 1), revealing the value of holding workshops in different regions of Europe. At the start of the project, most of the attendants came from laboratories that were well known internationally; with time, the diversity of participants and countries increased considerably. Since 2007, two workshops on different themes have been organized in parallel immediately before the European Society for Human Genetics (ESHG) Congress (Nice, Barcelona and Vienna), facilitating accessibility for interested people.

Table 1 The number of participants per country, city, institute and per type of workshop from 2005 until 2009

Since 2006, the registration form includes non-compulsory questions on the professional role of participants, the accreditation status of the laboratory and the genetic subspecialization (molecular genetics, biochemical genetics or cytogenetics). These data were used to develop cases and exercises specifically adapted to the background and level of experience of the attendees. The positions of participants (n=305) varied from laboratory and department directors (32%) to senior scientists (26%), quality managers (18%), technicians (13%) and medical doctors (11%). Participants (n=182) were divided between accredited laboratories (31%), laboratories preparing for accreditation (28%) and laboratories at a very early stage of the accreditation process (41%). When considering the different fields from which the participants of the workshops come from (n=176), it is clear that most of them work in molecular genetic testing laboratories (65%). Fewer participants came from cytogenetics (10%), biochemical genetics (1%) or a combination of the different genetic activities (15%). A part of the participants were from other medical laboratory disciplines such as clinical chemistry, microbiology, haematology and pathology (9%). In our experience, a mix of different specializations is a positive aspect, providing richer opportunities to learn from experiences from other areas.

Accreditation: how, what and why?

The ISO has developed an international standard for the accreditation of testing laboratories in general (ISO 17025) and one specifically for medical laboratories (ISO 15189).^{8, 9, 10, 11} ISO 15189 emphasizes the quality of contributions to patient care, as well as that of laboratory and management procedures, and is, therefore, the preferred standard for genetic testing laboratories.^{12, 13, 14, 15} In contrast, ISO 17025 is written in more general terms and is applicable to a wide range of testing environments. However, as the ISO 15189 standard is designed for a wide range of medical laboratories, a certain effort of reflexion and interpretation may be necessary to apply it in the context of genetic testing.

Personal copies of ISO standards can be easily bought through the website of ISO (http://www.iso.org) or through the National Standards Organization (eg, Bureau voor Normalisatie or NBN in Belgium, Deutsches Institut für Normung or DIN in Germany, Association française de Normalisation or AFNOR in France). An overview of these bodies can be found on the website of the European Committee for Standardization: http://www.cen.eu/cenorm/members/national+members/. In some countries, widely used local standards were developed, notably, Coördinatie Commissie ter bevordering van de Kwaliteitsbeheersing van het Laboratoriumonderzoek Praktijkrichtlijn in The Netherlands and Clinical Pathology Accreditation standards in the United Kingdom.^{16, 17, 18, 19} The use of these standards is disappearing, with both the Netherlands and the United Kingdom switching to ISO 15189 from 2010. An overview of the different quality standards for genetic testing laboratories in European countries is given in Figure 1.

Figure 1

Accreditation standards for genetic testing laboratories.

Apart from accreditation standards, OECD has published specific Guidelines for Quality Assurance in Molecular Genetic Testing, which might be considered as a sector-specific document to be used in combination with the existing accreditation standards.⁵ The contents of the guidelines are not formal requirements, but can be a useful complement to laboratories for improvement and harmonization. The minimum common requirements described address general principles and best practices, QAu, EQA, reporting of results and training for laboratory personnel. Laboratories can download these guidelines freely from the OECD website in English, French or Spanish: http://www.oecd.org/dataoecd/43/6/38839788.pdf.

There is still a persistent misunderstanding about the difference between accreditation and certification. Certification is defined by ISO as the ‘Procedure by which a third party gives written assurance that a product, process or service conforms to specific requirements’.^{20, 21} Its requirements address only the QMS, including procedures, a quality manual, documenting control, defining non-conformities (NCs), corrective and preventive actions (CAPA), performing internal audits and enhancing customer satisfaction; it does not necessarily include requirements of technical or analytical competence. Accreditation, in contrast, is defined by ISO as the ‘Procedure by which an authoritative body gives formal recognition that a body or person is competent to carry out specific tasks’. Although accreditation also considers the QMS, it has additional formal requirements of technical competence, including initial and continuous training of personnel, validation of methods and instruments, and internal and external quality control. As a result, certification (typically according to the ISO 9001 standard) should not be interpreted to mean that a laboratory has demonstrated the technical competence to produce valid data and results.²² On the other hand, ISO 15189 and ISO 17025 are accreditation standards, assuring technical competence of a laboratory.

Apart from the difference in the objectives of certification and accreditation, there is also a difference in the body that carries out the assessment and delivers the certification or accreditation certificate. Laboratories applying for ISO 9001 certification will be audited by a certification body, a third party, which is accredited by an accreditation body. Each country has multiple certification bodies. A few examples are AENOR, Bureau Veritas, CERMET, IQNet, TüV. In contrast, there is only one recognized national accreditation body (NAB) in each country that assesses laboratories against internationally agreed standards (Regulation (EC) No 765/2008). In Belgium, this is BELAC (Belgian Accreditation Body), in Switzerland, it is SAS (Swiss Accreditation Service). Other recognized national accreditation bodies can be found in Figure 2. The confusion about the difference between accreditation and certification also arises from the fact that, in the United States, the word accreditation is mainly used for certification, according to the definition of ISO.

Figure 2

Recognized national accreditation bodies in Europe.

The European cooperation for Accreditation (EA, http://www.european-accreditation.org) is a non-profit association and is the European network of the recognized NABs located in the European geographical area. A similar organization exists in other regions: InterAmerican Accreditation Co-operation, Asia-Pacific Laboratory Accreditation Co-operation and Southern African Development Community in Accreditation. One of its purposes is to develop and promote accreditation criteria and guidelines that will ensure harmonized performance of national accreditation bodies throughout the European economic area. Most of the EA accreditation body members signed a multilateral mutual recognition arrangement (MRA) to recognize the equivalence, reliability and, therefore, acceptance of accreditations and certifications across Europe. A certificate or inspection report issued by an accredited body in one country is recognized as equivalent to a certificate or inspection report issued by an accredited body in any of the countries that are signatories to the EA MRA. International Laboratory Accreditation Cooperation is the international umbrella organization that covers all national and regional accreditation organizations (Figure 3).

Figure 3

National, regional and international accreditation organizations.

Laboratories that want to proceed towards accreditation should decide on the accreditation standard and contact their NAB. The websites of these authorities provide useful information on the accreditation process, formal documents, contact details of experts and other accredited laboratories, specific for their own country. Furthermore, the laboratory needs to define whether all activities or only a selection will be accredited. This is called the scope of accreditation. Most NABs have formal application documents to be filled out and require preliminary documentation on the laboratory itself and the existing QMS. The accreditation body will then appoint a lead assessor and a technical assessor(s) who are experts in the field. This team of assessors will conduct the audit (external audit) in the laboratory and they will formally report the assessment findings to the NAB. In case of minor NCs, the assessor will later check whether the corrective actions were adequate, through submitted documentation. In case of major NCs, a new on-site assessment may be required to assess the effectiveness of the corrective actions. Accreditation may be granted for a period of up to 3 or 5 years, but the NABs will perform surveillance visits to ensure that the laboratory continues to meet requirements, which is typically every 1–2 years. The procedure for renewal of accreditation is usually very similar to the one for new applicants, but the laboratory might consider at that moment to widen the scope of the accreditation by adding new tests.

A good QMS in the laboratory has advantages such as increased transparency, traceability, uniformity, work satisfaction and better focus on critical points. On the contrary, it will require extra time on aspects such as document control and there is a danger of losing critical attitude and curbing innovation and changes. Therefore, a formal accreditation and the linked periodical audits are a stimulant for keeping the QS alive. Without accreditation, there is a danger of giving less attention to quality improvement. In addition, accreditation is a good way to demonstrate and attest competence and a worldwide tool to recognize laboratories. Finally, all parties (patients, families, the laboratory and clinicians) are benefited through better processes and quality of results.

KEY ELEMENTS OF A QMS ACCORDING TO ISO 15189

The initial step of introducing good quality management in the laboratory is the identification of the key elements of a QS. These elements need to be integrated with the existing processes and organization, through documentation of standard operating procedures (SOPs) and definition of objectives and policies in a quality manual. All essential elements of a QS are covered by the ISO 15189 accreditation standard in two distinct chapters: management requirements and technical requirements. Management aspects include document control, identification of NCs, implementing CAPA and action plans to continuously improve, performance of internal audit and MR, resolution of complaints, evaluation of external services, suppliers, contracts and referral laboratories. Technical elements enclose personnel and training, accommodation, equipment, validation and assuring quality of examination procedures by IQC, EQA, maintenance and calibration. The most important parameters of a QS (Figure 4) and their practical implementation in the laboratory are discussed in the subsequent paragraphs.

Figure 4

Elements of a quality system.

Organization and quality management

The genetic testing laboratory, or the organization of which the laboratory is a part, must be legally identifiable. The laboratory must define ‘who is legally responsible for the work of the laboratory in the event of legal action being taken’. Confidentiality is also critical in genetic testing services, and policies on this matter should be written down. Further, the laboratory should meet the needs of patients, clinical personnel and ISO requirements. All responsibilities and authorities of personnel should be defined and, as a result, conflicts of interest should be identified. These could include financial, commercial, research or other influences, external or internal. The design, implementation, maintenance and improvement of the QMS are responsibilities of laboratory management. Therefore, they must be provided with sufficient time and resources to carry out their duties, which may lead to the reorganization of certain positions. The quality manager is the person who takes overall responsibility for the QMS and reports directly to the level of laboratory management. However, it is important that the quality manager does not work in isolation, but that he or she interacts with and gets support from other people working in the laboratory with respect to the QMS. Typically, laboratories set up a quality team to divide work and get everybody involved. Finally, personnel should receive adequate training and supervision, depending on their experience and level of responsibility.

The whole QMS must be described in a quality manual. This quality manual is the formal, top-level document that establishes the quality policies and objectives of an organization. It is used internally, to outline the format of documents used in the QMS, as well as the roles and responsibilities of the personnel responsible for management of the QMS or technical procedures, and also externally to give a clear and precise view of the organization and activities of the laboratory to potential clients and inspectors. ISO 15189 provides clear instructions on the content of the quality manual and quality policy.

Document control and IT support

When implementing a QMS, several documents will be generated: SOPs, worksheets, log books, validation and training files, and a quality manual. These documents must be created, implemented, communicated and understood by all relevant personnel. Document control is the mechanism by which QMS documents are created, approved, distributed, reviewed, revised and archived. In other words, all documents are approved by authorized personnel before use and a distribution list is available that identifies the current valid versions. Documents are periodically reviewed or revised when necessary, and then reapproved. Invalid or obsolete documents are removed and the archived documents are recognizable to ensure that staff only uses the latest authorized versions. Finally, procedures must be defined on whether and how amendments, minor and major changes, including handwritten changes, can be made. Figure 5 is a schematic overview of the different phases in document control.²³

Figure 5

Different phases in document control.

All QMS documents must be uniquely identifiable and completed with a version number, a date of issue and the name or signature of the authorizing person. Each page should have the total number of pages mentioned in it to avoid losing pages without the user being aware. Documents should be written to be used, not for the auditors. Before any document is developed, ask what the purpose of the document is, who is going to use it and what would be the consequence if it were not created. With respect to procedures, the two golden rules are the following: ‘be precise, but not too precise’; and ‘write what you do and do what you write’. For example, if a test requires an incubation of 10–20 min, do not write 15 min. Technical instructions in particular should be designed to simplify future modifications: the use of general terms (eg, agarose gel, PCR machine) is recommended, with specific details defined in separate documents. Documents must not only be prepared by the quality manager but also by the people who will use them and who know the procedures. Should changes be made to procedures, a system should exist to inform all relevant people. This could be carried out in a regular meeting (with minutes, read by absentees), on an intranet or on a notice board.

The ISO standard only requires that the laboratory should have a document control system, but it does not define how the document control should be implemented. It can be entirely electronic, entirely on paper or a mix of both. Many of the existing electronic systems go beyond simple document control, and cover training, follow-up of CAPA, maintenance and calibration of equipment, auditing, and reagent and batch control. Such software packages aim to support the whole QS and are distinct from laboratory information management systems that include tracking and monitoring of laboratory samples and data only. A thorough Internet search (using combinations of the following key words: compliance, quality, management, software, ISO, laboratory, audit, document control) was performed to identify QMS software on the market. The list of software packages (Figure 6) is intended as a basis for selecting software that fits a particular laboratory's workflow; the authors have made the list as comprehensive as possible, while avoiding any judgement or preference for particular solutions. The motivation to change to a software-based QMS could include saving time because of reduced paperwork, better traceability, simplified follow-up of audits and corrective actions, and programmable notifications. Furthermore, a central data store, accessible from everywhere, makes work more efficient and well organized. On the other hand, besides the initial cost, it will take significant time and effort to train staff and implement the system, and to win everybody over to its advantages. A necessary starting point when selecting a system is compiling a list of requirements: define what is essential for the laboratory on the basis of what is already in place, the size of the laboratory, IT experience, long-term expectations and existing data. Figure 7 presents a list of criteria identified at the workshop. When currently available software systems were compared with these criteria, it was evident that all packages had interesting aspects, but that none yet really fulfilled all the ‘ideal’ criteria for quality management in genetics laboratories. Some barriers to implementing an electronic system exist, including convincing the management, extra workload and initial cost. Suggestions to overcome these problems might include using the principles of managing change processes (see further). Start by involving all relevant people from the beginning, including technicians, secretaries and IT personnel, as well as the quality manager and laboratory director. Listen to their ideas and distribute responsibilities to increase motivation. When people have the impression that they contribute to the decision making, they will be more open to implement and put effort into the new system. In addition, take the time to educate yourself through colleagues, the Internet and demonstrations. Visiting a laboratory that uses a system could help convince personnel of the value of changing. To reduce the excess workload, invest in training and work in a phase-to-phase approach, implementing and optimizing one module before you start implementing others; to reduce or spread the costs, consider combining or phasing the purchase of modules.

Figure 6

QMS software systems identified by Internet searching (March 2010).

Figure 7

Criteria that should be taken into account when selecting QMS software.

IQC and EQA

IQC and EQA are two distinct but complementary components to assure continuously the quality of the examination procedures in the laboratory.

IQC is an internal verification that the test yields consistent results day after day; in other words, the identification measure of precision, but not necessarily of accuracy. ISO 15189 requires that ‘the laboratory shall design IQC systems that verify the attainment of the intended quality of results’, but does not provide details of specific controls. The World Health Organization (WHO) defines IQC as ‘the set of procedures undertaken by the staff of a laboratory for continuously assessing laboratory work and the emergent results, in order to decide whether they are reliable enough to be released’.²⁴ Some possible approaches to IQC are discussed here. On one hand, the laboratory should avoid mistakes (ISO 15189, 5.6.1) in the process of handling samples, requests, examinations, reports and so on; on the other, the laboratory should determine uncertainty (ISO 15189, 5.6.2) where relevant and possible. For each test, the laboratory should identify and define the potential errors, risks and challenges (typically, during the validation phase); subsequently, specific IQC should be defined to assure each risk and potential problem. The defined controls should be documented in SOPs.^{25, 26} Errors can also be traced and reduced by keeping records of batch numbers of all laboratory solutions to improve traceability and troubleshooting. Labelling each tube with a unique code and double checking before, during and after transferring samples is also a preventive IQC measure. The standard states ‘There shall be effective separation between adjacent laboratory sections in which there are incompatible activities. Measures shall be taken to prevent cross-contamination’ (ISO 15189 5.2.6); consequently, pre- and post-PCR areas should be separated and filter tips used. Laboratories should always include negative, positive normal and specific mutation controls when appropriate and possible; the difficulty of obtaining a wide selection of mutation controls is a notable problem in rare disease testing.

EQA is defined by the WHO as ‘a system of objectively checking laboratory results by means of an external agency. The checking is necessarily retrospective, and the comparison of a given laboratory's performance on a certain day with that of other laboratories cannot be notified to the laboratory until some later. The main objective of EQA is not to bring about day-to-day consistency, but to establish interlaboratory compatibility’.²⁴ EQA is an important complement to IQC in which a large number of laboratories are provided with the same material and required to return results to a coordinating centre. The results are compared to determine the accuracy of the individual laboratory. In addition, EQA provides continuous education and training for laboratories as well.^{27, 28, 29, 30, 31} Accredited laboratories are required to ‘participate in interlaboratory comparisons such as those organized by EQA schemes’ (ISO 15189 5.6.4). EQA should, as far as possible, cover the entire range of tests, and the entire examination process, from sample reception, preparation and analysis to interpretation and reporting (ISO 15189, 5.6.4). For many rare diseases, no EQA scheme exists. ISO 15189 (5.6.5) states ‘whenever a formal interlaboratory comparison programme is not available, the laboratory shall develop a mechanism for determining the acceptability of procedures not otherwise evaluated’; examples include reference materials or interlaboratory exchange. Interlaboratory comparisons should cover the scope of services offered and there should be a formal mechanism of review and comparison of results. Figure 8 gives an overview of the main EQA providers within Europe in alphabetical order. It is essential to follow-up EQA results and discuss the reports in regular laboratory meetings; not only the negative remarks but positive results as well. Appropriate actions (preventive and corrective) should be implemented directly and documented. If fitting, the laboratory can report back to the EQA provider.

Figure 8

Overview of the main genetics EQA scheme providers in Europe.

Used together, IQC and EQA provide a method of ensuring accuracy and consistency of results and are vital tools in the QAu of the laboratory. The relationship between precision and accuracy may be illustrated by the familiar example of shooting arrows at a target (Figure 9). IQC and EQA are two complementary components of a QS, both evaluating performance in the laboratory: IQC is a daily check that results are reliable and consistent, whereas EQA is a retrospective comparison with other laboratories. In addition, EQA has an important educational purpose. EQA and IQC provide useful information for the MR of the laboratory (see further).

Figure 9

Accuracy and precision.

Diagnostic validation

It is a formal requirement of accreditation standards, including ISO 17025 and ISO 15189, that (genetic) tests and instruments must be validated before diagnostic use to ensure reliable results for patients, clinicians or referring laboratories and their quality must be maintained throughout use. In other words, the laboratory must demonstrate that their tests are fit for the intended use before application to patient samples. Figure 10 gives a summary of what ISO 15189 states with regard to validation.

Figure 10

What does ISO 15189:2007 say about validation?

The technical committee ISO/TC 212 (clinical laboratory testing and in vitro diagnostic test systems) is currently working on the periodic revision of ISO 15189. The revised version, foreseen in 2012, will elaborate more on validation requirements (personal communication WG1 ISO/TC 212). Below are some of the proposals to be included in this new version:

• ‘The laboratory shall validate examination procedures from non-standard methods, laboratory designed or developed methods, standard methods used outside their intended scope and modified validated methods.’

• ‘When examination procedures have been validated by the method developer (ie, the manufacturer or author of a published procedure), the laboratory shall obtain information from the method developer to confirm that the performance characteristics of the method are appropriate for its intended use. When changes are made to a validated examination procedure, the influence of such changes shall be documented and, if appropriate, a new validation shall be carried out.’

• ‘Examination procedures from method developers that are used without modification shall be subject to verification before being introduced into routine use. The verification shall confirm, through provision of objective evidence (performance characteristics), that the performance claims for the examination method have been met. The performance claims for the examination method confirmed during the verification process shall be those relevant to the intended use of the examination results.’

Verification and validation are two slightly different procedures (Figure 11). By default, all new laboratory procedures must be validated before application to clinical testing. In addition, a validation is necessary when major technical modifications to existing methods are carried out or when the performance of existing methods has been shown to be unsatisfactory. In vitro diagnostic devices with a CE mark or FDA approval or validated procedures implanted from an accredited expert laboratory require verification in most of the cases, on condition that the original protocols are used exactly as described; any change of conditions (eg, reduction of reagent volumes) would invalidate the performance criteria and lead to a need for validation.³²

Figure 11

Validation and verification.

There are several measurable parameters that should be taken into account during validation or verification. The estimation of accuracy is a key parameter. Accuracy consists of both precision and trueness for quantitative (eg, analysis of mosaicism) and semiquantitative (eg, sizing triple repeat, exon deletion) tests. Precision or ‘closeness of agreement between results of replicate measurements’ includes the following:

• Repeatability: within-run variation (same sample, same conditions).

• Intermediate precision: between-run variation within a single laboratory (different samples, operator, PCR machine).

• Reproducibility: between-run variation in different laboratories (different samples, operator, PCR machine).

• Robustness: variation when confronted with relevant challenges (eg, sample type, extraction methods, DNA concentrations, environmental conditions and so on).

Trueness is the ‘closeness of agreement with a reference value’. Appropriate reference materials are, therefore, essential and could include positive and negative/normal controls, certified reference materials, EQA materials, synthetic samples or material characterized by another technique.

The components of accuracy for qualitative tests (eg, mutation scanning, genotyping) are sensitivity and specificity. Sensitivity is a measure of how well the test detects positive results, whereas specificity describes how well negatives are detected.

Many laboratories already validate their tests in a satisfactory manner, but without necessarily documenting the experiments and conclusions. Thorough documentation during a validation process is essential, especially in the context of accreditation audits. Use pragmatic approaches, reconciling the formal requirements of accreditation standards while respecting the aim that ‘validation must be practical’, such as

• the design of IQC based on validation results;

• making full use of data that laboratories are already collecting, for example from IQC or EQA, for continuous validation.

There are no detailed practical guides for validation of medical genetic tests, for example stating how many samples or repeats are necessary or sufficient; moreover, the accreditation standards have no specific details about how to fulfil their requirements. The laboratory must decide on this, on the basis of their experience and performance requirements; it is the duty of the laboratory to provide evidence that the tests provided are reliable, and that the performance claims are correct. The Clinical and Laboratory Standards Institute, as well as Dr Westgard in his book on basic method validation, has published guidelines with the aim to support laboratories in their validation processes.^{33, 34, 35} Furthermore, within EuroGentest, a standardized framework for the validation and verification of clinical molecular genetic tests was developed to aid genetic testing laboratories.³²

In the end, validation is never finished. The implementation of quality indicators (see further) for systematically monitoring and evaluating the laboratory's contribution to patient care is a good way to continuously validate diagnostic tests, apart from IQC, EQA and other data.

Internal audit and follow-up of NCs

Internal audits are an essential part of a QS and should be considered as a peer review process to improve the system conducted by the laboratory itself and not as an inspection. ISO 15189 demands accredited laboratories to audit the main elements of their QS once every 12 months. In practice, this means that laboratories regularly perform small internal audits, covering a different aspect of the QMS every time.

There are two main types of internal audit: horizontal and vertical audit. A horizontal audit is a detailed check of a particular aspect of the documentation and implementation of the QMS or examination processes, for example examining a number of reports to determine whether appropriate interpretative comments and/or follow-up of abnormal results have been provided. A vertical audit is a detailed check that all elements, associated with a chosen examination (test), are implemented. For example, select a single request form and its associated sample (input) and follow it through every element of the process until the report (output) is produced. Independently from the type of audit, the auditor should combine interviews, document checking, observations and cross-checks in an effective and efficient way. It is crucial that the auditor's understanding of ISO requirements and quality procedures be current and accurate. In addition, he or she should have a basic knowledge of methods, instruments, calibration and be able to judge the competence of the laboratory.

An internal audit consists of three phases: preparation, execution, and reporting and follow-up. It is impossible to perform a good audit without thorough preparation; the audit plan needs to be flexible to permit changes, but should cover the objectives and scope of the audit. Furthermore, the individuals with responsibilities during the audit should be identified and informed if a date and time is known, as well as any people who will be directly involved in the audit. The preparation should include the identification of all reference documents (SOPs, standards and so on). Working documents, such as checklists and observation forms, are helpful to facilitate the auditor's investigations and to document and report the results.

The behaviour of an auditor and his way of communicating can be crucial for a successful audit. When the auditor begins, he should make the auditee feel at ease by being polite and not too formal. He should explain the planning and make clear that it is the system and not the person that is being audited; no ‘blame’ will be attached to anyone. It is important to find the balance between confrontation and empathy. Although it is essential not to hide or minimize when something is wrong, the auditor should stay calm and must not raise his voice. An auditor must stay objective and independent, and adapt to the behaviour of the auditee. Finally, the auditor should not interrupt the auditee, and should maintain regular eye contact, which additionally helps in monitoring the body language. Open questions should be preferred, avoiding the possibility of ‘yes/no’ answers. If necessary, subsequent questions may be asked to develop deeper answers, but ‘why’ questions should be avoided as they often trigger defensive reactions. It is preferable to start with ‘how’ questions, based on what can be seen: ‘Could you describe how you know the incubator is at 37°C?’. The use of words such as ‘maybe’, ‘a little bit’, ‘could’ may have miscellaneous effects; they can make the message less direct or threatening, but they may also give the impression that the auditor is not sure about himself. The auditor should mention when something is not conforming to the procedure, so that the auditee already understands during the audit that not everything is perfect. At the end of an internal audit, the auditor should focus on solutions and improvement and should include an indication of where improvement is necessary, as well as positive feedback. It is not appropriate to discuss a detailed action plan immediately after the audit.

Taking accurate notes during the audit is essential. On the basis of these, the auditor will make a clear and structured audit report within a defined time limit. This report should include the major and minor NCs, as well as mention positive points. Following the report, an action plan must be developed, identifying the responsible person and a deadline for each action. General elements, including the auditor's name, date and the standard used should be present, in addition to specific details such as the scope of the audit, the SOP numbers, equipment and so on. An example of a report is shown in Figure 12.

Figure 12

Example of an audit report.

A laboratory is committed to constantly evaluating its activities and to maintaining and improving its quality. Performing internal audits is one way of evaluating and improving a QS and detecting NCs. Other approaches include participating in EQA schemes, organizing customer satisfaction surveys or undergoing an external audit conducted by an external independent organization such as an NAB. When the laboratory comes across minor or major NCs, a corrective action and/or a preventive action should be implemented to eliminate the NC in the future. During the next internal audit, these elements should be checked again, to evaluate whether the action plan was effective. For example, a laboratory detects that one of its reagents was frozen because it was stored too close to the back of the fridge. The corrective action could be that the tests for which that particular reagent was used will be repeated; the preventive action could be that the reagents will be stored henceforth in boxes to avoid freezing in the future. After a while, the laboratory should evaluate, usually during an internal audit, if the corrective action was completed and if the preventive action was effective. Each action plan needs to be realized within a certain time frame and all responsibilities need to be defined. Therefore, it is helpful to develop user-friendly NC forms, so that all people working in the laboratory can report NCs and can indicate a responsible person and appropriate time frame. Involvement of the whole group is important to achieve successful improvement.

Reporting NCs, developing action plans and evaluating later is also known as the Plan-Do-Check-Act (PDCA) circle, or ‘Deming wheel’.³⁶ It is a very simple concept that helps to coordinate your quality improvement efforts: just as a circle has no end, the PDCA cycle repeatedly executes in pursuit of continual improvement. It emphasizes and demonstrates that improvement programmes must start with careful planning, must result in effective action and must move on again to careful planning in a continuous cycle (Figure 13).

Figure 13

Deming wheel for continual improvement.

Management review

According to the accreditation standards, ‘laboratory management shall review the QMS to ensure continuing suitability and effectiveness in support of patient care and to introduce any necessary changes or improvements. The results of the review shall be incorporated into a plan that includes goals, objectives and action plans’ (ISO 15189, 4.15.1).³⁷ During an MR, the emphasis is on developing an overview of the many elements of a QMS, such as the results of IQC, EQA, internal audits and external audits, previous MRs, quality indicators, turnaround times, status of preventive and corrective actions, NCs, complaints, feedback and evaluation of suppliers. Questions to be asked (and answered) include the following: ‘Do we still support our objectives and quality vision?’ and ‘Are we still satisfied about the way certain elements of the QMS are implemented?’. The management should dare to change directions or lower criteria when initial objectives are too ambitious, but they should still aim for continuous improvement. The standard states that an MR typically occurs once every 12 months. However, in some laboratories, it may be more efficient to perform segmental MRs, ensuring that by the end of the year all elements are tackled. As the information discussed during the MR is extensive, a good preparation is essential. The whole process is driven by the management, but they can delegate tasks to the quality manager to collect all the data, decide what needs to be discussed and how the data will be presented. In other words, he or she defines the agenda, invites the participants of the meeting (management or staff and usually the quality team members) and contacts the people in the laboratory responsible for collecting the necessary data summarized above, which form the input of the MR. Usually, the quality manager or laboratory director chairs the MR, but is not the only person who talks. One way to prepare for the meeting efficiently is to have a meeting in advance with all the people who will collect data. It might be useful to have ‘guidelines’ or a predefined template to standardize the way the results are presented by different groups. Thereafter, the collected data are provided to the management before the meeting in a structured manner. This leads to an MR meeting focused on the right topics, finished within a reasonable time. Difficult or technical subtopics could be addressed in a separate meeting with fewer participants. The outcome of the MR should include an action plan (‘what, who and by when?’) and the objectives of the next year, leading to a continuous improvement for the patients. In addition, resource needs or important decisions could be enclosed. All this information should be written in a report that is absolutely necessary, as it is a way of registering your quality. The report is dated and signed or approved afterwards. The quality manager is responsible for the follow-up of the action plan. All laboratory staff is informed of the results of the review. Quality indicators are important input elements used in an MR.

The standard mentions that ‘Laboratory management shall implement quality indicators for systematically monitoring and evaluating the laboratory's contribution to patient care. When this program identifies opportunities for improvement, laboratory management shall address them regardless of where they occur (ISO 15189, 4.12.4)’. Figure 14 shows a list of possible quality indicators that can provide a starting point for each laboratory.

Figure 14

Examples of quality indicators in the laboratory.

To facilitate a successful MR, the techniques of motivation and change management (see further) could be applied. Use a logical structure, in an understandable and meaningful way, to present the information. Start with some positive elements, show solutions, but keep focus and respect the timing. Consider the audience and adjust the content of the presentations to suit them. Listen actively and verify that everybody understands and agrees. Finally, an MR is a process in which the whole laboratory is involved, not only management and the quality manager. All people working in the laboratory can help to collect data where appropriate, and should be informed about the outcome and the action plan.

Human side of change processes

One of the biggest challenges in implementing a QS in the laboratory lies in overcoming the natural reluctance of team members and convincing them of the value of QAu to justify the effort that will be required. Implementing a specific change in the laboratory is a linear process. For example, the head of the laboratory decides, together with his staff members, to implement a QS. The different steps (writing SOPs, tracking NCs and so on) and when they should be implemented is definable. This linear process is usually organized with the help of techniques such as project management and is restricted in time. However, the behavioural change – the reaction of technicians in the laboratory – cannot be planned in time, as it is a cyclic process. The technicians will go through a learning process, during which they receive new information, experience certain emotions, try out new things and adapt to and integrate their knowledge. This process differs between individuals and cannot be managed in the same linear manner. Tension will emerge between those two processes and this will cause resistance during a change process (Figure 15). This resistance cannot be avoided, but when managed well, can increase the effectiveness and efficiency of the change you would like to implement. The cyclic process mentioned above encloses different phases people may go through when confronted with change, as described by Kübler–Ross.³⁸ Initially, people tend to deny the impending change and continue with their work. They progressively become aware that the change is significant and may begin to doubt and become angry. Consequently, people will negotiate and try to maintain the status quo. When it is accepted that change is inevitable, they may become depressed. After a period of inner struggle, people see new opportunities, start exploring them and develop alternatives to go about with the changes. In the last phase, they accept the situation and plans can be made to continue. Each individual will go through these phases, but at different speeds, resulting in the need for flexibility, in particular from the management. Figure 16 gives an overview of the different phases, as well as how you should react when implementing change. First, patiently repeat the message and the facts to people in the ‘shock and denial phase’. Second, give them the opportunity to blow off some steam and listen. If they start to negotiate, be firm, which can be followed by encouragement. Once people accept what is going to happen, do not minimize their efforts and be positive.

Figure 15

Change management – two interfering processes.

Figure 16

Phases people go through in times of change.

A useful tool to help navigate more efficiently through the whole change process is force field analysis, which helps to traverse the phases of change in a participative way with your team or department (group problem-solving tool). The underlying concept is that every situation involves a balance between conflicting forces, driving forces and restraining forces. Driving forces (or enablers) are actions, skills, equipment, procedures, people and other factors that help to move towards the desired objectives, whereas restraining forces (disablers) inhibit you from reaching the goals. Force field analysis is usefully carried out as a group exercise, involving the identification of different forces together. The forces can be prioritized and appropriate actions should be defined to minimize disablers and stimulate enablers so that change becomes possible. In Figure 17, a force field analysis is applied to implementing an IT support system in the laboratory. A second tool to help navigate more efficiently through the whole change process is stakeholder analysis. Stakeholders are individuals who have a direct interest in the change process (positive or negative); they have the information, resources and expertise important for the success of the change process or they have the authority to influence the success. The inclusion of all relevant stakeholders in this analysis is essential for the success of the change process. To perform this analysis, first create a list of all stakeholders (stakeholder map), for example laboratory technicians, quality manager, laboratory director, physicians and patients. Split up all stakeholders into clusters, on the basis of influence and impact (clusters 1–4 in Figure 18). This enables you to assign priorities within the stakeholders group (high, low, critical, important for decision making and so on). Finally, strategies should be developed to involve, inform or mobilize the different stakeholders (stakeholder management plan). For example, the strategy for stakeholders who have a low influence but high impact (cluster 3) should be oriented towards empowerment, which encloses involvement by giving limited power to decide.

Figure 17

Force field analysis example: implementing an IT support system.

Figure 18

Stakeholder analysis.

In case of major change processes, it can be valuable to develop a structured communication plan. The stakeholder map then functions as a starting point to identify the people you have to communicate with. The communication plan should include these stakeholders, the message you want to communicate, the actions needed, how and when you will communicate your change process to the stakeholders and, finally, who will be responsible for the actions. In addition, individuals think in different ways; therefore, they should be approached differently.³⁹ Gardner identified seven levers (‘7 RE's’) that aid or thwart the process of mind change. These levers are reason, research, resonance, redescriptions, resources and rewards, real-world events and resistance (Figure 19). Consider these levers to bring about significant changes in perspective and behaviour to reach all collaborators and improve your communication plan.

Figure 19

7 RE's.³⁹

The tension between the cyclic and linear process discussed above results mostly in resistance. This resistance is not problematic and if managed well could have a positive impact or even optimize the change process. When confronted with resistance, you could adopt the following four steps:

1. Listen and reward: ‘That is a good question’, ‘It is indeed important to take that into account’; ‘I understand why you have difficulties with that’.

2. Summarize to check if you understood your collaborator: ‘If I understand well …’, ‘So you think that …’

3. Keep asking questions: ‘Could you give some examples?’, ‘What do you mean exactly?’

4. React, depending on the time you have, the emotion of the collaborator or the difficulty of the question:

• Park or move the issue: ‘Is it ok that I make a note and come back to it later on?’

• Give more explanation: When you think a collaborator does not understand everything, explain it again in a different way.

• Start the discussion: If you want to know how others think about it. If it is obvious you cannot continue before a certain point is discussed.

• Take away the objective/give in: Sometimes a small concession could raise a lot of goodwill of the collaborators.

• Hold on to your point: If unrealistic questions are asked, stay friendly but firm.

CONCLUSION

Implementing a QMS is a process with different phases starting from the decision of the management to implementing a QMS, choosing an appropriate standard, delegating responsibilities and collecting information, through formalizing SOPs, offering training and performing validation and audits. This requires a lot of energy and time for laboratory personnel. However, all the invested time will return in the form of increased quality for the patient and confidence in test results, as well as higher efficiency and traceability.

This guidance document is a main outcome of the EuroGentest workshops on accreditation and quality management and represents a consensus opinion of experts, accredited and non-accredited laboratories. It aims to provide support for genetic testing laboratories that are implementing a QMS or working towards accreditation, using ISO 15189 as the reference standard. Involvement of all people working in the laboratory and dividing responsibilities will have a positive effect on the motivation of the group and consequently on the overall implementation. It is advised to start with the introduction of a ‘simple’ key aspect in the laboratory, for example document control. Later on, more complex elements can be tackled, such as validation and MR. On the other hand, elements of the standard are all linked and interact with each other. The results of an internal audit, for example, will be used in the MR. It is, therefore, impossible to consider the different elements of a QMS or ISO 15189 as purely isolated pieces.

From the workshops, the authors experienced that the common situation of participants encourages them to start or continue implementing and improving their QMS. As the workshops bring people together from different countries, fields and backgrounds, they offer a unique platform to share ideas and to learn from each other. Especially because ISO requirements do not often offer concrete ways on how to implement a QMS, discussing with peers might be very valuable.

These recommendations and the workshops, including the round tables at the annual ESHG Congress, aim to provide an effective and successful tool for spreading the message of quality, and encouraging and facilitating the uptake of and participation in quality management and accreditation in a harmonized way. They also contribute to promoting a culture of quality for the full testing process, which starts and ends with the patient. Given the importance of genetic testing, it is essential to aim for the highest levels of QAu and continuous quality improvement. Finally, for clinical genetic testing services that seek to work in a professional manner, it is essential to have an appropriate recognition of competence. Accreditation provides the ideal manner for this, because it formally and officially recognizes technical and scientific competence, facilitates exchange of services, provides a valuable management tool and enhances confidence that the needs and requirements of all users (clinicians, patients and families) are met.