http://orcid.org/0000-0002-8099-3043
Figures
Abstract
Nowadays, evaluating the quality of health services, especially in primary health care (PHC), is increasingly important. In a historical perspective, the Department of Health (United Kingdom) developed and proposed a range of indicators in 1998, and lately several health, social and political organizations have defined and implemented different sets of PHC quality indicators. Some systematic reviews in PHC quality indicators are reported but only in specific contexts and conditions. The aim of this study is to characterize and provide a list of indicators discussed in the literature to support managers and clinicians in decision-making processes, through an umbrella review on PHC quality indicators. The methodology was performed according to PRISMA Statement. Indicators from 33 eligible systematic reviews were categorized according to the dimensions of care, function, type of care, domains and condition contexts. Of a total of 727 indicators or groups of indicators, 74.5% (n = 542) were classified in process category and 89.5% (n = 537) with chronic type of care (n = 428; 58.8%) and effective domain (n = 423; 58.1%) with the most frequent values in categorizations by dimensions. The results of this overview of reviews are valuable and imply the need for future research and practice regarding primary health care quality indicators in the most varied conditions and contexts to generate new discussions about their use, comparison and implementation.
Citation: Ramalho A, Castro P, Gonçalves-Pinho M, Teixeira J, Santos JV, Viana J, et al. (2019) Primary health care quality indicators: An umbrella review. PLoS ONE 14(8): e0220888. https://doi.org/10.1371/journal.pone.0220888
Editor: Wisit Cheungpasitporn, University of Mississippi Medical Center, UNITED STATES
Received: March 18, 2019; Accepted: July 22, 2019; Published: August 16, 2019
Copyright: © 2019 Ramalho et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files, and are from studies indexed and available in four databases (Medline, Scopus, ISI-WOS and CINAHL via Ebsco Host).
Funding: The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was financed by FEDER - Fundo Europeu de Desenvolvimento Regional funds through the COMPETE 2020 - Operacional Programme for Competitiveness and Internationalisation (POCI), and by Portuguese funds through FCT - Fundação para a Ciência e a Tecnologia in the framework of the project POCI-01-0145-FEDER-030766 (“1st.IndiQare - Quality indicators in primary health care: validation and implementation of quality indicators as an assessment and comparison tool”).
Competing interests: The authors have declared that no competing interests exist.
Introduction
Primary health care (PHC) is defined by the World Health Organization (WHO) as the “essential health care based on scientifically sound and socially acceptable methods and technology, which make universal health care accessible to all individuals and families in a community. It is through their full participation and at a cost that the community and the country can afford to maintain at every stage of their development in the spirit of self-reliance and self-determination" [1]. Some studies suggest that health systems with better financial and clinical results are those with a greater focus on PHC, thus enhancing the sustainability of the entire health system [2–4]. This depends on providing high quality primary health care, hence raising the need to develop methods for quality assessment and monitoring [5]. One of these methods is the use of quality indicators—a quantitative measure of the activities, that can assist as a guideline for quality monitoring and evaluation of relevant patient care and support services [6,7,8].
Quality of care was defined by the Institute of Medicine (IOM) in 1999 as the degree to which health services increase the likelihood of desired outcomes and are consistent with current professional knowledge [9]. The evaluation of the degree of quality of care is done through indicators, a set of measures that assist health care quality monitoring and evaluation in several areas, such as governance, management, assistance and support [10,11]. The importance of indicators is given by the fact that they allow for signalling opportunities of improvement, and controlling compliance with the best existing clinical practices, through quantitative parameters (planning, organizational, clinical) aiming better processes and outcomes [12,13].
Studies of how quality can be assessed were conducted by Donabedian and Fleming, who categorized the information from which inferences can be drawn on the topic, in three categories: structure, process and outcome [14]. The “three-part” assessment approach performed by the authors is only possible because a good structure increases the probability of a good health care processes, and good processes increase the probability of good outcomes [14]. Importantly, for a process to be a valid measure of quality, it must be closely related to a result that people care about [12]. It is also worth remembering that we often find factors that interfere with patients' survival and health-disease dynamics, and in these cases, it may be useful for outcome measures to be adjusted for other factors (such as lifestyle, disease) to control confounders that may affect the analysis of outcome indicators [10]. The development and selection of indicators must meet requirements for use, such as validity, reliability, relevance, pertinence, applicability, data availability, minimum bias, and moreover based on the best evidence available [15,16].
For historical contextualization only, the National Health Service Executive and the Department of Health in United Kingdom (UK)—pioneers in this area—proposed a range of indicators in 1998, many of which would apply to primary health care groups [17]. The interest in assessing the quality of primary health care services has increased, especially after 2004, when the Quality and Outcomes Framework (QOF) was introduced in the UK [18–20]. After the development of the QOF, some pay-for-performance systems have been developed over the years. These were based on the concept of allocative efficiency: “the optimal use of resources to achieve the intended outcomes” [21]. As such, financial incentive schemes are being used for PHC units worldwide and professionals, representing a way of rewarding improvements in productivity and/or adaptation to better quality healthcare provision [22].
Lately, several health, social and political organizations such as World Health Organization (WHO), Organization for Economic Cooperation and Development (OECD), European Commission and the Agency for Research and Quality of Health Care (AHRQ), have defined and implemented different sets of quality indicators for primary care [23–25]. There are several studies proposing PHC quality indicators in different countries, which have led to some systematic reviews revealing substantial geographical variability regarding quality of primary care services [26]. Identifying papers referring to PHC quality assessment projects, these systematic reviews reported that the number and content of indicators and their domains varied among studies. Moreover, they demonstrated that the lack of standardization of collection tools across projects would lead to invalid comparisons [27–31].
Considering the importance of understanding the PHC context, identifying and measuring quality indicators are essential factors for the achievement of high-quality care [32]. Some systematic reviews related to the topic are available in the literature but with focus on specific contexts, making it necessary to synthesize and understand the reality of these indicators in a broader scope. The aim of this umbrella review is not an exercise for a meta-review, but rather to identify systematic reviews of studies on quality indicators (QI) for PHC to provide a list of selected indicators considered in systematic reviews.
Methods
An umbrella review was conducted, to collect and extract data from all systematic review studies uncovering PHC quality indicators. The methodology was performed according to PRISMA Statement [33] (Fig 1 and S1 File). All the phases were performed by two independent reviewers with a third as a tie-breaker, considering the eligibility criteria. Composing PICO, participants were the primary care systems and the intervention to be analysed is the implementation of quality indicators. The comparator was the categories such as context, dimension, type and domain of care, and the main outcome was the primary health care quality indicators to present a summary list of the indicators used in PHC. The protocol was registered at PROSPERO [34,35] with number CRD42019124170 (S2 File).
Search strategy
Identification phase.
From an initial set of studies, a search expression was defined and calibrated [36,37] through test rounds for each and combined terms in electronic databases. The search database eligible for calibration was MEDLINE. There were no restrictions on publication period or language. We considered studies from inception until 20th December 2018, the date when the search was performed. After the calibration, the most relevant search expression (S3 File) was used in four databases (MEDLINE, Web of Science, Scopus and CINAHL plus via EBSCOhost). The terms related to systematic reviews were chosen using information of balance between sensitivity and specificity terms, available in the literature [38–42].
Study selection
Eligibility criteria.
Included studies 1) are systematic reviews regardless of their objective or nature (including studies that have used a formal systematic review as their starting point) 2) have a primary health care scope and 3) aim at quality indicators assessment or development. We excluded studies that 1) did not have an abstract in the screening phase or 2) which, in the eligibility phase, did not have the full-text version available, even after direct contact with the author.
Screening phase.
Once we obtained all the articles, duplicate between databases were identified and excluded using Endnote. From 2817 articles, a total of 1480 remained after removing the duplicates and were evaluated in the screening phase (reading of title and abstracts) by two independent reviewers and a third as a tie-breaker.
Eligibility phase.
Full-texts of all the included articles were extracted (n = 33). As it was planned to contact the corresponding author if the full text of the article was not available, we used the ResearchGate website to extract full text articles, or to contact the authors for the articles that were not available. All eligible articles were assessed in full text format. The eligibility criteria were reapplied by two independent reviewers and a third as a tie-breaker, and the reference lists of each eligible article were scrutinized for any omitted studies.
Quality assessment and risk of bias.
The evaluation of the quality and risk of bias of the eligible systematic reviews was carried out by evaluation through AMSTAR-2 tool [43]. The disagreement between the reviewers was solved by consensus in an agreement meeting by three reviewers. The AMSTAR-2 tool was considered for the definition of quality classification, fulfilling the systematic review research model. Articles that meet AMSTAR-2 requirements have been classified as "HIGH"; those that did not meet up to 2 relevant requirements were classified as "MODERATE", and those with more than 2 requirements not appraised were classified as "LOW". This quality assessment was carried out in order to understand how the studies were conducted and how the indicators were selected. However, none of the selected articles were excluded based on this assessment because the objective of this umbrella review does not include results from implementation of indicators, only a list of indicators implemented. The AMSTAR-2 items #11 and #12 were not applicable to the studies.
Data collection process.
In first stage, a standard data extraction form was created, and general data extracted from each study included the following characteristics: article title, name of first author, publication type, country of origin, year of publication and indicators identified in the studies. Three reviewers independently extracted the data. Differences in data extracted was resolved by consensus method.
A second stage consisted in abstracting information regarding quality indicators using the primary studies in the systematic reviews included. This was necessary since some indicators identified in the systematic reviews lacked a proper description. Finally, indicators duplicated were identified by the reviewers involved in the first and second stage of data extraction and excluded through consensus.
Synthesis analysis.
Analysis of the indicators were carried independently by two reviewers and third as a tie-breaker, who categorized the indicators presented in the systematic reviews included, according to five classifications frameworks: Context reflects the WHO ICPC-2 chapters categorization (General and Unspecified; Blood, Blood Forming Organs and Immune Mechanism; Digestive; Eye; Ear; Cardiovascular; Musculoskeletal; Neurological; Psychological; Respiratory; Skin; Endocrine/Metabolic and Nutritional; Urological; Pregnancy, Childbearing, Family Planning; Female Genital; Male Genital; Social Problems) [44]; the dimensions of care was defined based on the framework proposed by Donabedian to assess quality of healthcare (structure, process and outcome)[10,14], type of care reflects whether an indicator is associated with acute, chronic, or preventive care [10,45]; function of care conveys information about the purpose of health care (screening and prevention, diagnosis, treatment, follow up and continuity) [10,45] and domains and domain of health care quality was defined based on the framework proposed by National Academy of Medicine (NAM)(former Institute of Medicine) in 2001 (safe, effective, efficient, timely, patient-centred, equitable)[9].
Frequencies were computed based on these frameworks to analyse and summarize the information extracted, in two perspectives: Indicators by Context and Dimensions of care; and Type, Function and Domain by Dimensions of care.
Results
Search and study selection
The identification phase results returned 2817 articles (being 419 MEDLINE, 1452 Scopus, 567 ISI-WOS, 379 CINAHL via EBSCOhost). After removal of duplicate articles our research started with 1480 articles. Title and abstract were scrutinized for relevance based on inclusion and exclusion criteria. From a total of 1401 excluded articles, 1332 did not meet the eligibility criteria and 69 had no abstract available. The eligibility phase started with 79 articles that were read in their full-text versions, checking for the eligibility criteria. The studies identified by that involved RAND methodology, their inclusion in the umbrella review was justified since the methods included an initial systematic review prior the implementation of a panel discussion for validating appropriateness of indicators. Since the goal was to be as inclusive/comprehensive as possible, these studies were also included. In the perspective of the authors, the exclusion of these studies could compromise comprehensiveness of the umbrella review. The excluded studies (n = 46) did not have a full text version available or did not meet the eligibility criteria. Thirty-three articles were selected [29,30,46–77], for qualitative analysis and for the quality and risk of bias assessment. (Fig 1)
The Quality and Risk of Bias Assessment was carried out using the AMSTAR-2 assessment tool [45]. This assessment performed by the reviewers classified the confidence rate of each review as "low" (n = 14), “moderate” (n = 17) or “high” (n = 3) (S4 File).
Among the studies with low overall confidence rate, the main points of non-compliance with the requirements were, the non-performance of adequate studies selection with no extraction in duplicate (at least two independent reviewers); studies presented the quantity of excluded articles but without proper justification; not considering risk of bias (RoB) in individual studies when interpreting / discussing the results of the review; not using a satisfactory technique to assess RoB in individual studies that were included in the review and did not provide a satisfactory explanation for, or discussion of any heterogeneity observed in the results.
Study characteristics.
The 33 articles in this umbrella review included articles from Canada (n = 5), Spain (n = 5) and the United Kingdom (n = 5), among other countries (Fig 2).
Although the diversity of countries where the systematic reviews were performed, all articles were evaluated in the English version, except article #16 (Spanish).
The reviews comprised a total of 1406 included primary studies and 21 national guidelines, The databases used to search for these articles were the most varied, with the most used databases: MEDLINE (100%) and EMBASE (70%) (Table 1). Seven hundred and twenty seven (n = 727) indicators were extracted from the systematic reviews and primary studies in the reviews (Supplementary Material S1 Appendix).
The dimension of care with the highest number of indicators by context was process (n = 548, 74.5%), followed by outcome (n = 146, 20.0%) and structure (n = 46, 6.0%). The frequency of indicators among the classification by dimension of care and condition contexts is shown in Table 2. When analysed by dimension of care and condition context, the indicator totals within each dimension (columns) could not be added up because there were indicators (n = 13) that participate in more than one context category within each dimension of care. The total number of indicators analysed was the denominator of the percentage in parentheses and refers to the total number of indicators in the extraction list (n = 727) indicated in the heading. The same is observed in context totals (lines). The ranking of the highest number of indicators found were classified in the categories A—general and non-specific followed by the K—Circulatory System categories specific, P–Psychological and R—Respiratory System. The categories B—Blood, hematopoietic and lymphatic organs, H—Ears and Z—Social Issues, had no indicators presented in the included studies.
Among the indicators of structure (n = 45), the indicators with the most frequent type of care were those classified in all three categories—Acute, Chronic and Preventive (n = 34, 45.3%), e.g. Professional profiles; Primary care expenditures; Availability of primary care services. Those of specific category of type of care were less frequent (Table 3).
Structure indicators were more commonly assigned to more than three functions of care (n = 35, 77.7%) (Diagnosis, Screening and Prevention, Follow-up and continuity, Treatment), eg Availability: Number of physicians per unit of population; Availability: Number of hospital beds per unit of population; Technical efficiency.
Most structure indicators were associated with the effective domain of health care quality (n = 22, 48.8%) e.g. Governance: (From) centralization of primary care management and service development; Integration of primary care in the health care system; Appropriate technology in primary care. No structure indicators was associated with the safe domain of health care quality.
Among the indicators of process (n = 542), Chronic care was the most frequent type of care observed (n = 355, 65.5%), e.g. Comorbid psychiatric conditions and response to treatment; Follow-up contacts during treatment episode after initial evaluation; Comprehensive diabetes care: HbA1c testing. Preventive care (n = 88, 16.2%) and all types of care (n = 80, 14.7%) shared similar frequencies (Table 4).
Treatment was the most frequent function of care of the process indicators (n = 254, 46.8%), e.g Tranquilisers prescribed: % of the recommended; Possible contraindications should be taken into account when antibiotics are prescribed; Co-prescription of itraconazole with simvastatin, or with atorvastatin at a dose ≥80mg. Screening and prevention and Follow up and continuity were also common, associated with 111 indicators each. Examples of Screening and Prevention indicators are: Pap smear rate; Urinary incontinence during initial dementia evaluation; Preventive care Immunizable conditions; Medical attention for nephropathy; and of Follow up and Continuity: Follow up by the same clinician; Plan for follow up care explained and scheduled; Extra pyramidal effects monitoring; Percentage of patients with asthma and measures of variability or reversibility recorded. Most process indicators were also associated with the effective domain of health care quality (n = 310, 57.2%) e.g. Follow-up contacts during treatment episode after initial evaluation; Coordinated care; Asthma: Percentage of children with follow-up from the same doctor for at least 80% of their visits. Also a common domain of health care quality in the listing was Safe (n = 152, 28%), e.g. Detection of Falls; Polyfarmacy; Systemic Lupus Erythematosus: Discussion about teratogenic risks of medication.
Among the indicators of Outcome (n = 140), Chronic care was the most frequent type of care observed (n = 67, 47.8%), e.g. Absenteeism from Work/School for Asthma; Proportion with increased BMI / abdominal waist line; Prevention of pressure ulcers in patients included in the chronic dependent patients care program; Duration of untreated psychosis. The frequency of indicators regarding acute care only (n = 51, 36.4%) and preventive care (n = 44, 31.4%) were similar (Table 5).
Treatment was the most frequent function of care within outcome indicators (n = 57, 40.7%), e.g Sedation side effects; Number of deaths in seven days between those whose calls were handled by doctors or nurses. Screening and Prevention (n = 25, 17.8%) and Diagnosis (n = 22, 15.7%) followed with similar number of outcome indicators. Examples of Screening and Prevention indicators are: Quality of maternal and child health care: maternal mortality rates; Quality of health promotion: Smoking rate; Preventive care: Low birth weight rate.
Finally, most outcome indicators were associated with the effective domain of health care quality (n = 91, 65%) e.g. Potentially preventable hospitalisation clinical indicator of Chronic Obstructive Pulmonary Disease; Comorbid psychiatric conditions and response to treatment. Also, a common health care quality domain in the listing was Patient-centered (n = 26, 18.5%), e.g. Patients with multiple chronic conditions and medications attended in primary care; Patient Quality of Life; Patient satisfaction with the family physician/specialist coordination of care.
Discussion
Primary health care (PHC) is where the patient's first contact with the health system occurs and comprises a range of actions which includes many dimensions, domains, and contexts [14]. Due to these characteristics, it becomes important to evaluate and monitor the quality of primary care [78–80]. It is established that primary care can lead to better health outcomes, lower costs, and greater equity in health [81] and this can be achieved by using QIs, a set of objective measures with clinical evidence [6,82–83] that can represent an acceptable standard of care across a specific patient population [84].
As the aim of this umbrella review of systematic reviews was to search indexed literature, in order to find a setlist of QI useful for monitoring quality in PHC, our study shows interesting answers to what was proposed, identifying 33 systematic reviews of studies on quality indicators in primary health care and providing a list of selected indicators considered in the included reviews. The study resulted in 727 quality indicators, which were later categorized by context, dimension, type of care, function and domain.
Context
Context of care was classified according to the International Classification of Primary Care (ICPC-2), which is recommended by the World Organization of Family Doctors (WONCA) for codification in this level of care.[44]. Although practical and useful for primary care, this classification represents a simplification and attempt at uniformization with other classification systems such as the International Classification of Diseases (ICD-11), which is not achieved completely [85]. Furthermore, since ICPC-2 is a classification based primarily in the location of the symptoms or disease, the authors could not define the context for 100 indicators, since they relate mostly to organizational measures not contemplated in adopted system. “Not defined” was the fourth most common context, representing 13.8% of the total of indicators found.
The majority of the indicators belong to the context category “A–General and Unspecified” (n = 112, 15.4%), which may reflect an attempt at creating indicators applicable to a wide range of procedures and contexts. Circulatory, psychological, respiratory, musculoskeletal and endocrine/metabolic diseases are the next most frequent contexts, indicating also a bigger concern for areas which are more prevalent in primary care (see Table 2).
Dimension
Most of the indicators found by the authors were related to the dimension of Process (n = 542, 74.5% of total). As defined by Donabedian, this dimension focuses in what is actually done, such as patient’s procedures in seeking care and practitioner’s activities while providing it [14]. Since QIs represent an opportunity for improvement in areas where quality standards are not met, process indicators may help implementing better procedures and guidelines, resulting in better health care. Outcome dimension was the second most frequent dimension (n = 140, 19.2%); since healthcare outcomes depend on the care provided, these indicators evaluate the result of the course of action of PHC professionals, unlike process indicators which evaluate a single aspect of care.
Type of care
Type of care was classified as acute, chronic or preventive, with “Chronic” being the most frequent. Indicators focused on chronic care are very helpful, since family doctors follow their patients longitudinally for many years, monitoring and managing the chronic diseases they develop throughout their lives [79]. The management and control of chronic conditions/diseases in the population is one of the main focuses of the activities of primary health care, being also the most studied and evaluated by the QIs, as our study demonstrates. Indicators such as control of prescriptions and monitoring of diseases such as asthma, COPD, hypertension and diabetes, as well as indicators of ambulatory care sensitive conditions that can generate avoidable hospitalizations are part of the list of indicators presented [30].
Function
Indicators relating to “Treatment” were the most frequent, followed by “Screening and Prevention” and “Follow-up and Continuity”. Once again, the results mirror important aspects of PHC. The consideration of the patient as a whole and the approach of disease in a holistic perspective imply that the healthcare provider must consider indications, potential adverse effects and comorbidities of each patient before elaborating a treatment plan [79]. Within outcome indicators, most these were focused on treatment, contributing to the evaluation of its complications and preventable hospitalizations, once again alerting providers to re-evaluate their patients and review therapeutic options.
Regarding “Screening and Prevention”, the prevention of disease as well as early diagnosis are the main focus of this level of care [1]; the development of screening programs for oncological conditions and adequate follow-up for prevention of complications contribute to better health care in this aspect.
Domain
“Effective” was the most common domain among the three dimensions of care. Indicators under this domain focus on the capacitation of PHC providers and their articulation with secondary care. Since the effectiveness of a health system depends on the quality of its primary care [29,86], it would be expected that this would be an area of interest.
Other domains such as “Patient-centered” or “Safety” were also commonly evaluated through QIs, demonstrating once again the concern for a holistic approach of PHC.
Limitations
Although there is a significant amount of literature on health quality indicators, some of them are not directly linked to PHC, making it difficult to extrapolate the conclusions of the QI that are applied mainly to the secondary and tertiary levels of attention. Most articles published on QIs in PHC tend to choose very limited and specific areas of health care, without a generic approach to PHC as a whole. The uniqueness and heterogeneity found in these studies show the importance of comprehensive systematic reviews on PHC.
Systematic reviews included in this paper selected primary studies using slightly different methodological assessment and statistical pooling; some of these articles did not discriminate how many primary studies were included in the analysis. The use of different databases in each systematic review and different methods for choosing search terms, calibration and specificity of the search expressions must be considered when interpreting the results.
The authors of this article have searched the primary studies included in each systematic review in order to obtain a list of PHC quality indicators. The lack of a uniform method to collect and present the QIs among the included reviews limited the ability to withdraw complete information from every paper. As an example, most studies were missing information regarding the numerator, denominator and calculation method for each QI.
Conclusions
This is, to the best of our knowledge, the first umbrella review focusing on QIs for primary healthcare in a border scope. We present a final list of indicators (S1 Appendix supplementary material) from eligible systematic reviews summarizing the indicators available in the literature, allowing us to understand which areas of primary care are better covered by these measures. The results of our umbrella review are valuable and imply the need for future research and practice regarding quality indicators, as a great opportunity for further studies to test the acceptability, feasibility, reliability, comparison tools and validity of those indicators, while also checking for problems with their implementation to PHC, with adequate information and registration systems. It also provides a ready way for clinicians, managers and health decision makers to gain a clear understanding of the most evidence-based publications related to PHC quality indicators.
Supporting information
S1 Appendix. Supplementary material–indicators list.
https://doi.org/10.1371/journal.pone.0220888.s005
(PDF)
References
- 1.
World Health Organization. Declaration of Alma-Ata. Adopted at the International Conference on Primary Health Care, Alma-Ata, 6–12 September 1978, USSR.
- 2. Starfield B, Shi L, Macinko J. Contribution of primary care to health systems and health. The Milbank quarterly. 2005;83(3):457–502. pmid:16202000
- 3. Smith J. Primary care: balancing health needs, services and technology. Int J Integr Care. 2001;1:e36.
- 4. Macinko J, Starfield B, Shi L. The contribution of primary care systems to health outcomes within Organization for Economic Cooperation and Development (OECD) countries, 1970–1998. Health services research. 2003;38(3):831–65. pmid:12822915
- 5. Fracolli LA, Gomes MF, Nabao FR, Santos MS, Cappellini VK, de Almeida AC. Primary health care assessment tools: a literature review and metasynthesis. Ciencia & saude coletiva. 2014;19(12):4851–60.
- 6. Agenda for change–Characteristics of Clinical Indicators. Joint Commission. QRB–Quality Review Bulletin, v. 15, n. 11, 1989.
- 7. van der Bij S, Khan N, Ten Veen P, de Bakker DH, Verheij RA. Improving the quality of EHR recording in primary care: a data quality feedback tool. Journal of the American Medical Informatics Association: JAMIA. 2017;24(1):81–7. pmid:27274019
- 8. Kemple T. Quality in primary health care. Complexity: a simple approach to quality. Are we there yet? BMJ (Clinical research ed). 2009; 338:b1812.
- 9.
Institute of Medicine (US) The National Roundtable on Health Care Quality; Donaldson MS, editor. Measuring the Quality of Health Care: A Statement by The National Roundtable on Health Care Quality. Washington (DC): National Academies Press (US); 1999. Available from: https://www.ncbi.nlm.nih.gov/books/NBK230819/ https://doi.org/10.17226/6418
- 10. Mainz J. Defining and classifying clinical indicators for quality improvement. Int J Qual Health Care. 2003;15(6):523–530. pmid:14660535
- 11. Klazinga N, Stronks K, Delnoij D, Verhoeff A. Indicators without a cause. Reflections on the development and use of indicators in health care from a public health perspective. Int J Qual Health Care. 2001;13(6):433–438. pmid:11769744
- 12.
Donaldson MS. Measuring the quality of health care. Washington, DC: National Academy Press; 1999. p. 3
- 13. Althabe F, Bergel E, Cafferata ML, Gibbons L, Ciapponi A, Alemán A, et al. Strategies for improving the quality of health care in maternal and child health in low- and middle-income countries: an overview of systematic reviews. Paediatr Perinat Epidemiol. 2008;22(s1):42–60. pmid:18237352
- 14. Donabedian A. The Quality of Care: How Can It Be Assessed? JAMA. 1988;260(12):1743–1748. pmid:3045356
- 15. Wollersheim H, Hermens R, Hulscher M, Braspenning J, Ouwens M, Schouten J, et al. Clinical indicators: development and applications. Neth J Med. 2007;65(1):15–22. pmid:17293635
- 16.
Guide to Inpatient Quality Indicators:Quality of Care in Hospitals–Volume, Mortality, and Utilization. AHRQ Quality Indicators; 2007. Available at https://www.ahrq.gov/downloads/pub/inpatqi/iqi_guide.pdf
- 17. McColl A, Roderick P, Gabbay J, Smith H, Moore M. Performance indicators for primary care groups: an evidence based approach. Bmj. 1998;317(7169):1354–60 pmid:9812935
- 18.
Department of Health. Rewarding quality and outcomes. In: Department of Health. The new GMS contract. London: The Stationery Office, 2003: 17–24.
- 19.
NHS Digital Quality and Outcomes Framework, Achievement, prevalence and exceptions data—2017–18 [PAS] https://digital.nhs.uk/data-and-information/publications/statistical/quality-and-outcomes-framework-achievement-prevalence-and-exceptions-data accessed 22 November 2018
- 20. Sahota N, Hood A, Shankar A, Watt B, Ramaiah S. Developing performance indicators for primary care: Walsall's experience. The British journal of general practice: the journal of the Royal College of General Practitioners. 2008; 58(557):856–61.
- 21. Gill P, Foskett-Tharby R, Hex N. Pay-for-performance and primary care physicians: lessons from the U.K Quality and Outcomes Framework for local incentive schemes. J R Soc Med. 2015;108(3):80–2. pmid:25792612
- 22. Allen T, Mason T, Whittaker W. Impacts of pay for performance on the quality of primary care. Risk management and healthcare policy. 2014; 7:113–20. pmid:25061341
- 23. Mattke S, Epstein AM, Leatherman S. The OECD Health Care Quality Indicators Project: history and background. International journal for quality in health care: journal of the International Society for Quality in Health Care. 2006;18 Suppl 1:1–4.
- 24.
WHO. Library Cataloguing-in-Publication Data Quality of care: a process for making strategic choices in health systems. 2006 accessed 22 November 2018. http://www.who.int/management/quality/assurance/QualityCare_B.Def.pdf
- 25.
Agency for Healthcare Research and Quality. Guide to the Prevention Quality Indictors. Rockville, MD: AHRQ; 2017. https://www.qualityindicators.ahrq.gov/Archive/PQI_TechSpec_ICD10_v70.aspxAccessed 22 November 2018.
- 26. Kerssens JJ, Groenewegen PP, Sixma HJ, Boerma WG, van der Eijk I. Comparison of patient evaluations of health care quality in relation to WHO measures of achievement in 12 European countries. Bulletin of the World Health Organization. 2004;82(2):106–14. pmid:15042232
- 27. Simou E, Pliatsika P, Koutsogeorgou E, Roumeliotou A. Quality Indicators for Primary Health Care: A Systematic Literature Review. Journal of public health management and practice: JPHMP. 2015;21(5):E8–E16. pmid:24374361
- 28. Baxter S, Johnson M, Chambers D, Sutton A, Goyder E, Booth A. The effects of integrated care: a systematic review of UK and international evidence. BMC Health Serv Res. 2018;18(1):350. Published 2018 May 10. pmid:29747651
- 29. Kringos DS, Boerma WG, Hutchinson A, van der Zee J, Groenewegen PP. The breadth of primary care: a systematic literature review of its core dimensions. BMC Health Serv Res. 2010;10:65. Published 2010 Mar 13. pmid:20226084
- 30. Caughey GE, Kalisch Ellett LM, Wong TY. Development of evidence-based Australian medication-related indicators of potentially preventable hospitalisations: a modified RAND appropriateness method. BMJ Open 2014;4:e004625. pmid:24776711
- 31. Campbell SM, Braspenning J, Hutchinson A, Marshall MN. Research methods used in developing and applying quality indicators in primary care. BMJ (Clinical research ed). 2003;326(7393):816–9.
- 32.
Canadian Institute for Health Information. Primary Health Care in Canada: A Chartbook of Selected Indicator Results 2016. Ottawa, ON: CIHI; 2016.
- 33. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA group. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med 2009; 6: e1000097. pmid:19621072
- 34. Moher D, Booth A, Stewart L. (2014). How to reduce unnecessary duplication: use PROSPERO. BJOG: An International Journal of Obstetrics & Gynaecology, 121(7), 784–786. pmid:24629162
- 35. Waugh N. Misgivings about PROSPERO in solving the problem of duplicate systematic reviews. BMJ. 2013;347:f5496. pmid:24022042
- 36. Wilczynski NL, McKibbon KA, Haynes RB. Enhancing retrieval of best evidence for health care from bibliographic databases: calibration of the hand search of the literature. Studies in health technology and informatics. 2001;84(Pt 1):390–3. pmid:11604770
- 37. Montori VM, Wilczynski NL, Morgan D, & Haynes RB. Optimal search strategies for retrieving systematic reviews from Medline: analytical survey. BMJ, 2004; 330(7482), 68. pmid:15619601
- 38. Boynton J, Glanville J, McDaid D, Lefebvre C. Identifying systematic reviews in MEDLINE: developing an objective approach to search strategy design. Journal of Information Science, 1998; 24(3), 137–154.
- 39.
McMaster Univesity Health Information Research Unit—HIRU ~ Search Strategies for MEDLINE in Ovid Syntax and the PubMed translation. https://hiru.mcmaster.ca/hiru/HIRU_Hedges_MEDLINE_Strategies.aspxacessed in 05th dec 2018.
- 40. Shojania KG, Bero LA. Taking advantage of the explosion of systematic reviews: an efficient MEDLINE search strategy. Effective clinical practice: ECP. 2001;4(4):157–62. pmid:11525102
- 41. Lee E, Dobbins M, DeCorby K, McRae L, Tirilis D, Husson H. An optimal search filter for retrieving systematic reviews and meta-analyses. BMC Medical Research Methodology, 2012; 12(1). pmid:22512835
- 42. Lunny C, McKenzie JE, McDonald S. Retrieval of overviews of systematic reviews in MEDLINE was improved by the development of an objectively derived and validated search strategy. Journal of clinical epidemiology. 2016;74:107–18 pmid:26723872
- 43. Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. Bmj. 2017;358:j4008. pmid:28935701
- 44.
WONCA–World Organization of Family Doctors. An introduction to the international classification of primary care. Version 2" at http://www.ph3c.org/PH3C/docs/27/000098/0000054.pdf Accessed on 05 Jul 2019.
- 45. Wenger NS, Roth CP, Shekelle P, Investigators A. Introduction to the assessing care of vulnerable elders-3 quality indicator measurement set. Journal of the American Geriatrics Society. 2007;55 Suppl 2: S247–52.
- 46. Jan C, Chiu T, Chen CY, Guo F, Lee MA. 10-year review of health care reform on Family Practice Integrated Care Project—Taiwan experience, Family Practice, Volume 35, Issue 4, 23 July 2018, Pages 352–357, pmid:29194539
- 47. Mazur A, Brindis CD, Decker MJ. Assessing youth-friendly sexual and reproductive health services: a systematic review. BMC Health Serv Res. 2018;18(1):216. Published 2018 Mar 27. pmid:29587727
- 48. Spencer R, Bell B, Avery AJ, Gookey G, Campbell SM. Royal College of General Practitioners. Identification of an updated set of prescribing—safety indicators for GPs. Br J Gen Pract. 2014;64(621):e181–90. pmid:24686882
- 49. Flodgren G, Pomey MP, Taber SA, Eccles MP. Effectiveness of external inspection of compliance with standards in improving healthcare organisation behaviour, healthcare professional behaviour or patient outcomes. The Cochrane database of systematic reviews. 2011(11):CD008992. pmid:22071861
- 50. Menear M, Dore I, Cloutier AM, Perrier L, Roberge P, Duhoux A, et al. Chronic physical comorbidity burden and the quality of depression treatment in primary care: a systematic review. Journal of psychosomatic research. 2015;78(4):314–23. pmid:25649274
- 51. Batbaatar E, Dorjdagva J, Luvsannyam A, Savino MM, Amenta P. Determinants of patient satisfaction: a systematic review. Perspectives in public health. 2017;137(2):89–101. pmid:27004489
- 52. Bekkering GE, Zeeuws D, Lenaerts E, Pas L, Verstuyf G, Matthys F, et al. Development and Validation of Quality Indicators on Continuing Care for Patients With AUD: A Delphi Study. Alcohol and alcoholism. 2016;51(5):555–61. pmid:27382086
- 53. Lopez-Vazquez P, Vazquez-Lago JM, Gonzalez-Gonzalez C, Pineiro-Lamas M, Lopez-Duran A, Herdeiro MT, et al. Development and validation of the knowledge and attitudes regarding antibiotics and resistance (KAAR-11) questionnaire for primary care physicians. The Journal of antimicrobial chemotherapy. 2016;71(10):2972–9. pmid:27353468
- 54. Barber CE, Marshall DA, Alvarez N, Mancini GB, Lacaille D, Keeling S, et al. Development of Cardiovascular Quality Indicators for Rheumatoid Arthritis: Results from an International Expert Panel Using a Novel Online Process. The Journal of rheumatology. 2015;42(9):1548–55. pmid:26178275
- 55. Boeckxstaens P, Smedt DD, Maeseneer JD, Annemans L, Willems S. The equity dimension in evaluations of the quality and outcomes framework: a systematic review. BMC health services research. 2011;11:209. pmid:21880136
- 56. To T, Guttmann A, Lougheed MD, Gershon AS, Dell SD, Stanbrook MB, et al. Evidence-based performance indicators of primary care for asthma: a modified RAND Appropriateness Method. International journal for quality in health care: journal of the International Society for Quality in Health Care. 2010;22(6):476–85.
- 57. Addington D, Kyle T, Desai S, Wang J. Facilitators and barriers to implementing quality measurement in primary mental health care: Systematic review. Canadian family physician Medecin de famille canadien. 2010;56(12):1322–31. pmid:21375065
- 58. Sans-Corrales M, Pujol-Ribera E, Gene-Badia J, Pasarin-Rua MI, Iglesias-Perez B, Casajuana-Brunet J. Family medicine attributes related to satisfaction, health and costs. Family practice. 2006;23(3):308–16. pmid:16461452
- 59. Olry de Labry Lima A, Garcia Mochon L, Bermudez Tamayo C. Identification of health outcome indicators in Primary Care. A review of systematic reviews. Revista de calidad asistencial: organo de la Sociedad Espanola de Calidad Asistencial. 2017;32(5):278–88.
- 60. James D. et al. Identifying criteria for assessing the quality of medicines use review referral documentation by community pharmacists. Int J Pharm Pract. 2008;16(6):365–74.
- 61. Ruiz-Canela-Caceres J, Aquino-Llinares N., Sanchez-Diaz J.M., Garcia-Gestoso M.L., de Jaime-Revuelta M.E., Praena-Crespo M. Indicators for childhood asthma in Spain, using the Rand method. Allergologia et immunopathologia. 2015;43(2):147–56. pmid:24731769
- 62. Smits KP, Sidorenkov G, Bilo HJ, Bouma M, Navis GJ, Denig P. Process quality indicators for chronic kidney disease risk management: a systematic literature review. International journal of clinical practice. 2016;70(10):861–9. pmid:27640992
- 63. Yazdany J, Panopalis P, Gillis JZ, Schmajuk G, MacLean CH, Wofsy D, et al. A quality indicator set for systemic lupus erythematosus. Arthritis and rheumatism. 2009;61(3):370–7. pmid:19248127
- 64. Le Maréchal M, Tebano G, Monnier AA, et al. Quality indicators assessing antibiotic use in the outpatient setting: a systematic review followed by an international multidisciplinary consensus procedure. J Antimicrob Chemother. 2018;73(suppl_6):vi40–vi49. pmid:29878218
- 65. Fujita K., Moles RJ, Chen TF. Quality indicators for responsible use of medicines: a systematic review BMJ Open 2018; 8:e020437. pmid:30012782
- 66. Chin WY, Lam CL, Lo SV. Quality of care of nurse-led and allied health personnel-led primary care clinics. Hong Kong medical journal = Xianggang yi xue za zhi. 2011;17(3):217–30. pmid:21636870
- 67. Hagen KB, Smedslund G, Osteras N, Jamtvedt G. Quality of Community-Based Osteoarthritis Care: A Systematic Review and Meta-Analysis. Arthritis care & research. 2016;68(10):1443–52.
- 68. Lake R, Georgiou A, Li J, Li L, Byrne M, Robinson M, et al. The quality, safety and governance of telephone triage and advice services—an overview of evidence from systematic reviews. BMC health services research. 2017;17(1):614 pmid:28854916
- 69. Sidorenkov G, Haaijer-Ruskamp FM, de Zeeuw D, Bilo H, Denig P. Review: relation between quality-of-care indicators for diabetes and patient outcomes: a systematic literature review. Medical care research and review: MCRR. 2011;68(3):263–89. pmid:21536606
- 70. Fernandez-Urrusuno R, Flores-Dorado M, Moreno-Campoy E, Montero-Balosa MC. Selection of indicators for continuous monitoring of the impact of programs optimizing antimicrobial use in Primary Care. Enfermedades infecciosas y microbiologia clinica. 2015;33(5):311–9. pmid:25444046
- 71. Martirosyan L, Voorham J, Haaijer-Ruskamp FM, Braspenning J, Wolffenbuttel BH, Denig P. A systematic literature review: prescribing indicators related to type 2 diabetes mellitus and cardiovascular risk management. Pharmacoepidemiology and drug safety. 2010;19(4):319–34. pmid:19960483
- 72. Byrne MJ, Tickle M., Glenny AM, Campbell S, Goodwin T, O'Malley L. A systematic review of quality measures used in primary care dentistry. International dental journal. 2018.
- 73. Boulkedid R, Abdoul H, Loustau M, Sibony O, Alberti C. Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PLoS One. 2011;6(6):e20476. pmid:21694759
- 74. Pugh MJ, Berlowitz DR, Montouris G, Bokhour B, Cramer JA, Bohm V, et al. What constitutes high quality of care for adults with epilepsy? Neurology. 2007;69(21):2020–7. pmid:17928576
- 75. Forbes LJ, Marchand C, Doran T, Peckham S. The role of the Quality and Outcomes Framework in the care of long-term conditions: a systematic review. The British journal of general practice: the journal of the Royal College of General Practitioners. 2017;67(664):e775–e84.
- 76. Kronenberg C, Doran T, Goddard M, Kendrick T, Gilbody S, Dare CR, et al. Identifying primary care quality indicators for people with serious mental illness: a systematic review. The British journal of general practice: the journal of the Royal College of General Practitioners. 2017;67(661):e519–e30.
- 77. Duhoux A, Fournier L, Menear M. Quality Indicators for Depression Treatment in Primary Care: A Systematic Literature Review. Current Psychiatry Reviews (2011) 7: 104. https://doi.org/10.2174/157340011796391166
- 78.
Foundation Canadian Health Services Research Foundation. Contexts and Models in Primary Healthcare and their Impact on Interprofessional Relationships. Ottawa, 2012. Available at https://www.cfhi-fcass.ca/Libraries/Commissioned_Research_Reports/ScottLagendyk-April2012-E.sflb.ashx
- 79.
WONCA Europe, 2011. The European Definition of General Practice / Family Medicine available at http://www.woncaeurope.org/sites/default/files/documents/Definition%203rd%20ed%202011%20with%20revised%20wonca%20tree.pdf
- 80. Tenbensel T, Burau V. Contrasting approaches to primary care performance governance in Denmark and New Zealand. Health Policy. 2017;121(8):853–61 pmid:28601435
- 81. Starfield B. Primary care and health. A cross-national comparison. Jama. 1991;266(16):2268–71. pmid:1920727
- 82.
Canadian Institute of Health Information, Enhancing the Primary Health Care Data Collection Infrastructure in Canada Report 2 –Pan-Canadian Primary Health Care Indicator Development Project. 2006
- 83.
Manitoba Primary Care Quality Indicators Guide ver. 3.0 –Accessed at https://www.gov.mb.ca/health/primarycare/providers/pin/docs/mpcqig.pdf on Feb 11th 2019
- 84. MacLean CH, Saag KG, Solomon DH, Morton SC, Sampsel S, Klippel JH. Measuring quality in arthritis care: methods for developing the Arthritis Foundation's quality indicator set. Arthritis and rheumatism. 2004; 51(2):193–202. pmid:15077259
- 85. Wood M, Lamberts H, Meijer JS, Hofmans-Okkes IM. The Conversion Between ICPC and ICD-10. Requirements for a Family of Classification Systems in the Next Decade. Family Practice, Vol. 9, nº 3. 1992
- 86. Lee A, Kiyu A, Milman HM, Jimenez J. Improving health and building human capital through an effective primary care system. J Urban Health. 2007 May;84(3 Suppl):i75–85. Review. pmid:17356902; PubMed Central PMCID: PMC1891639.