Evidence-based guidelines (EBG) and good practice statements (GPS)

Evidence-based guidelines (EBG) in medicine are aimed at improving medical practice and patients’ quality of life [1]. Available guideline documents in the pediatric nephrology subspecialty including nephrotic syndrome, chronic kidney disease-mineral and bone disorder, peritonitis, and hypertension are particularly helpful for clinicians in the evaluation, diagnosis, and management of respective diseases [2,3,4,5,6]. In conditions without enough evidence, clinical practice points (CPP) could also guide physicians’ clinical decisions [3]. Developing ungraded best or good practice statements (GPS) for circumstances where evidence is lacking would also be useful in clinical practice [7].

The Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) approach has been developed to facilitate the rigorous and transparent evaluation of overall available evidence and the development of EBGs to guide physicians’ decision-making in certain clinical conditions. In order to develop a guideline, the main initial elements are asking the right, answerable, practical clinical questions and informing these questions with the best available evidence, rating the overall quality of evidence for important and critical outcomes, and moving from evidence to decision [1, 8,9,10,11]. A strong recommendation is supported by moderate- to high-quality evidence (e.g., multiple randomized controlled trials). A conditional recommendation is provided when the evidence is low-quality and always warrants a shared decision-making approach [12].

During specialty and subspecialty training, EBGs are commonly used in daily practice. However, well-structured training would be desirable to enable clinicians to evaluate the rationale of a guideline document and implement it into patient care. Additionally, learning about basic and advanced concepts of moving from evidence to recommendations is a prerequisite in writing a guideline [1, 8,9,10,11].

Evaluation of the perception and knowledge of pediatric nephrologists on guideline development methodology

An online survey consisting of 30 questions was sent to IPNA members to evaluate the perception and knowledge of pediatric nephrologists on EBG development methodology. There were four main categories: (1) physician characteristics, and most commonly used guidelines in practice; (2) physician knowledge to assess methodologic limitations of studies; (3) the level of certainty of pediatric nephrology guidelines; and (4) interest in guideline development training (see Supplementary Material). We herein report the results of this survey to determine priority areas of guideline development methodology training as a part of the strategic planning of educational activities of the International Pediatric Nephrology Association (IPNA).

A total of 108 physicians responded to the survey; 44% of respondents have been practicing pediatric nephrology for more than 20 years, 30% for > 10–20 years, 22% for > 3–10 years, and only 4% for 1–3 years. Faculty members in academic centers comprised 72% of the participants, and 26% were non-academic pediatric nephrologists. Only 3% were pediatric nephrology trainees. Since the number of trainees was small, their answers were not formally compared to other groups.

The usefulness of different guidance documents

Most physicians stated that they use guidelines every day or once/twice a week (39% and 35%, respectively) in their practice, whereas 6% only rarely use them. Fifty-seven percent of pediatric nephrologists with > 20 years of experience (senior pediatric nephrologists) vs. 91% of those with > 3–10 years of experience (junior pediatric nephrologists) have been commonly using guidelines at least once a week or more. There were no significant differences between the answers for EBG and CPP/GPS/consensus papers. Overall, 59% and 29% of the physicians reported that guidance documents were very helpful or quite helpful in guiding their clinical decision-making. When stratifying by years spent in pediatric nephrology, almost half of the senior pediatric nephrologists reported that EBG and CPP/GPS/consensus papers were equally beneficial (47% and 45%) and these rates were significantly lower compared to the answers given by junior pediatric nephrologists (EBG 87% vs. 47%; CPP/GPS/consensus papers 78% vs. 45%). Similarly, most pediatric nephrologists in nonacademic centers (85–88%) but half of the academic faculty (52%) reported that EBGs and CPP/GPS/consensus papers were very helpful in daily practice. Ninety-one percent vs. 62% of the juniors and seniors respectively stated that guidelines changed their practice multiple times and quite often.

Most commonly used guidelines and factors relying on them

KDIGO guidelines were the most commonly used, followed by IPNA, ISPD, and KDOQI guidelines (81%, 62%, 33%, and 32%, respectively). When asked about a single specific guideline that has been most frequently used in daily practice in the past 2–3 years, IPNA steroid-resistant NS treatment guidelines [2], and ISPD pediatric peritonitis prevention and treatment guidelines [5] were the leading ones followed by the AAP hypertension guidelines [6] and the KDIGO glomerular diseases treatment guidelines [13]. Forty-nine percent of the participants reported that they read the whole document, and 27% read only statements and summary of findings tables. When asked to rank the factors that make them rely on a guideline, 58% of them stated that being a product of an international organization (KDIGO, IPNA, ESPN, Cochrane, etc.) was the most important factor. The second and third most important factors were the whole guideline document content (15%) and publication date (15%). The authors of the guideline were ranked as the least important factor by 27% of the physicians (Table 1).

Table 1 Factors for pediatric nephrologist to rely on a guideline
Full size table

Knowledge level of the physicians about GRADE methodology

Half of the respondents (48%) were not involved in guideline writing initiatives, 33% were involved in 1–2 projects, and 19% had 3 or more projects. Only 13% of pediatric nephrologists were formally trained, and 58% definitely wanted GRADE methodology training, while 54% stated that they were quite knowledgeable (63% of seniors vs. 39.1% of juniors), and 34% had limited or no knowledge (44% of seniors vs. 24% of juniors). Most physicians (overall 88%; 70% of juniors vs. 98% of seniors) reported that they consider the risk of bias assessment and 61% know how to evaluate methodological limitations of studies, while two-thirds and almost one-half stated they were not knowledgeable about evaluating hazard graphics (66.6%) and overall certainty of evidence (44%), respectively (Table 2). Regarding the evaluation of the overall certainty of the evidence, 77% of the participants stated that risk of bias assessment was the most important element, followed by publication bias (29%) and inconsistency of evidence extracted from different studies (27%); 19% and 18% reported imprecision and indirectness of evidence. Almost two-thirds (62%) of the participants either agreed or strongly agreed that the risk of bias assessment is largely ignored in many guidance papers based on expert opinion (53% and 8%, respectively).

Table 2 Assessment of knowledge level of the physicians about GRADE methodology
Full size table

In the evaluation of available pediatric nephrology guidelines, based on the answers of 47% of the participants, most of the guideline recommendations were informed by moderate certainty of evidence (Fig. 1A). Most respondents (78%) stated that pediatric nephrology guidelines were mainly based on limited pediatric evidence-based data, while 35% and 4% of the junior and senior pediatric nephrologists, respectively, thought that they were mainly derived from extrapolation of adult data. Fifty-nine percent of physicians agreed that they typically present conditional recommendations (Fig. 1B). According to 73% of participants, GPSs can be considered trustworthy clinical guidelines whenever enough evidence is not available (Fig. 1C).

Fig. 1
figure 1

General perception about pediatric nephrology guidelines. A General opinion on rating of the certainty of the evidence for recommendations in pediatric nephrology guidelines. B. What do pediatric nephrology guidelines typically present? C Can ungraded good practice statements be considered trustworthy clinical guidelines in pediatric nephrology?

Full size image

Inferences from the survey

This survey provides data that many physicians rely on guideline documents and largely use them in the daily care of pediatric nephrology patients. Most respondents placed higher reliability on international or global guidelines such as KDIGO and IPNA and preferred using them. Thus, the wide use of guidance documents puts a large responsibility on guideline development teams to follow a rigorous methodology [1, 8,9,10,11, 14]. Learning how to assess the methodological limitations of studies by using validated risk-of-bias assessment tools for randomized and non-randomized trials is essential for guideline developers [15]. It is a rapidly growing area that deserves much more attention and knowledge, and thus, having a team member trained in the methodology of EBG development is one of the keys to creating trustworthy clinical guidelines. On the other hand, guideline users should also be knowledgeable about the guideline development process and methodological details. Based on our survey, a significant proportion of respondents had limited knowledge about GRADE assessment and in line with this a very small proportion examined the methodology of the guideline, rationale of statements, and summary of findings tables. Since young pediatric nephrologists use the guidelines more frequently than seniors and report less knowledge of the methodology, guideline literacy may be a “must” in pediatric nephrology practice, and priority in methodology training should be given to young pediatric nephrologists. A training syllabus may be prepared, and after a structured training series, future guidelines can be prepared by using scientific and rigorous methodology on the prespecified patient/population, intervention, comparison and outcomes (PICOs) with close collaboration with IPNA working groups to help clinicians’ decision-making in order to improve clinically important patient-centered outcomes.

The overall response rate was around 10% among IPNA members. This is the main limitation of this study. Another one is preferential participation by pediatric nephrologists who are interested in the survey topic. Considering that 75% of the respondents had more than 10 years of experience in pediatric nephrology and only 4% had 1–3 years of experience, remarkably higher response rates of senior pediatric nephrologists from academic centers may be assessed as a “risk of bias.” Additionally, the low response rate among young pediatric nephrologists may be because of their limited knowledge of EBG development methodology.

In conclusion, our main purpose was to obtain an overview of the knowledge of pediatric nephrologists about guideline methodology. Since many topics in pediatric nephrology pose a significant challenge due to scarce evidence with methodological limitations in studies, an increased understanding of EBG development methodology should be encouraged, and young pediatric nephrologists should be the focus group for EBG methodology training.