Development of Antibiotic Classification for Measuring Antibiotic Usage in Korean Hospitals Using a Modified Delphi Method

Antimicrobial resistance is one of the greatest threats to public health.1 It diminishes the effectiveness of antimicrobials to infectious diseases, leading to increased mortality, prolonged hospitalization, and increased medical costs.2 Unfortunately, Korea shows higher antimicrobial consumption rates among countries participating in the Organization for Economic Cooperation and Development countries.3 To promote the proper use of antibiotics, several antimicrobial stewardship programs have been implemented by the government over the last decade in Korea.4 Despite these government-led efforts, antibiotic consumption, especially for broad-spectrum antibiotics, have continued to increase.5

Considering the lack of progress in reducing antibiotic consumption, we suspect efforts to improve awareness of antibiotics and to promote voluntary participation in antimicrobial stewardship programs at the individual hospital level may prove more beneficial. Interestingly, providing antibiotic consumption numbers for comparison among individual hospital has been found to be an effective strategy through which to promote antimicrobial stewardship.6 Accordingly, the Centers for Disease Control and Prevention (CDC) operates the National Healthcare Safety Network (NHSN) and seeks to help individual hospitals analyze data on hospital-wide antibiotic use and to compare one's data to national benchmarks.7

In 2019, the Korean CDC initiated a project to develop a system for measuring and comparing antibiotic usage rates in hospitals in Korea. As a part of this project, the Korean Society of Infectious Diseases has begun to work in cooperation with the Korean Society for Antimicrobial Therapy and the Health Insurance Review & Assessment Service (HIRA) to develop a means with which to classify antibiotic usage in Korean hospitals using a modified Delphi method.

To do so, two series of modified Delphi studies were performed from July to August 2019. The first series sought to classify antibiotics used in Korean hospitals; the second was conducted to analyze antibiotic components according to antibiotic classes. Each Delphi study included two rounds of surveys in order to gather opinions and to refine the information related to each.

The questions for the survey in the first round were adopted from the antibiotic classification of the NHSN.8 We excluded antibiotics not available in Korea at the point of the survey. The appropriateness of each question was evaluated by a four-point Likert scale (1 = very inappropriate, 2 = inappropriate, 3 = appropriate, and 4 = very appropriate). In addition to closed-ended questions, the survey participants could freely express their opinions about the items. The questions for the survey in the second round were developed to reflect responses in the first round: some items were modified or added if > 30% of the respondents expressed the same opinion. The mean score for each question and opinions obtained from the first round were presented in the questionnaire as a reference. The evaluation of each question was performed using a four-point Likert scale, as in the first round.

We recruited a total of 12 panel experts, including infectious diseases physicians (10), a professor of preventive medicine (1), and a researcher at the HIRA Service (1). All of them had a Ph.D. as their highest level of education. The questionnaire was sent to these individuals via e-mail and reminders were sent if the survey had not been returned. Each survey was open for 1 week.

To analyze responses provided in the survey, content validity ratios (CVRs) were calculated. We used the formula CVR = (n_e − N/2)/(N/2), where n_e represents the number of panel experts rating an item as “appropriate” (score of 3 or 4) and N represents the entire number of panelists. The minimum CVR was determined by the number of experts participating in each round and set to 0.56 for the 12 participants.9

The response rates for each round in the first series were 58.3% (7/12) and 75.0% (9/12), respectively. Table 1 shows the results of the surveys. Most of the subjects in the NHSN's antibiotic classification for adults were accepted, except for “antibacterial agents posing the highest risk for Clostridioides difficile infection” (CVR = −1.000) and “agents predominantly used for surgical site infection prophylaxis” was excluded (CVR = 0.333). All subjects for children, however, were rejected. Finally, a total of six classifications were accepted: 1) broad-spectrum antibacterial agents predominantly used for hospital-onset infections in adults (CVR = 1.000), 2) broad-spectrum antibacterial agents predominantly used for community-acquired infections in adults (CVR = 1.000), 3) antibacterial agents predominantly used for resistant gram-positive infections in adults (CVR = 1.000), 4) narrow-spectrum beta-lactam agents in adults (CVR = 1.000), 5) antibacterial agents predominantly used for extensive antibiotic resistant gram-negative bacteria in adults (CVR = 1.000), and 6) total antibacterial agents (CVR = 1.000). Based on the results of the first series, the second series was conducted, with response rates for each round of 58.3% (7/12) and 66.7% (8/12). Table 2 shows the antibiotic components of each classification according to the survey.

Table 1
Antibiotic classification in Korean hospitals according to a modified Delphi method

• Click for larger image
• Click for full table
• Download as Excel file

Table 2
Consensual definition of antibiotic components according to the antibiotic classification in Korean hospitals

• Click for larger image
• Click for full table
• Download as Excel file

This work comprises the first consensus study of an antibiotic classification system in Korea. Although the AWaRe classification of the World Health Organization has been widely adopted, too many antibiotics are included in each class, and the priority of antibiotics to be controlled is somewhat different from that in Korea: for instance, glycopeptide and carbapenem are considered as antibiotics with a higher risk for developing resistance than 4th generation cephalosporins.10 Therefore, we benchmarked the antibiotic classification of the NHSN and modified it in consideration of the importance and applicability to Korean hospitals.

As this was the first attempt to develop a nationwide antibiotic measuring system, antibiotic classifications that might be difficult to measure with the system were excluded. In fact, HIRA data, which are expected to be a resource of the system, do not provide information on the duration of antibiotic use, and therefore, antibiotic consumption could only be calculated with defined daily dose, not with days of therapy.11 For this reason, all items for children were excluded, despite the importance of antimicrobial stewardship programs in children.

Most panel experts agreed that the proper use of “antibacterial agents posing the highest risk for C. difficile infection” and “agents predominantly used for surgical site infection prophylaxis” should be improved; however, they were pessimistic about the applicability of such classification to the antibiotic measurement system. Many expressed that the classification “antibacterial agents posing the highest risk for C. difficile infection” might be unclear, because any antibiotic can predispose an individual to colonization by C. difficile and because there are no domestic data about high-risk antibiotics for C. difficile infection.12 Furthermore, they pointed out the fact that antibiotics used for surgical site infection prophylaxis are being frequently used to treat other infectious diseases and that accurate measurement of agents predominantly used for surgical site infection prophylaxis might not be feasible.

As with other Delphi processes, the study has some limitations. First, the composition of the expert panel was limited to some specialties. Even though infectious diseases specialists are key personnel in antimicrobial stewardship programs in Korean hospitals,13 the participation of other stakeholders should have been encouraged. Second, the surveys were administered online without face to face meetings, and this might have disturbed the communication among the panel. To minimize such limitations, we provided the mean scores for each question and the opinions expressed in the initial rounds of the survey in the following round. Third, antibiotics for children were excluded from classification due to limitations with the data sources.

Notwithstanding, our study not only lays the groundwork for developing an antibiotic measuring system in Korean hospitals, but also could help other researchers in performing studies on antibiotic consumption. To conclude, this study provides a means with which to classify antibiotic usage in Korean hospitals. This classification may guide the development of a system for measuring antibiotic usage in individual Korean hospitals. After additional research, further revisions might be necessary in the future.