Introduction

The problems associated with the irrational use of drugs and self-medication are international and have been reported to be widespread [11, 20]. The Lao People’s Democratic Republic (Laos) is no exception [12, 15–17]. Laos is one of the world’s poorest countries with a GDP per capita of about 310 USD per year. Health indicators are poor with an infant mortality rate of 82 per 1000 live births and a maternal mortality rate of 530 per 100,000 live births. Life expectancy is one of the lowest in the region – 57 years for men and 61 years for women [9].

Drug and Therapeutics Committees (DTCs) were introduced into a number of high-income countries decades ago as a means to improve the use of medicines, and they have gradually spread into low-income countries as well [14]. The establishment of DTCs has been reported to improve drug use and reduce costs in hospitals and associated health care facilities [4, 22]. The tasks of a DTC relate to all aspects of drug management, including developing locally adapted drug policies, evaluating and selecting drugs for the formulary list, developing, adapting and implementing standard treatment guidelines, assessing drug use to identify problems, conducting interventions to improve drug use, managing adverse drug reactions and rectifying medication errors [4]. The establishment of committees bearing the responsibility for carrying out these tasks has been encouraged and promoted in developing countries through a combination of training and follow-up support [3]. However, very little information is available on the effectiveness of DTCs in developing countries.

The hospital system in Laos is almost exclusively public with a few central hospitals and 18 provincial hospitals. Government expenditure on health is only about 4 USD per person per year, and out-of-pocket expenditure on drugs and hospital services is high.

Indicators for “Rational Use of Drugs” (RUD) and Standard Treatment Guidelines (STG) were developed within the Lao National Drug Policy Programme, and DTCs were established in all provincial hospitals in Laos during 1999–2000, with specified tasks (Appendix 1). Treatment indicator scores for three major diseases improved with the introduction of STGs and regular feedback interventions to the prescribers by members of the DTCs in eight provincial hospitals [21]. However, problems with sustainability were detected during supervisory visits [5].

The aims of this study were to determine factors in the working environment that relate to DTC performance and to conduct an educational intervention to improve DTC performance by means of feedback of indicator scores and interview information targeted toward DTC members.

Materials and methods

Study design and samples

This was a quasi-experimental (before and after) study (Fig. 1). Drug and Therapeutics Committees at central and provincial hospitals were included if they had been established at least one year before the start of the study and could be considered as functioning according to the guidelines set up for DTCs in Lao PDR by the Ministry of Health (MoH). Two central and seven provincial hospitals were thus recruited to the study, including the ninety-four committee members from these hospitals.

Fig. 1
figure 1

Process of intervention and data collection

Full size image

The study was performed in three phases during 2002. A pre-intervention phase with baseline data collection was followed by an intervention phase with three feedback sessions every other month and a third phase with data collection 3 months after the last feedback session. Data were also collected before the second and third intervention session.

Data collection tools

DTC indicators

Eight indicators were developed and tested by the research team as a means of reflecting most of the 13 tasks (Appendix 1) assigned to DTCs in Laotian hospitals. The indicators (Appendix 2) were aimed at assessing details bearing on the DTC’s structure (1), reporting system (2), activities (3), feedback to prescribers (4), drug information (5), adverse drug reaction control (6), monitoring of drug costs cost and consumption using DDD (defined daily dose) (7) and management of hospital medical routines (8) (details about this last indicator are not reported). Each component of the indicator was weighted (only positive weights) according to assessments agreed to by the researchers, comprising medical doctors, pharmacists and health-systems’ researchers. Each indicator could give a maximum score of 10. After training to ensure intra- and inter-rater reliability, data were collected by a field implementation team consisting of the four Laotian authors and three other experienced staff from the Ministry of Health (MoH; see Acknowledgements). For each hospital, two team members were assigned to perform all four data collections.

STG indicators

STG scores on the management of malaria, diarrhoea and pneumonia, the three most common infectious diseases in Laos, were collected using the STG indicator charts developed by the MoH [21]. The indicators reflect information on the diagnosis, treatment and follow-up of these diseases and used information that had been stored in the hospital record-keeping system. A task force comprising DTC members collected data on 30 cases for each disease (encompassing the Departments of Infectious Diseases, Internal Medicine and Paediatrics). For easy comparison the maximum score was 10. These data had already been routinely collected before the start of the study.

RUD indicators

Indicators related to the RUD had previously been developed by the MoH [5] based on World Health Organization (WHO) indicators and had also been routinely collected for some time by the DTCs. The maximum score was 10.

Self-administered questionnaire

To identify factors which could impact on DTC performance, the DTC members were interviewed before and after the intervention using a self-administered questionnaire [1] presented by the field implementation team. The informants were asked about their views on predefined factors that may influence, prevent or facilitate a successful performance of their DTC. They could also give open comments. All eligible DTC members responded in both the pre-intervention period (n=90) and in the post-intervention period (n=94). Data were analysed in detail to explore both the factors that were perceived to impact on the performance of the DTCs and the informants’ suggestions for better DTC performance.

Intervention

Feedback was based on the indicator scores (DTC, STG, RUD) recorded at baseline and at two consecutive times of data collection. The main aim of the feedback sessions was to encourage DTCs to perform the 13 tasks evaluated in an appropriate manner. The members of the implementation team assisted the DTCs in planning the feedback sessions, which were conducted after each of the three data collection periods to ensure the continued influence of the evaluations. The Head of the DTC at each hospital led the meeting by introducing the session and stimulating the discussion. The DTC task force first presented the main messages of the STG and RUD scores and subsequently highlighted those aspects which could be improved. The intention of the DTC task force was that these messages should subsequently be discussed with prescribers at the different departments in the hospital. Following the presentation, two of the members of the implementation team presented the DTC indicator scores that had been recorded and suggested areas where improvement could be considered. These suggestions were discussed among the DTC members before any decisions were taken. The feedback sessions were performed in a similar manner in all hospitals.

Data analysis

All data from the scoring of indicators were entered and analysed using the EPI INFO and SPSS (SPSS Inc, Chicago, Ill.) statistical programmes. The Pearson chi-squared test was used to evaluate the potential association of DTC members’ tasks and their performance. The one-way analysis of variance test was used to assess the degree of statistical significance of the difference in mean scores of the DTC, STG and RUD indicators collected during the four rounds of data collection. The Scheffe test was used to assess the statistical significance between two groups of means in post hoc comparisons.

Results

Performance of DTC Units

The successive changes in the overall mean scores recorded in the four periods of data collection are shown in Table 1. Following the educational feedback sessions attended by the DTC members, the total mean score of all DTC indicators for all hospitals improved significantly – from a pre-intervention score of 4.7 to a post-intervention score of 6.9 (p<0.001). The indicators for activity, feedback to prescribers and drug information in particular showed significant improvement, with high scores in the post-intervention period. The indicator on DTC structure was above nine already at baseline, while the indicator for the reporting system was lower due to the discovery that meeting summaries were not being sent to DTC members.

Table 1 Means of DTC (Drug and Therapeutics Committees), STG (Standard Treatment Guidelines) and RUD (rational use of drugs) scores for the nine hospital DTCs recorded in the four rounds of data collections
Full size table

The mean STG score also improved significantly after the intervention (7.4–8.2, p<0.01), while no significant change was detected in the mean RUD indicator score. The absence of any obvious improvement in the latter could partly be due to quite a high mean score at baseline (7.7) and to the fact that the indicator on use of traditional medicines was zero or very low for all hospitals.

None of the DTC units monitored adverse drug reactions (ADR) or side effects (SE) during the pre-intervention period (Table 2). There was a significant improvement in the mean ADR score compared with baseline (p<0.01), but the score recorded in the post-intervention was still low. However, during the course of the study four of the DTCs assigned DTC members to be responsible for ADR/SE activity and subsequently scored between 5.5 and 7, while the other DTCs had zero scores. Drug cost monitoring was not effective in four of the hospitals (Table 2), while three DTCs performed well (score: 8–10) and two moderately well (score: 5).

Table 2 DTC, STG and RUD scores for the nine DTCsa preceding and following the intervention
Full size table

Views of DTC Members

Of the DTC members 64% held the position of Head of Department or Hospital Director. Four out of five DTC members had been recruited to the permanent DTC task forces, and more than one-half of them were Heads of Department.

When DTC members were asked about the number of tasks they had to perform daily due to the projects they were involved in or positions that they held, 70% stated that they had more than three tasks to perform. A strong association was detected between the positions held by these DTC members and their daily task load (p=0.001) (Table 3). The data indicate that the current DTC members are overloaded and that they do not have enough time to perform their DTC tasks appropriately.

Table 3 Associationa between positions held by DTC members and number of tasks (n=94)
Full size table

After the intervention, many DTC members stated that any poor DTC performance could be due to the irregularity at which DTC meetings were held (60%), the many positions held by DTC members (80%) or a lack of interest shown by the Board of Hospital Directors for DTC activities (48%). Additional proposed causes of poor DTC performance were insufficient knowledge of the DTC members (66%) and the loss of trained DTC members to other positions/places (63%).

When DTC members were asked about how DTC activities can be improved, many of the DTC members suggested training DTC members specifically for carrying out DTC activities (77%), recruiting more dynamic DTC leaders (72%) and improving the reporting system (56%) and meeting techniques (48%). Only a few suggested allocating funds for DTC activities (13%).

Discussion

There is a surprising lack of well-designed studies aimed at assessing the effectiveness of DTCs internationally. In our study, the overall performance of the nine DTCs in Lao hospitals improved significantly after three feedback sessions despite the fact that DTC members were overloaded with other work tasks. In particular, the indicators on DTC activity, feedback to prescribers and drug information showed major performance improvement. In contrast, the monitoring of adverse drug reactions and drug consumption showed an improvement in only four hospitals. The scores for the management of major infectious diseases in relation to standard treatment guidelines also improved significantly during the same period. Previous studies conducted on methods to improve prescribing and to solve problems associated with prescribing [8, 21] showed that guidelines combined with feedback audit is one way to support high-quality clinical practice [1, 18].

An overall low mean DTC indicator score (below half of maximum) was detected during the pre-intervention period. Our findings suggest that the inefficient performance recorded in the pre-intervention period might have been due to DTC members being overloaded with other work activities such that meetings and STG and RUD feedback sessions were held on irregular bases and the meetings were poorly attended by the DTC members. Several of the current Lao DTC task force members are busy Heads of Department and as such are probably more useful as prominent members of their profession in providing a one-on-one feedback to members of their medical staffs as a means of improving the RUD and STG indicator scores than as DTC task force members. Improvement can further be achieved by motivating staff through feedback and by reducing barriers, such as a lack of time, in order to improve efficiency [10]. Awareness of the quality of one’s work could be one such motivating factor.

Another possible reason for the sub-optimal DTC performance may be that the Board of Directors of these hospitals do not understand the role of the DTC and, consequently, may not acknowledge its importance and use it in their daily hospital management. As such, they have not stimulated DTC activities adequately as required. In addition, it may be due to a lack of regular supervision of DTC activities from the MoH. According to recommendations from WHO, the organizational development and performance of the DTC should be monitored continuously and be documented using indicators that are considered to be core parameters of its goals and objectives [3, 4].

During the intervention, four DTCs started to organize ADR activities related to the monitoring of ADR control. However, the score was still zero in the other five hospitals. Relying on physicians’ reporting has previously met with little success [7]. The management of ADRs should include prompt feedback of individual patients and grouped data for educational and risk management purposes. Risk managers should play a significant role in all aspects of a surveillance programme [2, 6, 13].

One limitation of this study is that the context – with only nine functioning DTCs – did not allow for a control group. Expectancy and attention effects must therefore be taken into consideration when the outcome is analysed. However, the DTC members had already routinely collected data for the RUD and STG indicators, and there seemed to be no obvious reason why they should change their ways of data collection because of the study. These indicators show the performance of the doctors in the departments and cannot directly be influenced by the DTC members. The specific DTC indicators, however, were collected by the implementation team during this study, which might introduce a bias although the repeated data collections probably reduce this bias to some extent. While not rigorously evaluated, the fact that the same indicators were used during all data collection strengthens the validity of the comparison between the scores before and after the intervention.

The first intervention seems to have had the greatest impact on the scores. This could be due to an initial attention effect. However, this higher level was maintained or improved upon despite the more common observation that initial improvements are often short-lived. The repeated feedback sessions have probably contributed to this sustained effect. There were no direct economic incentives linked to the implementation.

There is a global interest in finding indicators that can be used on a routine basis to monitor DTC performance. It is essential that self-assessment of this kind is performed with well-defined indicators in order to maintain the reliability of scorings. The indicators used in this study can be collected by the DTC member themselves as a means of carrying out an assessment of the performance in relation to identified tasks (standards). The DTC can thereby play an important role in the cycle of ensuring the quality of therapeutic care [4]. The costs for this monitoring can be allocated within existing budgets if the periods for collecting data for the different indicators are spaced conveniently. Decision-makers in Laos have previously shown interest in using research evidence of this kind as one basis for policy-making [19].

This study has demonstrated that monitoring of the performance by means of a set of indicators improved DTC work in several areas, including disease management, when feedback was given using an audit cycle approach. The development of methods to study DTC performance is recommended in all countries, regardless of income level.