With the IARF of each hypothetical case, the aspects that cause the most errors in the Pharmacotherapy Review process were verified. Three aspects were selected to be analyzed: the identification of intentional errors, the adequate adjustment of medication usage schedule and the final classification of the VEs, which established whether the review and scheduling are considered adequate, partially adequate or inadequate. The results for each aspect evaluated are presented in Table 3:
Table 3
Revisor | Identification of intentional erros (%) | Proper schedule adjustment (%) | Final classification |
Adequate (%) | Partially Adequate (%) | Inadequate (%) |
Specialist 1 | 20.0 | 40.0 | 65.0 | 10.0 | 25.0 |
Specialist 2 | 40.0 | 80.0 | 73.0 | 7.0 | 20.0 |
Specialist 3 | 30.0 | 60.0 | 62.0 | 10.0 | 28.0 |
Specialist 4 | 50.0 | 70.0 | 69.0 | 9.0 | 22.0 |
Specialist 5 | 70.0 | 80.0 | 75.0 | 11.0 | 14.0 |
Electronic system | 80.0 | 100.0 | 88.0 | 6.0 | 6.0 |
In these three aspects of evaluation, the system obtained a greater number of correct answers as compared to those of the pharmaceutical experts. The identification of intentional errors by the system was not well evaluated in the hypothetical case where a drug that is not indicated for the patient's clinical condition appears in the prescription. The reason is due to the fact that the system does not automatically remove the drug from the prescription it only warns and leaves the decision to the user. The VEs considered that an observation should be included in the schedule, informing if a drug is contraindicated, but decided not to proceed. The other case was the prescription of the drug “alendronate sodium”: the VEs considered that the directions on the use of this drug should be more detailed when related to the schedule and include information based on the Ministry of Health clinical protocol16. For these two situations, adjustments in the registration of medications would resolve the negative assessment of the review by the system.
Regarding the schedules adjustment, those set up by the system were the only ones that were considered adequate in 100% of the reviews. Therefore, the selection of GA as the main system intelligence resource proved to be efficient in adjusting the medications use schedule. Optimization problems, in general, have a finite space of solutions and restrictions for solutions to be considered about adequate. In order to carry out the scheduling with the selection of the most adequate times for the medicine use, the finite space for solutions is constituted by the hours of the day and the medicines that need to be scheduled. The restrictions are the patient's routine impositions and the interactions between medications and between medications and food that sometimes end up causing the patient to abandon treatment17. In some cases, there are a lot of restrictions, which would prevent other algorithms from finding a solution. The GAs, even without finding an optimal solution, show the best solution found given the restrictions related to the patient's routines.
Finally, in the last classification of the pharmacotherapy review, the reviews made by the system were those that obtained the best ratings, being considered adequate (88%). The pharmaceutical expert who came closest to this rate obtained 75% of the reviews considered adequate.
In an attempt to establish agreement between the reviews carried out by the pharmaceutical experts and the reviews carried out by the system, the indices presented in Table 4 were obtained. The reviews carried out by the pharmaceutical experts were called A1, A2, A3, A4 and A5 and the review performed by the system was called A6.
Table 4
Agreement rates between pharmaceutics experts and electronic system among pharmacotherapy review.
| Weighted Kappa | CI 95% |
A6 - A1 | 0.3547 | (0.1926–0.5168) |
A6 - A2 | 0.3719 | (0.1823–0.5615) |
A6 - A3 | 0.3458 | (0.1907–0.501) |
A6 - A4 | 0.3971 | (0.2241–0.5701) |
A6 - A5 | 0.4790 | (0.2759–0.6822) |
CI = Confidence Level |
The agreement rate obtained was considered reasonable (0.21–0.40). The highest agreement occurred with the A5 schedule, and was considered moderate (0.41–0.60). The initial hypothesis was that the system was capable of reproducing the reasoning of the pharmaceutical experts and setting the schedules with the same quality standard. However, the results obtained show that the system surpassed the schedule quality performed by human specialists, mainly in the identification of prescription errors and adequacy of schedules in connection with the patients' routine. The low levels of agreement are then interpreted as positive, since the revisions carried out by the system were better evaluated qualitatively.
The number of pharmaceutical experts who set the schedules to be compared with the schedules set by the system (5) and the number of simulated hypothetical cases (10) are not sufficient to establish a statistically significant sample, which was a limitation of this study. However, it was possible to verify that the system was able to set schedules with a standard of equal quality and even superior to those of the experts who participated in the investigation. This according to the evaluation carried out by the VEs.
Establishing the system's reliability index would be a feature that could be used to validate the system for reproducibility, that is, to validate multiple schedules of the same hypothetical case, performed by the system, at different times, and verify that all these schedules would be classified as "adequate". This analysis was not performed because the only way to assess whether the schedule is adequate or not, in the methodological parameters that were used, would be through the validation performed by the VEs in the consensus meetings, which would demand an excessive time for analysis thus making the process unfeasible.
It is noteworthy that the system only obtains good results if the registrations are carefully performed. The intelligent reasoning provided by GA will only work if the database has the correct relationships, including with regard to writing patterns and the qualities of observations. The cases in which the system did not obtain the expected result included situations in which the adjustment in the medication record would have cured the fault indicated by the EVs. In the pharmaceutical care settings where medications are dispensed, the main difficulties found for the pharmacotherapy review service to be fully carried out are the lack of quality information and the excessive time required to look for prescription potential inconsistencies and for suggesting a drug use strategy considering the patient's routine.
Failure to carry out the review of pharmacotherapy as recommended by the WHO has a direct impact on the RUM promotion, since the pharmacist, through a set of actions called pharmaceutical care (PC), which includes pharmacotherapy review, is an extremely important agent in promoting the RUM, interacting between the prescriber and the drug end user.
The fast search for information can be performed using an information system; however, in addition to having the information available, it is necessary to handle it, to establish the adjustment of doses and schedule consistent with the patient's routine; hence, this system can be used as a resource to support decisions, with the final decision always being the responsibility and the technical competence of human beings.