Development and validation of the ACE tool: assessing medical trainees’ competency in evidence based medicine

Background While a variety of instruments have been developed to assess knowledge and skills in evidence based medicine (EBM), few assess all aspects of EBM - including knowledge, skills attitudes and behaviour - or have been psychometrically evaluated. The aim of this study was to develop and validate an instrument that evaluates medical trainees’ competency in EBM across knowledge, skills and attitude. Methods The ‘Assessing Competency in EBM’ (ACE) tool was developed by the authors, with content and face validity assessed by expert opinion. A cross-sectional sample of 342 medical trainees representing ‘novice’, ‘intermediate’ and ‘advanced’ EBM trainees were recruited to complete the ACE tool. Construct validity, item difficulty, internal reliability and item discrimination were analysed. Results We recruited 98 EBM-novice, 108 EBM-intermediate and 136 EBM-advanced participants. A statistically significant difference in the total ACE score was observed and corresponded to the level of training: on a 0-15-point test, the mean ACE scores were 8.6 for EBM-novice; 9.5 for EBM-intermediate; and 10.4 for EBM-advanced (p < 0.0001). Individual item discrimination was excellent (Item Discrimination Index ranging from 0.37 to 0.84), with internal reliability consistent across all but three items (Item Total Correlations were all positive ranging from 0.14 to 0.20). Conclusion The 15-item ACE tool is a reliable and valid instrument to assess medical trainees’ competency in EBM. The ACE tool provides a novel assessment that measures user performance across the four main steps of EBM. To provide a complete suite of instruments to assess EBM competency across various patient scenarios, future refinement of the ACE instrument should include further scenarios across harm, diagnosis and prognosis.


Patient scenario
"Jane is a 42 year-old female Caucasian, who lives with her partner in metropolitan Melbourne, Australia. Jane is a lawyer, who quit smoking three years ago, after being a 'pack-a-day' smoker since her early 20s. Since her late 30s, Jane has received treatment for hypertension. Her medical history is otherwise unremarkable. At her most recent visit to her family doctor, Jane mentions that she has seen reports on the television about a new study investigating the preventive effects of aspirin. She has heard that aspirin may be beneficial in protecting against cardiovascular disease. Jane wonders whether she should be taking aspirin, given her history with hypertension, but wonders whether also being a diabetic might negate any benefit."

Clinical Question
"Is aspirin effective in reducing the risk of cardiovascular disease?" Limit to randomised controlled trials 75

A randomised controlled trial of aspirin for the prevention of cardiovascular disease Background
Aspirin is effective in the treatment of acute myocardial infarction and prevention of cardiovascular disease in men and women. Previous studies on the use of aspirin in primary prevention of cardiovascular disease have demonstrated a positive effect in men, yet the benefit in women remains uncertain. The aim of this study was to assess the effect of aspirin in the prevention of cardiovascular disease in women.

Methods
The study design was a randomised, double-blinded, placebo-controlled, trial of low-dose aspirin in the prevention of cardiovascular disease in women. The design of the study has previously been described in detail. In brief, between January 2002 and January 2012, letters of invitation were mailed to 500,000 women in the greater city of Melbourne, Victoria, Australia. A total of 63,250 volunteered to enrol in the study. Women were eligible if they were 40 years of age or older; had no history of coronary heart disease, cerebrovascular disease, no previous side-effects to taking aspirin and were not currently taking aspirin or any non-steroidal anti-inflammatory drug (NSAID) medication. A total of 31,150 women met the inclusion criteria of which 15,100 were randomised (through the generation of a computer generated scheme) to receive aspirin and 15,102 were randomised to receive the placebo. Written informed consent was obtained from all participants prior to commencement in the study. The trial was approved by the ethics board at the governing hospital and university institution. Participants in both groups were required to present every 6 months at the study site centre for assessment and to receive their medication. Medication was provided by the site pharmacy, which allocated identical appearing aspirin and placebo tablet in blister packs to the study's participants independent to the study's investigators. All participants were followed for myocardial infarction, stroke or death from cardiovascular causes. Medical records were obtained for all women in whom a cardiovascular event was recorded. These records were reviewed by an end-point committee, consisting of study investigators blinded to the treatment. The primary end point was cardiovascular events -a combination of myocardial infarction, stroke or death from cardiovascular causes. Only confirmed end-points of cardiovascular events were included in this study. Cox proportional hazard models were used to calculate hazard ratios and 95% confidence intervals for the comparison of event rates in the aspirin and placebo groups after adjustment for age.

Results
Both aspirin and placebo groups were similar with respect to baseline characteristics ( Table 1). The average duration of follow-up from randomisation to the end of the trial was 4.2 years (range, 2.3 to 5.0 years). Throughout the duration of the trial, drop-outs occurred. Data presented is based on participants that completed the trial during the study period. A total of 422 women in the aspirin group and 478 women in the placebo group had a cardiovascular event (Hazard Ratio, 0.83; 95% confidence interval, 0.77 to 1.01). There was no evidence that any of the cardiovascular risk factors considered, except smoking status and hyperlipidemia, modified the effect of aspirin on the primary end-point.

Discussion
In this large study, involving 63,250 women, a 100 mg daily dose of prophylactic aspirin is associated with a reduced risk of major cardiovascular events. No significant evidence was found that age, hypertension, diabetes or BMI modified the effect of aspirin. Middle aged women who adhere to a daily low dose of aspirin can significantly reduce the risk of cardiovascular disease. The rate of benefit is large, with a cardiovascular event prevented for every 269 women treated with aspirin.