Commentary: We still need observational studies of drugs—they just need to be better

The paper by de Vries and colleagues examined different observational study designs used to assess the effects of statin use and demonstrates three key points. First, the need for accurate matching on key prognostic variables—in this case age. Second, the need to consider the underlying biology when designing epidemiological studies. Third, they suggest that a case–control study nested within a selected group of people may be more prone to bias than a population-based study. However there may be a bigger issue: both the observational designs found a substantial protective effect, while robust data from randomized trials suggests there is no effect. Does this mean we should do away with observational studies of drugs?

Clinically important but unexpected adverse effects of drugs are often too rare to be detected in randomized trials. Spontaneous reports of possible adverse effects are often sufficient to lead to withdrawal of a drug from the market.¹ High quality adequately powered observational studies would provide a much better evidence base for such decisions. Indeed, observational studies to assess possible adverse events of currently used drugs are now required under guidance issued by the European Medicines Agency.² The importance of adverse effects is clear. However, unexpected beneficial effects (such as statins preventing fractures) could also be of great potential benefit to patients. Most drugs affect many different cells tissues and organs, but during drug development interest is largely focused on a therapeutic effect in a single organ system. Studies encompassing all the likely beneficial and adverse effects of a drug can help inform prescribing decisions. Computerized clinical data offer an attractive way to study unexpected drug effects in a large, representative population. However, the paper by de Vries demonstrates some of the possible problems that can arise when using such data.

A major problem is that data on potential confounding variables may be incomplete or not available at all. For example, as noted by de Vries, UK primary care databases have not previously included measures of socioeconomic status. Individual anonymized postcode linkage to small area measures of social deprivation is now underway or completed in a number of primary care databases.^3–5 In addition, data completeness on factors such as smoking and body mass index is improving all the time. In the GPRD data used by de Vries, smoking status was missing on ∼40% of people. However, in the more recent data, >90% of adults have their smoking status recorded.

Progress on measurable confounding factors is welcome, but ‘invisible’ or unmeasurable factors, such as frailty, may be even more important in determining who is exposed to a drug. The decision to prescribe a drug for a particular patient is inextricably bound up with their health. Strong selection biases are likely to operate, whereby people who are exposed to a particular drug are systematically different to people who are either not exposed at all or are exposed to an alternative drug, variously called confounding by indication or ‘channelling bias.’⁶ There are two different forms of such selection bias. First, the disease that prompted the decision to treat a patient may itself increase the risk of the outcome of interest. Second, a patient's perceived risk of the outcome may influence the choice of drug. The example of antidepressants and the risk of hip fracture illustrates these potential biases. The use of tricyclic antidepressant (TCA) drugs has been associated with hip fracture, possibly due to an increased tendency to fall as a result of orthostatic hypotension, sedation, and/or confusion.⁷ However, this increased risk may be partly owing to people who are depressed being at increased risk of fracture. The newer selective serotonin reuptake inhibitor (SSRI) antidepressants are less sedating and have fewer anti-cholinergic side effects and, thus, might be expected to cause fewer problems with falls and fractures. However, a large case–control study surprisingly found the risk of hip fracture associated with use of SSRIs was actually higher than that associated with TCAs⁸ (Table 1). Such a result could be partly explained by doctors preferentially prescribing SSRIs rather than TCAs among frail patients at greater risk of falling. To investigate and try to overcome such selection bias, Hubbard et al.⁹ undertook both a classical case–control analysis followed by a self-controlled case-series analysis using data from the GPRD. In the self-controlled case-series, the incidence of an outcome during predefined risk periods following the initiation of exposure is compared within person to the incidence during baseline non-risk time periods, thus eliminating between person confounding (Figure 1).^10,11 Unlike statins and hip fracture, in the case of antidepressants, the hypothesis was that the risk was increased immediately following initiation of the drug and then fell gradually. This study showed two things. First that while the use of antidepressants was associated with an increased risk of hip fracture, the risk had previously been over-estimated. Second, the increased risk was similar for TCAs and SSRIs. The innovative use of the case-series design, thus, managed to largely overcome the problem of selection bias. Another alternative design to overcome such selection biases is a matched cohort, in which people are matched on their propensity (or likelihood) to be prescribed a drug.¹²

Figure 1

Open in new tabDownload slide

Pictorial representation of case-series analysis⁹

Difficult to measure selection biases may well explain the discrepancy between randomized trials and observational studies in the case of statins and hip fracture. People who opt for statins may be more health conscious and, thus, be, on average, more active or eat healthier diets, reducing their risk of hip fracture. Alternatively, some physicians, seeing statins as an investment in health a long time in the future, may choose not to prescribe statins to patients they perceive as frail or as having a relatively limited life expectancy. Again, this would produce an exposed group with a lower risk of fracture. Overcoming such selection biases and the confounding they produce present major challenges to epidemiologists. We will go on needing observational studies to assess the effects of drugs. Innovative design strategies may be needed to produce reliable answers.

References