Evidence in medicine: math versus biology!

The drive for optimal clinical decisions based on “best” evidence has gained significant momentum in the last few decades. In parallel with the evidence-based medicine approach, various “hierarchy of evidence” stratifications have also emerged1–3. Overall, those stratifications attempt to characterize underlying bias (that is, validity) and emphasize the relative quality of different forms of evidence. In a traditional pyramidal representation (Figure 1)2,3, systematic reviews and meta-analyses of randomized clinical trials (rcts) represent the pinnacle of the pyramid, followed by individual rcts and observational studies (for example, case–control or cohort studies). Conversely, case reports and series, often called “anecdotal evidence,” are placed at the bottom of the pyramid. Notwithstanding a number of limitations and criticisms4, including an overreliance on systematic reviews and meta-analyses, clinical decision-making continues to be guided by such “hierarchies of evidence.” Increasingly, however, the evidence-based medicine approach—and thus clinical decision-making—is coming to rely on mathematical or statistical characterizations of biologic processes (for example, disease progression or relapse, patient survival, and quality of life). The applications of math and statistics in clinical medicine are numerous and include characterizations of biologic observations and mathematical modelling of health outcomes5. As examples of the former, the biologic observations encountered in rcts (such as disease progression or relapse, patient survival, and quality of life) are often represented by bio-statistical measures that include estimates of effect size and the uncertainties involved (for example, type i and ii errors, p values, and 95% confidence intervals)6,7. As well, meta-analyses often use mathematical measures that extrapolate efficacy from individual rcts to compute compound measures of effect size and uncertainty. Not surprisingly, for clinicians who chose a science path driven by human biology, the underlying math and statistics are not infrequently perceived as complex. Nonetheless, a working knowledge of the underlying statistical methods is necessary for clinicians to properly interpret the current evidence. Furthermore, knowledge that is more in-depth is also often required to properly interpret the clinical evidence and avoid the pitfalls of mere binary interpretation of study outcomes as simply positive or negative6,7. Clinical evidence that stems from biologic observations cannot address all pertinent gaps in knowledge. As an


Evidence in medicine: math versus biology!
T. Younis mbbch,* † M. Thana md,* and C. Skedgel mde phd ‡ § The drive for optimal clinical decisions based on "best" evidence has gained significant momentum in the last few decades. In parallel with the evidence-based medicine approach, various "hierarchy of evidence" stratifications have also emerged [1][2][3] . Overall, those stratifications attempt to characterize underlying bias (that is, validity) and emphasize the relative quality of different forms of evidence. In a traditional pyramidal representation ( Figure 1) 2,3 , systematic reviews and meta-analyses of randomized clinical trials (rcts) represent the pinnacle of the pyramid, followed by individual rcts and observational studies (for example, case-control or cohort studies). Conversely, case reports and series, often called "anecdotal evidence," are placed at the bottom of the pyramid. Notwithstanding a number of limitations and criticisms 4 , including an overreliance on systematic reviews and meta-analyses, clinical decision-making continues to be guided by such "hierarchies of evidence." Increasingly, however, the evidence-based medicine approach-and thus clinical decision-making-is coming to rely on mathematical or statistical characterizations of biologic processes (for example, disease progression or relapse, patient survival, and quality of life).
The applications of math and statistics in clinical medicine are numerous and include characterizations of biologic observations and mathematical modelling of health outcomes 5 . As examples of the former, the biologic observations encountered in rcts (such as disease progression or relapse, patient survival, and quality of life) are often represented by bio-statistical measures that include estimates of effect size and the uncertainties involved (for example, type i and ii errors, p values, and 95% confidence intervals) 6,7 . As well, meta-analyses often use mathematical measures that extrapolate efficacy from individual rcts to compute compound measures of effect size and uncertainty. Not surprisingly, for clinicians who chose a science path driven by human biology, the underlying math and statistics are not infrequently perceived as complex. Nonetheless, a working knowledge of the underlying statistical methods is necessary for clinicians to properly interpret the current evidence. Furthermore, knowledge that is more in-depth is also often required to properly interpret the clinical evidence and avoid the pitfalls of mere binary interpretation of study outcomes as simply positive or negative 6,7 .
Clinical evidence that stems from biologic observations cannot address all pertinent gaps in knowledge. As an example, observational studies and rcts cannot in themselves typically examine all the potential lifetime clinical and economic effects of cancer screening strategies that involve a variety of modalities or time intervals in various at-risk population-based cohorts. Mathematical models such as decision-analysis frameworks are particularly helpful when traditional clinical study designs are neither feasible nor practical 5 . A working knowledge of the building blocks underlying such models is therefore warranted.
All mathematical models incorporate a variety of input parameters within a framework structure to compute relevant outcomes (Figure 2). The underlying frameworks can range from relatively simple decision trees to more complex state-transition or Markov models and to even more complex microsimulation models and discrete event simulations, depending on the nature of the process to be modelled. The input parameters (clinical or economic, or both) are often derived from various sources of observational data (for example, rcts and records of medical resource use), but can also include estimates based on assumptions and expert opinion. Model outputs typically include clinical and economic outcomes such as incidence or prevalence, survival, quality of life, life-years, quality-adjusted life-years, costs, and cost-effectiveness. The effects of the uncertainties in input parameters on model outcomes are usually examined through one-way sensitivity analyses and probabilistic sensitivity analyses. The former examine the effect of each parameter independently; the latter simultaneously incorporate all parameter uncertainties so as to understand how interactions in parameter uncertainty could affect model outcomes. Models are also often subjected to a number of validation processes (for example, face, internal, and external validity) in an attempt to reduce bias.
A critical appraisal of the evidence stemming from models in medicine is essential 5 . Indeed, a "hierarchy of evidence" approach ( Figure 1) for economic and decision analyses does exist 3 . As an example, within the levels of evidence set out by the Oxford Centre for Evidence-Based Medicine 3 , an analysis incorporating clinically sensible costs or alternatives, one or more systematic reviews of the evidence, and multiway sensitivity analyses would be placed higher than another analysis incorporating limited or poor-quality inputs, no systematic review or reviews of the evidence, and only limited or one-way sensitivity analyses. An analysis that does not incorporate sensitivity analyses would rank even lower, and evidence based on "expert opinion" alone would constitute the lowest level of evidence. Conversely, a properly conducted systematic review of multiple economic analyses would often represent the highest level of economic evidence. Other approaches for the critical appraisal of model-based evidence are also available, including scoring systems for economic analyses 8 and guidelines for good modelling practices 5 and for the conduct of systematic reviews for economic evaluations 9 . Evidence in medicine that is based on mathematical modelling therefore varies with respect to its strengths and should affect clinical decisions only accordingly.
In this issue of Current Oncology, Gauvreau and colleagues 10 highlight Canada's OncoSim platform-a Web-based suite of cancer-specific microsimulation models designed to augment conventional resources for population-level decision-making in Canada. OncoSim currently comprises three in-depth cancer models (lung, colorectal, cervical), which are built on a common framework, with another in-depth model (breast cancer) and a generalized model (25 cancers) in development. The microsimulation platform is led and supported by the Canadian Partnership Against Cancer, with development by Statistics Canada and funding by Health Canada. The OncoSim platform is freely available online, with user capabilities that allow for customization of a number of input parameters and model outputs. In OncoSim, the input parameters are derived from various Canadian sources, with model validation (that is, face, internal, and external validity) and fit against observed data (calibration). Its outputs can be customized to include cancer burden and outcomes (for example, incidence, mortality, life expectancy, and quality-adjusted life-years) as well as health care resources and economics (for example, procedures or treatments, health care costs, and cost-effectiveness). One-way sensitivity analyses can currently be conducted, and the future plan is to incorporate multiway probabilistic sensitivity analyses.
Notwithstanding the limitations inherent in all microsimulation models, OncoSim is a powerful Webbased tool that projects the future burden of cancer and its economic effects in Canada and that allows for a critical assessment of the benefits and costs associated with various potential population-based cancer control interventions. Indeed, OncoSim has already been used to support a number of clinical decisions (for example, colorectal and lung cancer screening guidelines prepared by the Canadian Task Force on Preventive Health Care) and reports (for example, the system performance reports produced by the Canadian Partnership Against Cancer) at the national and jurisdictional levels.
In an era dominated by evidence-based medicine, applications of math to characterize or model biologic observations and processes are inevitable and cannot be avoided or dismissed as "black boxes." As health care providers, we have come a long way in better understanding the biostatistics of observational and experimental studies. We now must also better understand-and scrutinize-the mathematical modelling platforms that have become an integral component of today's evidence in medicine.