ReviewMathematical models in the evaluation of health programmes
Introduction
Although policy decisions in public health would ideally be based on evaluations that measure effect directly, modelling does, and often should, play a major part in large-scale evaluations. The purpose of our Review is to assist readers to assess critically and interpret appropriately the results of such modelling exercises. We first present practical and theoretical reasons why models are and should be used in large-scale evaluations. We then summarise the types of different modelling approaches and discuss how models and their outputs should be judged.
Section snippets
The role of models in programme evaluations
Public health programmes need to be evaluated on whether anticipated benefits are indeed happening and whether they are cost effective.1, 2 For many programmes, a randomised controlled trial is not an option for ethical or practical reasons (eg, programme implementation is done by large groups, such as national health systems, which cannot be randomised). Alternatively, the evaluation question might not be whether an intervention is effective, but whether it is being successfully implemented.
Mathematical modelling methods
Mathematics provides a precise quantitative language to describe the relation between variables and changes in states, and in medicine we can represent mathematically the clinical course of disease, the distribution of disease across populations and over time, and the mechanisms that generate disease.28, 29 The development of a model of infection, disease, or death requires us to precisely set out our assumptions about the parameters and processes influencing health, and enables us to calculate
Assessing models
Panel 1 provides a summary checklist of items that should accompany the most rigorous model analyses. None of these indicators of model quality guarantees that the model produces accurate outputs, but the presence of this information will help readers assess the appropriateness of the model.
It is essential to have a clear understanding of the model's structure to judge model results and outputs. We believe that in addition to the full technical description of the model, there should also be a
Discussion
With recent increases in resources committed to improve global health, there is growing demand for accountability and efficiency in programme implementation and the need for good evaluation. In many situations, available data and appropriate modelling techniques can clarify, within a causal framework, the relation between programme inputs and effect. Modelling exercises that are well done can provide credible evidence of the value of programmes and guide the roll out and improvement of
References (71)
- et al.
Assessing evidence for behaviour change affecting the course of HIV epidemics: a new mathematical modelling approach and application to data from Zimbabwe
Epidemics
(2009) - et al.
Estimation of the number of people incubating variant CJD
Lancet
(1998) - et al.
Child survival gains in Tanzania: analysis of data from demographic and health surveys
Lancet
(2008) - et al.
Estimates of maternal mortality worldwide between 1990 and 2005: an assessment of available data
Lancet
(2007) - et al.
Public-health impact of accelerated measles control in the WHO African Region 2000–03
Lancet
(2005) - et al.
Estimating the global burden of HIV/AIDS: what do we really know about the HIV pandemic?
Lancet
(2004) - et al.
Chapter 18: public health policy for cervical cancer prevention: the role of decision science, economic evaluation, and mathematical modeling
Vaccine
(2006) - et al.
Behavioural strategies to reduce HIV transmission: how to make them work better
Lancet
(2008) - et al.
The history and challenge of HIV prevention
Lancet
(2008) - et al.
Transmission dynamic modelling of the impact of human papillomavirus vaccination in the United Kingdom
Vaccine
(2010)
Universal voluntary HIV testing with immediate antiretroviral therapy as a strategy for elimination of HIV transmission: a mathematical model
Lancet
ONCHOSIM: a model and computer simulation program for the transmission and control of onchocerciasis
Comput Methods Programs Biomed
Latin hypercube sampling and the sensitivity analysis of a Monte Carlo epidemic model
Int J Biomed Comput
Modeling and analysing HIV transmission: the effect of contact patterns
Math Biosci
HPV-16 infections and cervical cancer: modeling the influence of duration of infection and precancerous lesions
Epidemics
Saving the lives of South Africa's mothers, babies, and children: can the health system deliver?
Lancet
Appropriate evaluation of HIV prevention interventions: from experiment to full-scale implementation
Sex Transm Infect
Evidence-based public health: moving beyond randomized trials
Am J Public Health
Evaluating large-scale HIV prevention interventions: study design for an integrated mathematical modelling approach
Sex Transm Infect
The five-year evaluation of the global fund to fight aids, tuberculosis, and malaria—synthesis of study areas 1, 2 and 3: March 2009
How to pay “cash-on-delivery” for hiv infections averted: two measurement approaches and ten payout functions—working paper 210
Statistical inference in a stochastic epidemic SEIR model with control intervention: ebola as a case study
Biometrics
The dynamics of measles in sub-Saharan Africa
Nature
The global burden of measles in the year 2000—a model that uses country-specific indicators
J Infect Dis
Design and interpretation of vaccine field studies
Epidemiol Rev
Developing and evaluating complex interventions: new guidance—Sept 29, 2008
Evaluation designs for adequacy, plausibility and probability of public health programme performance and impact
Int J Epidemiol
Understanding the impact of male circumcision interventions on the spread of HIV in southern Africa
PLoS One
Male circumcision for HIV prevention in high HIV prevalence settings: what can mathematical modelling contribute to informed decision making?
PLoS Med
Some principles in study design for preventing HIV transmission: rigor or reality
Am J Public Health
Can population differences explain the contrasting results of the mwanza, rakai, and masaka hiv/sexually transmitted disease intervention trials?: a modeling study
J Acquir Immune Defic Syndr
Chapter 21: modelling the impact of HPV vaccines on cervical cancer and screening programmes
Vaccine
Cost-effectiveness of human papillomavirus vaccination and cervical cancer screening in women older than 30 years in the United States
Ann Intern Med
Infectious diseases of humans
Modeling infectious diseases in humans and animals
Cited by (177)
Evaluation of the public policy impacts on Monkeypox in Brazil
2024, Evaluation and Program PlanningAn innovative chain coding mechanism for information processing and compression using a virtual bat-bug agent-based modeling simulation
2022, Engineering Applications of Artificial IntelligenceCitation Excerpt :In other words, agent-based modeling is a different research direction than equation-based modeling (Van Dyke Parunak et al., 1998; Antunes and Takadama, 2007). However, research shows that it has benefits over other numerical modeling strategies (Marshall, 2017; Garnett et al., 2011): first, agents can have different features that allow simulating complex behaviors; and second, agents communicate with one another to create productive arrangements. Modeling these “connections” allows different types of network analysis that are almost impossible using mathematical techniques.
A creative chain coding technique for bi-level image compression inspired by the NetLogo HIV agent-based modeling simulation
2022, Journal of Computational Science