Elsevier

The Lancet

Volume 378, Issue 9790, 6–12 August 2011, Pages 515-525
The Lancet

Review
Mathematical models in the evaluation of health programmes

https://doi.org/10.1016/S0140-6736(10)61505-XGet rights and content

Summary

Modelling is valuable in the planning and evaluation of interventions, especially when a controlled trial is ethically or logistically impossible. Models are often used to calculate the expected course of events in the absence of more formal assessments. They are also used to derive estimates of rare or future events from recorded intermediate points. When developing models, decisions are needed about the appropriate level of complexity to be represented and about model structure and assumptions. The degree of rigor in model development and assessment can vary greatly, and there is a danger that existing beliefs inappropriately influence judgments about model assumptions and results.

Introduction

Although policy decisions in public health would ideally be based on evaluations that measure effect directly, modelling does, and often should, play a major part in large-scale evaluations. The purpose of our Review is to assist readers to assess critically and interpret appropriately the results of such modelling exercises. We first present practical and theoretical reasons why models are and should be used in large-scale evaluations. We then summarise the types of different modelling approaches and discuss how models and their outputs should be judged.

Section snippets

The role of models in programme evaluations

Public health programmes need to be evaluated on whether anticipated benefits are indeed happening and whether they are cost effective.1, 2 For many programmes, a randomised controlled trial is not an option for ethical or practical reasons (eg, programme implementation is done by large groups, such as national health systems, which cannot be randomised). Alternatively, the evaluation question might not be whether an intervention is effective, but whether it is being successfully implemented.

Mathematical modelling methods

Mathematics provides a precise quantitative language to describe the relation between variables and changes in states, and in medicine we can represent mathematically the clinical course of disease, the distribution of disease across populations and over time, and the mechanisms that generate disease.28, 29 The development of a model of infection, disease, or death requires us to precisely set out our assumptions about the parameters and processes influencing health, and enables us to calculate

Assessing models

Panel 1 provides a summary checklist of items that should accompany the most rigorous model analyses. None of these indicators of model quality guarantees that the model produces accurate outputs, but the presence of this information will help readers assess the appropriateness of the model.

It is essential to have a clear understanding of the model's structure to judge model results and outputs. We believe that in addition to the full technical description of the model, there should also be a

Discussion

With recent increases in resources committed to improve global health, there is growing demand for accountability and efficiency in programme implementation and the need for good evaluation. In many situations, available data and appropriate modelling techniques can clarify, within a causal framework, the relation between programme inputs and effect. Modelling exercises that are well done can provide credible evidence of the value of programmes and guide the roll out and improvement of

References (71)

  • RM Granich et al.

    Universal voluntary HIV testing with immediate antiretroviral therapy as a strategy for elimination of HIV transmission: a mathematical model

    Lancet

    (2009)
  • AP Plaisier et al.

    ONCHOSIM: a model and computer simulation program for the transmission and control of onchocerciasis

    Comput Methods Programs Biomed

    (1990)
  • SK Seaholm et al.

    Latin hypercube sampling and the sensitivity analysis of a Monte Carlo epidemic model

    Int J Biomed Comput

    (1988)
  • J Jacquez et al.

    Modeling and analysing HIV transmission: the effect of contact patterns

    Math Biosci

    (1988)
  • I Baussano et al.

    HPV-16 infections and cervical cancer: modeling the influence of duration of infection and precancerous lesions

    Epidemics

    (2010)
  • M Chopra et al.

    Saving the lives of South Africa's mothers, babies, and children: can the health system deliver?

    Lancet

    (2009)
  • TB Hallett et al.

    Appropriate evaluation of HIV prevention interventions: from experiment to full-scale implementation

    Sex Transm Infect

    (2007)
  • CG Victora et al.

    Evidence-based public health: moving beyond randomized trials

    Am J Public Health

    (2004)
  • MC Boily et al.

    Evaluating large-scale HIV prevention interventions: study design for an integrated mathematical modelling approach

    Sex Transm Infect

    (2007)
  • The five-year evaluation of the global fund to fight aids, tuberculosis, and malaria—synthesis of study areas 1, 2 and 3: March 2009

  • TB Hallett et al.

    How to pay “cash-on-delivery” for hiv infections averted: two measurement approaches and ten payout functions—working paper 210

  • PE Lekone et al.

    Statistical inference in a stochastic epidemic SEIR model with control intervention: ebola as a case study

    Biometrics

    (2006)
  • MJ Ferrari et al.

    The dynamics of measles in sub-Saharan Africa

    Nature

    (2008)
  • CE Stein et al.

    The global burden of measles in the year 2000—a model that uses country-specific indicators

    J Infect Dis

    (2003)
  • ME Halloran et al.

    Design and interpretation of vaccine field studies

    Epidemiol Rev

    (1999)
  • Developing and evaluating complex interventions: new guidance—Sept 29, 2008

  • JP Habicht et al.

    Evaluation designs for adequacy, plausibility and probability of public health programme performance and impact

    Int J Epidemiol

    (1999)
  • TB Hallett et al.

    Understanding the impact of male circumcision interventions on the spread of HIV in southern Africa

    PLoS One

    (2008)
  • Male circumcision for HIV prevention in high HIV prevalence settings: what can mathematical modelling contribute to informed decision making?

    PLoS Med

    (2009)
  • M Susser

    Some principles in study design for preventing HIV transmission: rigor or reality

    Am J Public Health

    (1996)
  • RG White et al.

    Can population differences explain the contrasting results of the mwanza, rakai, and masaka hiv/sexually transmitted disease intervention trials?: a modeling study

    J Acquir Immune Defic Syndr

    (2004)
  • GP Garnett et al.

    Chapter 21: modelling the impact of HPV vaccines on cervical cancer and screening programmes

    Vaccine

    (2006)
  • JJ Kim et al.

    Cost-effectiveness of human papillomavirus vaccination and cervical cancer screening in women older than 30 years in the United States

    Ann Intern Med

    (2009)
  • RM Anderson et al.

    Infectious diseases of humans

    (1991)
  • MJ Keeling et al.

    Modeling infectious diseases in humans and animals

    (2007)
  • Cited by (177)

    • An innovative chain coding mechanism for information processing and compression using a virtual bat-bug agent-based modeling simulation

      2022, Engineering Applications of Artificial Intelligence
      Citation Excerpt :

      In other words, agent-based modeling is a different research direction than equation-based modeling (Van Dyke Parunak et al., 1998; Antunes and Takadama, 2007). However, research shows that it has benefits over other numerical modeling strategies (Marshall, 2017; Garnett et al., 2011): first, agents can have different features that allow simulating complex behaviors; and second, agents communicate with one another to create productive arrangements. Modeling these “connections” allows different types of network analysis that are almost impossible using mathematical techniques.

    View all citing articles on Scopus
    View full text