Informing policy makers on the efficiency of population level tobacco control interventions in Asia: A systematic review of model-based economic evaluations

Background Economic evaluations of tobacco control interventions support decisions regarding resource allocation in public health policy. Our systematic review was aimed at identifying potential bias in decision models used to estimate the long-term costs and effects of population-based tobacco control interventions in Asia. Methods We included studies conducted in Asian countries and using a modelling technique to evaluate the economic impacts of one or more population-based tobacco interventions in line with the Framework Convention on Tobacco Control (FCTC). We assessed the structure, input parameters, and risk of bias for each model, and performed a narrative synthesis of the included studies. Results Nine model-based economic evaluation studies of population-based tobacco interventions were identified. About 60% of the criteria for reporting quality were met in all studies, indicating that reporting generally lacked transparency. The studies were highly heterogeneous in terms of the scope, types, and structures of their models and the quality of input parameters. One-third of the models applied in the studies scored a high risk of bias, with problems mostly falling into the following categories: model type, time horizons, and smoking transition probabilities. Conclusions More data are needed to provide high-quality evidence regarding the cost-effectiveness of tobacco control policies in Asia. Strong evidence at the country level hinges on the availability of accurate estimates of the effects of the interventions, the relative risks of smoking, and the price elasticity of the demand for tobacco. Simple transfers of models built in Western populations do not suffice. Protocol registration PROSPERO CRD 42019141679.

: Overview of studies included in the review 10 Table S3: Characteristics and structure of the models 13

Appendix: S3 Quality of reporting checklists
The full checklist is provided [1].

S1
Is the objective of the evaluation and model specified and consistent with the stated decision problem? S1 Is the primary decision-maker specified? S2 Is the perspective of the model stated clearly? S2 Are the model inputs consistent with the stated perspective? S2 Has the scope of the model been stated and justified?

S2
Are the outcomes of the model consistent with the perspective, scope and overall objective of the model?

S3
Is the structure of the model consistent with a coherent theory of the health condition under evaluation? S3 Are the sources of data used to develop the structure of the model specified? S3 Are the causal relationships described by the model structure justified appropriately? S4 Are the structural assumptions transparent and justified?

S4
Are the structural assumptions reasonable given the overall objective, perspective and scope of the model? S5 Is there a clear definition of the options under evaluation? S5 Have all feasible and practical options been evaluated? S5 Is there justification for the exclusion of feasible options?

S6
Is the chosen model type appropriate given the decision problem and specified causal relationships within the model?

S7
Is the time horizon of the model sufficient to reflect all important differences between options?

S7
Are the time horizon of the model, the duration of treatment and the duration of treatment effect described and justified?

S8
Do the disease states (state transition model) or the pathways (decision tree model) reflect the underlying biological process of the disease in question and the impact of interventions? S9 Is the cycle length defined and justified in terms of the natural history of disease?

D1
Are the data identification methods transparent and appropriate given the objectives of the model? D1 Where choices have been made between data sources, are these justified appropriately?

D1
Has particular attention been paid to identifying data for the important parameters in the model? D1 Has the quality of the data been assessed appropriately? D1 Where expert opinion has been used, are the methods described and justified?

D2
Is the data modelling methodology based on justifiable statistical and epidemiological techniques? D2a Is the choice of baseline data described and justified? D2a Are transition probabilities calculated appropriately? D2a Has a half-cycle correction been applied to both cost and outcome? D2a If not, has this omission been justified? If relative treatment effects have been derived from trial data, have they been synthesized using appropriate techniques?

D2b
Have the methods and assumptions used to extrapolate short term results to final outcomes been documented and justified? D2b Have alternative assumptions been explored through sensitivity analysis?

D2b
Have assumptions regarding the continuing effect of treatment once treatment is complete been documented and justified? D2c Are the costs incorporated into the model justified? D2c Has the source for all costs been described? D2c Have discount rates been described and justified given the target decision-maker? D2d Are the utilities incorporated into the model appropriate? D2d Is the source for the utility weights referenced? D2d Are the methods of derivation for the utility weights justified?

D3
Have all data incorporated into the model been described and referenced in sufficient detail?

D3
Has the use of mutually inconsistent data been justified (i.e. are assumptions and choices appropriate)? D3 Is the process of data incorporation transparent?

D3
If data have been incorporated as distributions, has the choice of distribution for each parameter been described and justified?

D3
If data have been incorporated as distributions, is it clear that second order uncertainty is reflected? D4 Have the four principal types of uncertainty been addressed? D4 If not, has the omission of particular forms of uncertainty been justified?

D4a
Have methodological uncertainties been addressed by running alternative versions of the model with different methodological assumptions?

D4b
Is there evidence that structural uncertainties have been addressed via sensitivity analysis?

D4c
Has heterogeneity been dealt with by running the model separately for different subgroups? D4d Are the methods of assessment of parameter uncertainty appropriate?

D4d
If data are incorporated as point estimates, are the ranges used for sensitivity analysis stated clearly and justified?

C1
Is there evidence that the mathematical logic of the model has been tested thoroughly before use? C2 Are any counterintuitive results from the model explained and justified?

C2
If the model has been calibrated against independent data, have any differences been explained and justified?

C2
Have the results of the model been compared with those of previous models and any differences in results explained?

Appendix: S4 Quality of sources of evidence
The full checklist is provided [2].

Reference
Level of quality of evidence used proposed by Cooper "Use of evidence in decision models" *** 1 Direct utility assessment for the specific study from a sample: High 1 Indirect utility assessment from specific study from a patient sample with disease(s) of interest: using a tool validated for the patient population High 2 Indirect utility assessment from specific study from a patient sample with disease(s) of interest using tool not validated for the patient population 3 Direct utility assessment from a previous study from a sample either: 3 Indirect utility assessment from previous study from patient sample with disease(s) of interest: using tool validated for the patient population 4 Indirect utility assessment from previous study from patient sample with disease(s) of interest: using tool not validated for the patient population or method of elicitation unknown

S5 ECOBIAS checklist for bias in economic evaluation
The full checklist is provided [3]. Bias related to baseline data Are probabilities, for example, based on natural history data? Is transformation of rates into transition probabilities done accurately? 7 Bias related to treatment effects Are relative treatment effects synthesized using appropriate meta analytic techniques? Are extrapolations documented and well justified? Are alternative assumptions explored regarding extrapolation? 8 Bias related to quality-of-life weights (utilities) Are the utilities incorporated appropriate for the specific decision problem? 9 Non-transparent data incorporation bias Is the process of data incorporation transparent? Are all data and their sources described in detail? 10 Limited scope bias Have the four principles of uncertainty (methodological, structural, heterogeneity, parameter) been considered? Bias related to consistency 11 Bias related to internal consistency Has internal consistency in terms of mathematical logic been evaluated?    3  3  3  3  3  1  3  3  3 D1 Has the quality of the data been assessed appropriately?  Is the data modelling methodology based on justifiable techniques?   1  3  3  3  3  3  3  NA  3   D2a Is the choice of baseline data described and justified?   3  3  3  3  3  3  3  3  3 D2a Are transition probabilities calculated appropriately?
Has a half-cycle correction been applied to both cost and outcome?  NA  3  3  3  3  3  1  3  3 D2b Have the methods and assumptions used to extrapolate short term results to final outcomes been documented and justified?
Have alternative assumptions been explored through sensitivity analysis?
D2b Have assumptions regarding the continuing effect of treatment once treatment is complete been documented and justified?
D2c Are the costs incorporated into the model justified?
D2c Has the source for all costs been described?