Validating the Assumptions of Population Adjustment: Application of Multilevel Network Meta-regression to a Network of Treatments for Plaque Psoriasis

Background Network meta-analysis (NMA) and indirect comparisons combine aggregate data (AgD) from multiple studies on treatments of interest but may give biased estimates if study populations differ. Population adjustment methods such as multilevel network meta-regression (ML-NMR) aim to reduce bias by adjusting for differences in study populations using individual patient data (IPD) from 1 or more studies under the conditional constancy assumption. A shared effect modifier assumption may also be necessary for identifiability. This article aims to demonstrate how the assumptions made by ML-NMR can be assessed in practice to obtain reliable treatment effect estimates in a target population. Methods We apply ML-NMR to a network of evidence on treatments for plaque psoriasis with a mix of IPD and AgD trials reporting ordered categorical outcomes. Relative treatment effects are estimated for each trial population and for 3 external target populations represented by a registry and 2 cohort studies. We examine residual heterogeneity and inconsistency and relax the shared effect modifier assumption for each covariate in turn. Results Estimated population-average treatment effects were similar across study populations, as differences in the distributions of effect modifiers were small. Better fit was achieved with ML-NMR than with NMA, and uncertainty was reduced by explaining within- and between-study variation. We found little evidence that the conditional constancy or shared effect modifier assumptions were invalid. Conclusions ML-NMR extends the NMA framework and addresses issues with previous population adjustment approaches. It coherently synthesizes evidence from IPD and AgD studies in networks of any size while avoiding aggregation bias and noncollapsibility bias, allows for key assumptions to be assessed or relaxed, and can produce estimates relevant to a target population for decision-making. Highlights Multilevel network meta-regression (ML-NMR) extends the network meta-analysis framework to synthesize evidence from networks of studies providing individual patient data or aggregate data while adjusting for differences in effect modifiers between studies (population adjustment). We apply ML-NMR to a network of treatments for plaque psoriasis with ordered categorical outcomes. We demonstrate for the first time how ML-NMR allows key assumptions to be assessed. We check for violations of conditional constancy of relative effects (such as unobserved effect modifiers) through residual heterogeneity and inconsistency and the shared effect modifier assumption by relaxing this for each covariate in turn. Crucially for decision making, population-adjusted treatment effects can be produced in any relevant target population. We produce population-average estimates for 3 external target populations, represented by the PsoBest registry and the PROSPECT and Chiricozzi 2019 cohort studies.

where the event probability in each category is ; and 4 =1 , = 1 for each , , . The category event probabilities are transformed onto the linear predictor scale using the probit link function Φ −1 (·) (the Normal inverse cumulative distribution function). We use the probit link function here for comparability with previous analyses, but another suitable choice would be the logit link function. The linear predictor for an individual on treatment in trial with covariate vector is ( ). The parameters are study-specific baselines, 1 are coefficients for prognostic variables, and 2, are coefficients for effect modifiers specific to each treatment .
The effect of treatment (at the individual level), , is defined with respect to the network reference treatment 1, and we set 1 = 0 and 2,1 = 0. Some coefficients in 1 or 2, may be set to zero, if it is known that a particular covariate is not prognostic or effect modifying respectively. on 2 and 3 , which are automatically truncated to satisfy the ordering constraints (A.2). We also place vague N(0, 10 2 ) prior distributions on each of the parameters , 1 , 2, , and .
Aggregate outcomes are vectors of summary outcome counts in each category • = ( • ;1 , . . . , • ; ) T . We work with these category counts in "exclusive" format, where individuals are only counted in the highest category they achieve (as opposed to "inclusive" counts where individuals are counted in every category up to and including the highest category achieved; it is a straightforward matter of addition or subtraction to convert between formats).
These summary data are given a Multinomial likelihood, with the average event probabilities in each category obtained by integrating the individual-level model (A.1b) over the covariate joint distribution (·) in each arm of each AgD study: where¯; are the average event probabilities in each category and = • ; is is the sample size in each arm. We compute the integrals for¯; in (A.3c) using Quasi-Monte Carlo integration [13] with˜= 1000 integration points˜drawn from joint distribution (·) of the covariates on each treatment in study , so that

Accounting for studies reporting a subset of categories
It is not uncommon for some studies to report only a subset of categories. These studies can be incorporated at either the individual or aggregate level by modifying equations Equation (A.1) or Equation (A.3), respectively, to involve the relevant latent cutpoints [4]. That is, given a reported set of categories 1 < · · · < in study , the individual-level event probabilities in (A.1b) for the reported categories become Similarly at the aggregate level, the average event probabilities (A.3b) in each of the reported categories become¯; =¯; =¯; −1 −¯; for = 1, . . . , .

A.1.2 Producing population-average estimates for populations of interest
Population-average estimates of quantities of interest to decision-making, such as average treatment effects and average event probabilities, can be produced by averaging estimates of individual-level quantities over the covariate joint distribution in the target population [13].
For decision-making based on cost-effectiveness models, the typical inputs are the populationaverage event probabilities for a cohort-based model (e.g. a decision tree or Markov model) or individual event probabilities for an individual-based model (e.g. a discrete event simulation).
The target population need not be one of the studies in the network; indeed, it is more likely represented by a registry or cohort study [11].
To estimate the proportion of individuals achieving each PASI endpoint in a given population,

A.1.3 Transforming information on baseline response
As described in Appendix A.1.2, to produce estimates of quantities of interest such as predicted probabilities of response we require a distribution for the baseline response probit probability