Count outcome meta-analysis for comparing treatments by fusing mixed data sources: comparing interventions using across report information

Assessing interventions applied to target populations is a matter of prime interest. Studies are usually undertaken to see whether an alternative intervention is superior (or at least equivalent) to a comparable standard intervention. This is typically achieved by comparing alternative and standard intervention within a given study, and the developed meta-analytic methodology is building on this assumption. Very little work has been delivered when studies only report results on one of the interventions only, but not on both. This is the situation we consider here, and it is motivated by study reports on two surgeries for treatment of asymptomatic antenatally diagnosed congenital lung malformations in young children. Reports are often only available for one of the two, and restricting analysis on those with results on both surgeries will restrict data to 33% of the potential sources. We show in this paper how data sources can be fused and under which condition this fusion will provide valid results. Application to the case study shows the potential gain of the suggested approach in reaching a more conclusive analysis. We argue that studies should best allow within-study comparison, but if only one intervention information is available (for example, as the required surgery expertise for the comparative intervention is not deliverable at the respective site), harnessing one-group information can provide additional insights.


Introduction and motivation
This work is motivated by a meta-analysis using reported data comparing thoracoscopic, or keyhole surgery, and open surgery for treatment of asymptomatic antenatally diagnosed congenital lung malformations in young children. The mean age of the children involved in the studies is 15 months, and both surgeries have no deaths reported. Thoracoscopy has become more widely used because it requires only a small incision in the chest wall. We consider the following question: How does keyhole perform versus open w.r.t. total complications? Adams et al. (2017) considered a meta-analysis of 12 reports comparing keyhole and open surgery as listed in Table 1. These data allow a standard meta-analysis as follows. For each study, an effect measure, here the risk ratio, is calculated associated with an estimate of its standard error. This allows a calculation of a summary measure with 95% confidence interval. We use here the package STATA15 (Stata-Corp. 2017) in connection with an add-on package metan (see also Palmer and Sterne 2009) for delivery of the calculation. The results are displayed in Table 2. This is an example of a standard, two-stage meta-analysis where in the first stage for each study, an effect measure is calculated and in the second stage the studyspecific effect estimates are further analyzed. This approach is extensively described in the existing literature (Borenstein et al. 2009;Cooper et al. 2009;Schwarzer et al. 2015). In the application study here, there is a significant beneficial effect of keyhole surgery w.r.t. the number of complications (which includes bleeding, wound or chest infections, or tracheal injury among others) and the effect is homogeneous over the studies as the test of homogeneity is not significant. These results are also visualized in the forest plot in Fig. 1. Note that all but one of the studies show non-significant Count outcome meta-analysis for comparing treatments by fusing… results, whereas the meta-analytic summary estimator clearly does. This demonstrates one of the benefits of a meta-analysis. In addition to the 12 studies that have been used in Adams et al. (2017) as these included information on both treatment groups and, hence, allowing a conventional meta-analysis, there were 24 additional reports available, of which 15 had only   Tables 3 and 4.
These additional 24 studies were ignored in Adams et al. (2017) as for any of these it is not possible to calculate a study-specific risk ratio estimate since a comparator treatment is missing. Hence, this does not allow a conventional two-stage meta-analysis where in the first stage a within-study effect is estimated and then this effect estimate is further analyzed in a second stage. This setting of having only one  1 3 Count outcome meta-analysis for comparing treatments by fusing… result per study available (with the comparator result missing) has not been considered in meta-analysis. To overcome this difficulty, we suggest a one-stage modelling approach which will allow to use the information from all 36 studies and which we will detail in the following section.

A count modelling approach using Poisson regression
We consider the number of complications X as a Poisson count with mean E(X) = n where n is the size of the study report. Clearly, = E(X)∕n is the incidence risk of complications. We write for report i for j = 1 (treatment=keyhole) and j = 0 (comparison=open), so that the risk ratio RR = 1 ∕ 0 , assumed to be independent of the study i, for the time being. Taking logarithms on both sides of (1), we yield where = log( 1 ∕ 0 ) is the log-risk ratio, is the log-baseline risk, and log n ij enters as an offset (a covariate with a fixed, known coefficient) into the modelling. Finally, it is assumed that the count X ij follows a Poisson distribution where Po(x| ) = exp(− ) x ∕x!.

Fusion of the Poisson likelihoods
According to the available data, we have the following, three different likelihoods.
The first likelihood appears for those studies where both, keyhole and open surgery, information is available: where k 0 are the reports involving both techniques. The second likelihood occurs for those studies with only information on keyhole surgery: where k 1 are the reports involving only keyhole. Finally, the third likelihood occurs for those studies with only open surgery information: where k 2 are the reports involving only open surgery. This leads to the joint likelihood where stands for a generic parameter.

Poisson likelihoods with random effect for study
It appears reasonable to capture the baseline variation across studies with a random effect. Hence, let i ∼ N( , 2 ) be a normal random effect with mean and variance 2 . Then, the likelihood for studies with information on keyhole and open surgery becomes: where ( i | , 2 ) is a normal density with mean and variance 2 with similar expressions for the other likelihoods: and Again, we can form the joint likelihood In Table 5, we find the analysis for the studies with information on both groups, hence using L 0 , and for the studies including mixed arm information, in other words using the joint likelihood L. We note that the latter analysis shifts the borderline

3
Count outcome meta-analysis for comparing treatments by fusing… significance of the risk ratio to a clearly significant result. For both analysis, the baseline random effect i for study is significant and more precisely has a positive variance, significantly different from zero. The model is easily extendible to allow heterogeneity of effect across studies where i is now a normal random effect for study report i. For example, the likelihood for studies with only information on keyhole becomes with similar expressions for the other likelihoods corresponding to the available study information. In Table 6, a model evaluation is provided which shows that there is no evidence for heterogeneity of effect across studies.

Simulation study
We evaluated the performance of the two Poisson regression methods: one based only on the studies with information on both arms and the other based additionally on the studies including mixed arm information, by means of simulation. We consider a Poisson model that allows a random effect for study. In the simulation study, the data were generated from two, potentially different, Poisson distributions for the treatment and comparison groups, respectively. The number of studies (k) was chosen as 20, 40, 60, and 80. Furthermore, the simulated meta-analytic data included 50% of all studies with information in two arms and 50% of all studies with information in one arm, the latter having an equal split on treatment and comparison group. The settings were set to mimic the data on comparing open and keyhole surgery. We used = −2, 2 = 0.7 , and = −0.5 and 0.5, leading to the true risk ratios of 0.61 and 1.65, respectively. For each situation, 1000 simulation replications were used. The performance of the estimators in the Poisson model with baseline random effect was evaluated in terms of bias and root mean squared error (RMSE). As seen in Tables 7 and 8, the bias of the log-risk ratio ( ̂ ) and the bias of the variance of baseline risk ( ̂2 ) were closer to zero when using the studies with mixed arm information in comparison with the respective bias obtained from the method using the studies with information on both arms only, in almost all cases. The RMSEs of ̂ and the RMSEs of ̂2 computed from the method based on mixed arm information were smaller than those of the compared method in all cases. Our results emphasize that Poisson regression analysis using all available information can provide a benefit in a meta-analysis. At least in the situation studied here, it yields good performance in terms of bias and mean squared error of the estimated parameters of interest.

Diagnostics
Clearly, the approach suggested here goes beyond the conventional within-study comparison to estimate the treatment effect. Hence, we must considerate that comparing treatment across studies might lead to a different result than comparing treatment within studies. In the following, we outline a strategy to diagnose a potential discrepancy between study estimates using both arm information and study estimates using one arm information only. The strategy is as follows:

3
Count outcome meta-analysis for comparing treatments by fusing… • fit the model for all reports using = ( , 2 , ) • fit the model for all reports but with 1 for the subset of reports with both surgeries and with 2 for the subset with only one surgery • evaluate on a 2 -scale with 3df • in the case here, 2 log = 6.14 with associated p-value = 0.1051 which is above the conventionally used threshold of 0.05, so that we do not reject the common parameter model.
A more direct (but also more limited) approach is as follows: Define the indicator variable and the effect variable and assess treatment × both/mixed information interaction S × T by means of investigating the coefficient for significance in the model (12): where the treatment t = 0, 1 indicates open and keyhole surgery, respectively, and s = 0, 1 indicates whether the study has only one type of surgery (0) or both (1).
We conclude from the analysis in Table 9 that there is no evidence that keyhole/open effect is differential in reports with both surgeries reported to reports with only one surgery (the treatment effect is not affected by the type of study report), so that conclusions might be based upon the total of 36 reports (Fig. 2). (12) log E(X it ) = log n it + log j = log n it + + × t + (s × t),

Discussion
The paper is based on the idea of fusing several likelihoods. Here, we used mixed Poisson likelihoods. This model is often used for rates where events occur within a given person-time. If the person-time is identical for all individuals under risk, the persontime reduces to the sample size. In the latter case, the binomial model would then occur as an alternative. Also, the Poisson model is not the only possible model for offset settings, and here, an alternative could be the negative-binomial distributions. In any case, the arguments of fusing likelihoods would be identical. In addition, we argue that the mixed Poisson model that we have used here and which uses a random effect for the factor study, provides quite a flexible model. It remains in the debate how much information can be gained from reports providing only one intervention outcome, in particular, for comparative analysis. We have indicated that gain can be reached, but it is limited. In addition, it is more appropriate from the statistical perspective to have all available information included in the analysis. Clearly, there is no doubt to use all report information if interest is in absolute risk, whether there is one-group information or two-group information per study. Of course, there is then also the question how this information could be combined, but we leave this for another discussion.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creat iveco mmons .org/licen ses/by/4.0/.

Fig. 2 Summary plot: "both"
is based on all studies with complication information in open and keyhole surgery, "only one" is based on those study with reports only in one of the two groups (open or keyhole surgery), and "all" is a merger of "both" and "only one"