Meta-Analysis of Fixed, Random and Mixed Effects Models

A statistical procedure used for integrating the results obtained from a number of findings is termed as Meta-analysis. In a systematic review, all the information collected can be best used by increasing the power of analysis. The precision of treatment effects can be improved and accessed by statistically combining the results of similar studies. A very common degenerative disease among the elderly population with multiple genetic and environmental factors is AgeRelated Macular Degeneration (AMD) that leads to the distorted central vision and severe visual loss in the early and advanced stage, respectively. In this study, the data sets from the results of 20 studies on the effectiveness of the treatments against AMD disease were analysed using fixed, random and mixed effect models. Various plots Forest, Funnel, Radial, QQ were also fitted for fixed, random and mixed effects models. KeywordsSystematic review, Fixed effect model, Random effect model, Meta-analysis, Mixed-effect model.


Introduction
A number of studies relating to the same basic question are available in science as it is a knowledge based process where new ideas always help to improve upon the basic results (Shoemaker et al., 2003). Researchers have been focusing to cumulate and integrate the results obtained from multiple studies following a systematic review i.e. meta-analysis supported by statistical methodology (Glass, 1976;Olkin, 1995).
Statistical synthesis resulting from a series of studies is referred to as meta-analysis in which statistical procedures can be applied to any set of data. It provides more precise estimates of treatment effect as compared to estimates provided by any individual studies. It is useful only when the data have been collected in a systematic manner, that is the process of either locating, appraising and synthesizing data systematically from numerous sources or from selected studies. Meta-analysis provides more precise estimates of treatment effect as compared to estimates provided by any individual studies. This also enables us to achieve robust effects for a large number of the sampled population if the size of the treatment effect is consistent over the studies. Further, it also helps us to identify sources relating to the magnitude of effect size for different treatment effects across the study.
Meta-analysis provides more precise estimates of treatment effect as compared to estimates provided by any individual studies. This also enables us to achieve robust effects for a large number of the sampled population if the size of the treatment effect is consistent over the studies. Further, it also helps us to identify sources relating to the magnitude of effect size for different treatment effects across the study.
Estimate of the strength of the relationship between two different variables can be integrated by utilizing systematic review approach. The basic aim behind using this approach is to represent the results of different studies on a common scale. In meta-analysis, different outcome measures such as odd ratios, risk ratios and many other measures (Borenstein et al., 2009;Fleiss and Berlin. 2009) relying upon the nature of the study and the available information are used for the integration of results.
A combination of research hypotheses can be addressed by conducting a systematic review that integrates the outcomes of different problems. This can be achieved by the common measure of effect size that quantitatively measures the strength of phenomenon and is modelled using meta regression.
In the present work, we focus on three models of Fixed Effects (FE), Random Effects (RE) and Mixed Effects (ME) models. In Section 2, fixed effects model, along with estimation procedure is discussed. Section 3 focuses on random effects model and its comparison with fixed effects model. Restricted Maximum Likelihood (RML) method is explained in Section 4. Basic introduction to mixed effects model is given in Section 5. The real life practical example involving the implementation of these models is discussed in Section 6. Test for Funnel Plot asymmetry and Trim and Fill method are discussed in Sections 7 and 8 respectively. Conclusions follow in Section 9.

Fixed Effects Model
The major assumption for fixed effects model is that all studies in the meta-analysis share a common overall effect size with same impacts from all other risk factors.
Each observed effect size ̂ could vary among studies because of the random errors from each study. The basic purpose behind using fixed effects model is to make a conditional inference about studies included in the systematic review. The basic assumption in fixed effects model is that all factors influencing the effect size are same in all the studies. Consider

Weighted Estimator of
If weighted least square method is used to fit the model, then for ℎ study, ̂ is assigned an appropriate weight . The weighted estimator ̂ of treatment effect is given by Moreover, for independent studies There can be different choices for which are listed below: where is the fixed number of studies.
(ii) = , where is the number of subjects in study and is the total number of subjects = ∑ =1 . and Placebo Group (P) for study i. (iv) = 1 .

Unweighted Estimator of
Unweighted least square provides an estimator ̂ of treatment effect as and in this case An approximate 95% Confidence Interval (CI) for is written as Where estimate of and its variance depends upon weighted or unweighted estimate of treatment effect.

Remark:
In case of dichotomous response, the data can be classified by a (2 × 2) table with respondents and non-respondents as columns and treatment and control group as rows. Let be the no. of observations in cell, = 1, 2, 3, 4 for each study. Then ( ) and ( ) can be calculated from hypergeometric distribution. Association between treatment and response across studies is evaluated using Mantel-Haenszal statistics, which is square of equation (8).

Random Effects Model
Random effects model is an extension of fixed effects model. Heterogeneity among the true effects of different studies may be due to differences in the methods and sample characteristics and such studies are included in most of the meta-analysis problems. One of the methods for designing the heterogeneity is to consider it as simply random. The random effects meta-analysis model assumes that the treatment effect ̂ from study is an estimate of its own true treatment effect with variance . Within and between study variability, important sources of heterogeneity in multiple studies can be associated with the random effect meta-analysis model. An unconditional inference about a large set of studies from which studies are included in the meta-analysis is provided by random effects model. For = 1, 2, … , , and where 2 represents the between study variability around the global treatment effect .
Further, the random effects model is written as = + + where ~ [0, ] and ~ [0, 2 ] describe the within center and between the center variation respectively.
We assume that and are independent and therefore the random effects in (9) can be rewritten as is estimated by the weighted mean as with standard error (SE) estimated as and the weights are given by The 95% Confidence Interval (CI) for is written as There are various methods to estimate 2 and the extensively used estimate is by DerSimonian and Laird (1986) calculated by the method of moments and given as: The significance of heterogeneity across findings is tested by utilizing statistic Q. Within and between studies variability which is a significant cause of heterogeneity for systematic review can be integrated by random effects model. The random effects model is more conservative since The choice whether to use fixed effects model or a random effects model should be used to synthesize treatment effects across studies in the meta-analysis, is not entirely concerned with a test for heterogeneity of treatment effects among the studies. Rather model selection shall depend on a common effect size. If not, random effects model should be preferred.

Restricted Maximum Likelihood Method (RML)
For estimating 2 , Restricted Maximum Likelihood (RML) method is used. The Restricted Maximum Likelihood Estimator (RMLE) is approximately unbiased and quite efficient (Normand, 1999;Viechtbauer, 2005). This method suggests retaining the form of the DerSimonian-Laird confidence interval but replacing ̂2 in (17) by ̂2 , the restricted maximum likelihood estimator of 2 . It can be obtained by maximizing the restricted log-likelihood function as shown below: and . An approximate 95% Confidence Interval (CI) for 2 is given by

Mixed Effects Model
Fixed effects and random effects models together constitute a statistical mixed effects model. These models are most commonly practised in the arena of physical, biological and social sciences for either making repeated measurement on the same statistical units or for measuring the clusters of related statistical units. These models are preferred over Analysis of Variance (ANOVA) as they deal with missing values. These models can also consist of one or more moderators for diverting the directions or strength of the relationship between a predictor and an outcome. These combinations result in the mixed effects model given by : True effect of ℎ study for = 1, … , , : The value of the ℎ moderator variable for the ℎ study, 0 : The expected effect size when = 0 for = 1, … , , : Influence of ℎ moderator on the size of the true effects for a one unite increase in , 2 : Amount of residual heterogeneity among the true effects. The main purpose behind this is to analyse moderators effects on the size of the average true effect.

Example
A very common degenerative disease among the elderly population with multiple genetic and environmental factors is Age-related Macular Degeneration (AMD) that leads to the distorted central vision and severe visual loss in the early and advanced stage, respectively. Analysis of data is carried out using fixed, random and mixed effects meta-analysis models. The details of studies conducted by 20 authors during 2009-2012 along with frequencies of treatment (+ve andve) and control group (+ve and -ve) are mentioned in Table 1. The results of 20 studies in terms of forest plot are presented in Figure 1 showing the relative risk of AMD in the treated vs. control groups with the incorporation of 95% confidence interval based on fixed effect model. The consistency of outcomes with differing precisions, that is, with heteroscedastic sampling variances, is analysed by radial plots. In a radial plot, for fixed effects model, the inverse of standard errors (1/√( ) ) is drawn on the horizontal line and the specific observed results standardized by related standard errors ( /√( ) ) on the vertical line where denotes the observed effect in the ℎ study as shown in Figure 2. An arc is plotted on the right side of the plot and the values of the observed outcomes are indicated by projecting a line from (0, 0) through a particular point within the plot onto this arc.
Several estimates with different standard errors are represented by radial plots. It can be used as an alternative to a forest plot to examine heterogeneity in a meta-analysis. The radial plot is shown as Figure 2 Figure 2 shows that the third and last study estimates of the average relative risk of AMD from the right end of the arc are the outliers and the other points are consistent with a radial line corresponding to about -0.12 relative risk of AMD disease. For random effects model, analysis of data gives an estimate of , corresponding Standard Error (SE) and 95% Confidence interval. These values are displayed in Table 3. Since p-value <0.05, hence the null hypothesis that = 0 is rejected at 5% level of significance.  Table 4 shows the values of estimates and 95% confidence intervals for heterogeneity and variability for random effects model. Higgins and Thompson in 2002 implied different parameters to analyse the heterogeneity (Higgins and Thompson, 2002). Percentage of 2 determines the amount of the overall variability in the effects size estimates attributing to the heterogeneity among the true effects. Proportion of the overall variability in the observed results to the amount of sampling variability (̂2 = 0 implies 2 = 1) is referred as 2 . The heterogeneity test (Q= 21.2986, df=19, p>0.05) implies homogeneity among the true effects.
The forest plot for random effects model is shown in Figure 3.  The existence of heterogeneity and different types of publication bias is best detected by funnel plots. For models without moderators, these types of plots are plotted as shown in Figure 4 where the observed outcomes are taken on the horizontal line and corresponding SE on the vertical line.
The estimate based on the model is shown by a vertical line and confidence interval region is plotted over this value with ±1.96( ). When the publication bias is not present, it is assumed that high precision findings are drawn near the average whereas the low precision findings are expanded constantly on the both sides of the average, creating a distribution with funnelstructure. Publication bias is represented by deviation from this shape. 95% of the outcomes should be within the funnels defined by the straight lines when the heterogeneity is not present. From the Figure 4 it is observed that there is no publications bias as the studies with low precision are expanded constantly on both sides of the average. Also, no heterogeneity is seen as 95% of the studies lie within the funnel plot.
Radial plot for random effects model is given in the following Figure 5. For a random effects model, the function uses 1/√( +̂2) on the X-axis and /√( +̂2) on the Y-axis. Figure 5 shows that the third and last study estimates of the average relative risk of AMD from the right end of the arc are the outliers and the other points are consistent with a radial line corresponding to about -0.13 relative risk of AMD disease. Other important diagnostic tools in the meta-analysis are QQ plots. The horizontal axis represents the theoretical quantiles of a normal distribution while the vertical axis shows the observed quantiles of the (externally) standardized residuals. A line with a slope of 1, going through the (0, 0) point and a pseudo confidence envelope are added to the plot where the line added to act as a reference. Envelopes are developed by using the quantile sets of simulated residuals (Cook and Weisberg, 1982). The QQ plots for random effects models are drawn in Figure 6 along with 95% CI for quantile line.  Table 5 shows the values of estimates, corresponding standard errors, p-values and 95% confidence intervals for mixed effects model. The estimates of heterogeneity and variability are displayed in Table 6. Testing for moderators gives (df = 2) = 09.8804, p-value = 0.0072.
Testing for Residual Heterogeneity gives (df = 17) = 11.4182, p-value = 0.8340 The overall heterogeneity is evaluated by comprehending two moderators in the model and is given as Testing of the significance of individual coefficients by a statistical model that depends on standard normal distribution whereas the test depends on chi-square distribution with r degrees of freedom, where is the no. of coefficients examined.
In the absence of moderators in the model, Cochran's Q-test (Cochran, 1954) is used for checking if the variability in the observed effect sizes or results is greater than the expected based on sampling variability alone. However, in the presence of moderators, test for residual heterogeneity, is used for checking if the variability in the observed effect sizes or results that does not include moderators in the model is greater than the expected based on sampling variability alone.
The following Figure 7 shows the funnel plot for mixed effects model. Residuals against their corresponding standard errors are shown on the X-axis and a vertical line with a confidence interval region with ±1.96( ) is drawn at zero. It shows no heterogeneity as 95% of the studies lie within the funnel plot.

Funnel Plot Asymmetry Test
Funnel plots are an important graphical device for determining a particular type of publication bias. Unpublished studies with small or non-significant findings give rise to the asymmetric funnel plots, which can be determined by correlating the results with sampling variances, standard errors or sample sizes. Begg andManumdar (1994), andEgger et al. (1997) reported regression test and rank correlation test, respectively for checking the asymmetry of the funnels plots (Begg andMazumdar, 1994, Egger et al., 1997). Many researchers further reported the extensions, modification and further developments of the regression tests (Macaskill et al., 2001;Sterne and Egger, 2005;Harbord et al., 2006;Peters et al., 2006;Rucker et al., 2008;Moreno et al., 2009).
For the given data, the regression test with a weighted regression model using the standard error as the predictor, the value of t-statistics is calculated as t =-1.5980, df =18, p-value =0.1275 thereby commending symmetry in the funnel plot for the random effects model.
Influence of the potential moderators is controlled by compounding mixed effects model with a sample size of findings as a potential predictor. Regression test for Funnel plot asymmetry for mixed effects model is given by z =2.1459, p-value =0.0319. Hence, for the mixed effects model, this test proposes asymmetry in the funnel plot.

Trim and Fill Method
The trim and fill method proposed by team of Duval is based on nonparametric (rank-based) data augmentation technique (Duval and Tweedie, 2000a;Duval and Tweedie, 2000b;Duval, 2005).
The number of missing findings in systematic review arising from the suppression of most extreme outcomes on one side of the funnel plot is determined using this method. Thereafter the observed outcomes are added to the funnel plot to make it more symmetric. This method can only be used for fixed or random effects model. The method is not only used to provide more significant estimates of the overall effect but also determines the sensitivity of the results for a specified procedure.
Estimated number of missing studies was attained by using trim and fills method after fitting the fixed effects model. Test for heterogeneity: Q (df =23) =38.8460 with p-value=0.0206. Table 7 gives the estimate of with standard error, p-value and 95% confidence interval. From the above results, it is concluded that the treatment effect is statistically insignificant with the missing studies filled in.
A funnel plot with the filled-in studies for fixed effects model is shown in Figure 9.  Estimated number of missing studies on the right hand side of funnel plot shown in Figure 9 is 4 with SE = 3.0122.
Estimated number of missing studies was also attained by using trim and fills method after fitting the random effects model. Figure 10 shows a funnel plot with filled in studies for random effects model.
Test for heterogeneity: Q (df = 20) = 25.7118 with p-value= 0.1755. Table 8 gives the estimate of with standard error, p-value and 95% confidence interval. From the above results, it is concluded that the treatment effect is statistically insignificant with the missing studies filled in. Table 9 shows the estimates of heterogeneity and variability parameters and corresponding 95% confidence intervals based on the trim and fill. The amount of heterogeneity in the true log relative risks is estimated to be ̂2 = 0.0152. Substantial amount of heterogeneity was achieved as shown by Q test (Q= 25.7118, df= 20, p>0.05).  Figure 10. Funnel plot based on the trim and fill method for random effects model Estimated number of missing studies on the right hand side of funnel plot as shown in Figure10 is 1 with SE = 2.9059.
As compared to results of the fixed effects model and random effects model without the missing studies, the result is statistically significant for both the fixed and random effects model.

Conclusions
In this study, the effect of treatment against AMD disease data sets obtained from outcomes of 20 findings was analysed. The treatment is statistically significant for fixed and random effects models. For mixed effects model, only year moderator comes out to be significant. However, for omnibus test , both moderators have a significant influence on the treatment. Various plots forest, funnel, radial, QQ are also fitted for fixed, random and mixed effects models. Regression test suggest that the funnel plots are symmetric for random effects model while for mixed effects models they are found to be asymmetric. Overall, it is concluded that the treatment effect is statistically insignificant with the missing studies filled in for both the fixed and random effects models.