MetaEasy : A Meta-Analysis Add-In for Microsoft Excel

Meta-analysis is a statistical methodology that combines or integrates the results of several independent clinical trials considered by the analyst to be ‘combinable’ (Huque 1988). However, completeness and user-friendliness are uncommon both in specialised meta-analysis software packages and in mainstream statistical packages that have to rely on user-written commands. We implemented the meta-analysis methodology in an Microsoft Excel add-in which is freely available and incorporates more meta-analysis models (including the iterative maximum likelihood and proﬁle likelihood) than are usually available, while paying particular attention to the user-friendliness of the package.


Introduction
Meta-analysis can be defined as the statistical analysis of a large collection of analysis results from individual studies -usually Randomised Controlled Trials (RCTs) -for the purpose of integrating the findings (Glass 1976).Although the debate regarding the quality and application caveats of the method is ongoing (Egger and Smith 1997;Bailar 1997), a Medline (http://medline.cos.com/)search by the authors reveals that the number of meta-analysis studies published in peer-reviewed journals seems to be growing exponentially (Figure 1).Published meta-analysis studies (search criterion: Publication Type=meta-analysis) have risen from 274 in 1990 to 2138 in 2005, while published work that is either a meta-analysis or deals with meta-analysis issues (search criterion: Keyword=meta-analysis) has increased from 329 to 3350, in the same period.
A major issue in meta-analysis is the almost inevitable clinical or methodological heterogeneity among the combined studies (Eysenck 1994).If the study results differ greatly (large Publication type: meta-analysis Keyword: meta-analysis (including meta-analysis studies) Figure 1: Number of meta-analysis publications (Medline search).
heterogeneity) then it may not be appropriate to combine them, since an estimate of overall effect in such a case is of questionable value (Egger, Smith, and Phillips 1997;Thompson 1994).Nevertheless, the statistical error that stems from this between-study diversity can be quantified and modelled (Higgins and Thompson 2002;DerSimonian and Laird 1986), and a meta-analysis on carefully selected Randomised Controlled Trials, in terms of compatibility, is the next best thing to a new large and expensive, prohibitively so in many cases, RCT.
Although SAS (SAS Institute Inc 2003), Stata (StataCorp. 2007), and SPSS (SPSS Inc 2006) do not include embedded meta-analysis commands, some user-written commands and/or macros exist that deal with this shortcoming -some more successfully than others (Egger, Sterne, and Smith 1998).Alternatively, a researcher may use more user-friendly specialist meta-analysis software packages, like MetaWin (Rosenberg, Adams, and Gurevitch 2007), the free but potent RevMan (The Cochrane Collaboration 2008) or the new, promising and also free MIX (Bax, Yu, Ikeda, Tsuruta, and Moons 2006).However, in most of the above, the outcomes of the included studies that will be 'fed' into the models must all be expressed in identical format (eg group means and standard deviations; odds ratios and sample sizes) for all the studies.Therefore, the researcher must go through a preliminary process of transforming study outcomes that may have been disseminated using a variety of statistical parameters (eg means and SDs; t-values; p-values) to the common format, a tedious task that requires at least some statistical adeptness.
Since the meta-analyst will often use a spreadsheet to summarise the reported study outcomes and other details, in order to better organise the analysis, we developed an add-in for Microsoft Excel that automates many meta-analysis processes and provides support for the task.The first purpose of our software is to calculate an effect size and its standard error, from the specific combination of input parameters supplied by the user for each outcome, using one of the methods described by the Cochrane Collaboration (Higgins and Green 2006).
The methods that can be used with each set of input parameters are automatically identified and the one that produces the most precise estimate of the effect is selected.Following that, a forest plot is created (Lewis 2001) summarising all the outcomes organised by study.Finally, various meta-analysis models, the more advanced of which (maximum likelihood, profile likelihood and permutations method) are not available in any other meta-analysis software package even though they have been proved to be more robust for normally distributed effects (Brockwell and Gordon 2001;Follmann and Proschan 1999), are used to calculate an overall mean effect and its variance.Results of the models are displayed in a second forest plot, while a variety of heterogeneity measures are provided to help the user decide on the appropriate model for the analysis at hand.

Data entry
After the add-in has been installed a meta-analysis menu will be available on the Microsoft Excel menu-bar.Once the worksheets have been formatted using the Format Sheets command, data can be entered on the first sheet (Figure 2) -a maximum of 10 meta-analyses can be accommodated in a workbook but we will refer to a single one for simplicity.The fields on the data-entry worksheet are described in Table 1.
Figure 3: Meta-analysis add-in menu.
Once data has been inputted all processes are executed by selecting one of the two available meta-analysis options that will be explained in more detail later: Include all secondary outcomes or Secondary only when no primary ones (Figure 3).Table 1: Fields in the data-entry worksheet.Fields marked with an asterisk are required for an effect size calculation method or more, but not all.The method(s) involved are shown in brackets.

Effect size and standard error calculation methods
A description of the eight methods that are used for the effect size and SE calculations is provided in this section, all inferred from The Cochrane Collaboration Handbook for Systematic Reviews of Interventions version 4.2.6,Section 8.5 (Higgins and Green 2006).Methods have been labelled 1a, 1b, 2, 3, 4, 5, 6, and 7.The first three deal with dichotomous data, the next four with continuous and the last method applies to both types.Each method can compute an effect size and variance from a different combination of available statistical parameters (sample sizes, means, t-values, etc).Table 2 provides information on the specific parameters that are needed as input for the application of each method.
The items that appear on Table 2 are described below: NIa is the size of the intervention group.
NCa is the size of the control group.
Ia is the number of events in the intervention group (always less than NIa).
Ca is the number of events in the control group (always less than NCa).
OR is the Odds Ratio: CI95 (OR) is the 95% Confidence Interval for the Odds Ratio (fields lCI95 (OR) and uCI95 (OR) in the data sheet).
MD is the means difference of the two groups, either provided or calculated with: MD = mean(Ia) − mean(Ca).
CI95 (MD) is the 95% Confidence Interval for the means difference (fields lCI95 (MD) and uCI95 (MD) in the data sheet).CI95 (Ia) is the 95% Confidence Interval for the mean of the intervention group (fields lCI95 (Ia) and uCI95 (Ia) in the data sheet).
CI95 (Ca) is the 95% Confidence Interval for the mean of the control group (fields lCI95 (Ca) and uCI95 (Ca) in the data sheet).
p-value is the p-value of the test.
t-value is the t-value of the test.
df the degrees of freedom of the t-test.
For some outcomes enough data is provided for the application of more than one of the methods.In such cases, the effect size and SE are calculated using all possible 'options' which enables the user to compare the results and the accuracy of the information supplied by the study in question (Figure 4).Nevertheless, only one method is finally selected to provide us with effect sizes and standard errors for the plots and the meta-analysis.The methods have been prioritised according to expected precision: that is, the expectation that the effect size and associated variance computed from the input data will be accurate.As a general rule, the fewer the number of mathematical transformations involved in getting from the 'raw data' to the statistical parameters used as input for the method, the higher the expected precision.Priority orders are displayed in  choice when data is dichotomous, C3 to the third option when data is continuous etc) and the 'best' available option is automatically selected.In many studies, as a result of rounding etc the reported p-value is not very precise; therefore methods 6 and 7 are only used when no other method can be employed.Since 1a and 1b require the exactly same input, the former is arbitrarily prioritised over the latter, which -in effect -is never used in further analyses but is provided for comparison.The methods are described in detail in Appendix A.

Results summary
Once effect sizes and standard errors have been computed, we select the most precise result available for each outcome and summarise the results using a forest plot type graph (Figure 5).Forest plots normally display information on a single outcome from each study along with an overall effect estimate at the bottom of the plot.However, in our summary forest plot all outcomes for which an effect and a SE could be computed are included, so that the reader can perceive a general overview of all the outcomes of interest (Figure 6).

Meta-analysis methods overview
A primary concern for meta-analysts is statistical heterogeneity in true effect size across the studies included in an analysis, often attributed to clinical and/or methodological diversity (Higgins and Green 2006).More specifically, clinical heterogeneity describes variability that arises from different populations, interventions, outcomes and follow-up times, while methodological heterogeneity is related to differences in trial design and quality (Thompson 1994).
If the variation among the evaluated true effects of individual studies is not above that expected by chance (homogeneity not rejected) researchers usually select the fixed-effect model (Brockwell and Gordon 2001) to combine the separate estimates into a single result.However, medical research studies, even into the same issue, can vary on so many factors that homogeneity is a rare commodity and some degree of variability between studies may be anticipated (Thompson and Pocock 1991).The best viable approach in such cases is to summarize the results of the heterogeneous studies using a random-effects model (DerSimonian and Laird 1986).Models of this family take into account the identified between-study variation, estimate it and generally produce wider confidence intervals for the overall effect than a fixed-effects analysis.
As an alternative to frequentist random-effects meta-analysis, a researcher may choose a Bayesian approach to estimation of the between-study variance (Smith, Spiegelhalter, and Thomas 1995;Sutton and Abrams 2001).Unlike most conventional random effects methods Bayesian methods attempt to model the uncertainty in the between-study variance, but require specification of a prior distribution, a sometimes difficult task that is particularly crucial when the number of studies included in the meta-analysis is small Lambert, Sutton, Burton, Abrams, and Jones (2005).
We have implemented seven frequentist meta-analysis methods in the add-in, for calculating a mean effect estimate and its confidence intervals (Figure 7): t-test (T), fixed-effects model (FE), DerSimonian and Laird random-effects model (DL), Q-based method (Q), maximumlikelihood random-effects model (ML), profile-likelihood random-effects model (PL) and permutations method utilising a DL random-effects model (PE).Where multiple primary outcomes are available for a study, their median is used in the models.If no primary outcomes are available, the secondary outcomes median is used instead.
In addition, a variety of heterogeneity measures are provided to help the user decide on the fixed or random effects approach (either DL, PL or ML; PE and T are special cases).Cochran's Q provides a p-value for the test of homogeneity, when compared with a χ 2 k−1 distribution (Brockwell and Gordon 2001), where k is the number of studies.However the test is known to be poor at detecting heterogeneity since its power is low when the number of studies is small (Hardy and Thompson 1998).I 2 is deemed to be more reliable in assessing inconsistency between studies, with values of 25%, 50% and 75% corresponding to low, moderate and high heterogeneity respectively (Higgins, Thompson, Deeks, and Altman 2003).H 2 M is the Figure 8: Forest plot on median outcomes and meta-analysis models.
only measure that is truly independent of k and it takes values in the [0, +∞) range with 0 indicating -a somewhat worrying -perfect homogeneity (Mittlbock and Heinzl 2006).It has also been suggested that the estimate of between-study variance, τ 2 , should be reported (Higgins and Green 2008), therefore the estimated between-study variance, where relevant to the model, is also displayed.
The MA methods and heterogeneity measures along with certain prerequisite assumptions are described in the following section, while the resulting forest plot (which again uses the median for studies with multiple primary outcomes) can be observed in Figure 8.

General
Let us consider a group of studies, whose overall effect we wish to estimate.The simplest way to estimate a confidence interval for the population mean is to treat the effect for each study as a single observation from a population of studies, and apply the usual Normal or T distribution assumptions.
where X is the sample mean of the effect sizes, s their standard deviation and z, t n−1 are the critical values for the two distributions.
This method, although simple, is thought to be deficient compared to the more advanced techniques since it does not take into account the variances on the effect estimates within the studies -provided they have been reported, which is not always the case.Nevertheless, it has been evaluated as a reliable alternative in performing meta-analyses, particularly if control for type I error rate is important (Follmann and Proschan 1999).This finding was supported by a simulation study that compared the performance of meta-analysis methods across effects of various distributions (Kontopantelis and Reeves 2008).

Fixed effects
If the within study variances are available, a researcher can take these into account in the estimation of an overall effect µ and a confidence interval.The fixed-effects model assumes that within-study variances may differ, but that there is homogeneity in effect size across all studies.It can be defined as: where, for study i: Y i is the effect size estimate, θ i the true effect size and e i the random error.Under the fixed effects approach, the true study effects are all assumed to be equal ) and the only deviations from the true effect size occur because of errors e i , representing the imprecision of results within each study.Those errors are assumed to be independent and normally distributed with mean zero and variance σ 2 i , therefore the estimated effect sizes Y i are also normally distributed with variance σ 2 i , but with mean θ i .The fixed effect estimate of the overall effect µ is usually calculated as a weighted average, using the within study variances σ 2 i as precision weights (Brockwell and Gordon 2001): where ŵi = 1/σ 2 i and var(μ F ) = 1 k i=1 ŵi .

Random effects
Often the homogeneity assumption is unlikely and variation in the true effect across studies is assumed, which is attributed to differences in design, circumstances, populations of participants and dose/type of treatment offered (Van den Noortgate and Onghena 2003).Whenever heterogeneity is identified with Cochran's Q -test (Cochran 1937) or other appropriate statistical measures like I 2 or H 2 M , a random effects model may be preferred over the fixed effects model.More specifically I 2 = 100 Q − df Q (Higgins and Thompson 2002) and (Mittlbock and Heinzl 2006) are both measures that use the relation of the between-to the within-study variance to evaluate heterogeneity.
The random-effects approach assumes that true effects θ i are normally distributed.This incorporates a second error term in (3), which accounts for across study variability, and the model becomes (Brockwell and Gordon 2001): In this case, the overall effect estimate is provided by: where . This model, by assuming that the studies at hand are a random sample out of a population of studies and modelling the variability between them, provides wider confidence intervals for the estimate of the overall effect and is generally considered to be a more conservative approach than the fixed effects model (Poole and Greenland 1999).
The variance parameter τ 2 , of the between studies error term i , is a measure of the between study heterogeneity.Since it is rarely -if ever -known, it needs to be estimated.DerSimonian and Laird's estimator for τ 2 is the method most widely used (DerSimonian and Laird 1986).However, the overall effect and the between study variance can also be estimated with methods such as the simple maximum likelihood and the profile likelihood proposed by Hardy and Thompson (Hardy and Thompson 1996).The simple maximum likelihood estimates μ mi and τ 2 mi can be obtained from the log-likelihood function in (8).
The log-likelihood function is solved iteratively for the simple maximum likelihood method, and the estimated variances are generally more accurate than the ones provided by the DL method.The profile likelihood method is also iterative, but provides confidence intervals for the estimates by taking into account the fact that when estimating one parameter the other one is also unknown and needs to be estimated as well.Computationally, it can be described as more advanced version of maximum likelihood which converges to a solution using nested iterations.Simulations show that this produces more accurate confidence intervals than other methods (Kontopantelis and Reeves 2008).For more details on these methods see Brockwell and Gordon (2001).
Another proposed random-effects method has been described by Follmann and Proschan (1999), which we have conveniently labelled the permutations method.The method can be described in three steps.First, a matrix of all possible combinations of study outcomes from a meta-analysis is created by permuting the effects signs.Then the DL method is used to compute the overall effect for each combination.Finally, using the created distribution for MetaEasy: A Meta-Analysis Add-In for Microsoft Excel the overall effect estimate μpe and the hypothesis that µ is zero, a Confidence Interval for the overall effect is calculated.In other words, rejection (or not) of the hypothesis is decided on the position of the observed mean estimate within the distribution.
Since more often than not Cochran's Q-test is the main determinant in the fixed or randomeffects model selection, we also include a mixed approach that combines these two models according to the outcome of the Q-test.If the homogeneity hypothesis is rejected, the DL random-effects model is selected; otherwise we choose the fixed-effects model.In accordance with Brockwell and Gordon (2001) we label this method 'Q'.All models (except T and PE, for which it does not apply) use the inverse variance method to calculate the overall effect and standard error of a meta-analysis.

Validation
Methods and graphs were validated using eight published meta-analyses whose full datasets were readily available (Laurant, Reeves, Kontopantelis, Hermens, Braspenning, Grol, Wensing, and Sibbald 2008).Validation was performed with Stata version 9.2 (StataCorp.2007) and more specifically programs metan version 2.34 (Harris, Bradburn, Deeks, Harbord, Altman, and Sterne 2008) and meta version 2.06 (Sharp and Stern 1997).Results agreed completely for the methods that are available in both platforms: fixed-effect model (inverse variance), random-effects model (DerSimonian and Laird, inverse variance), Cochran's Q and I 2 .Comparisons for the more advanced maximum likelihood, profile likelihood and permutation methods could not be performed, since they are not included in any of the statistical packages considered in this paper.

Using MetaEasy
The add-in is freely available for download from the National Primary Care Research and Development Centre's web page (http://www.npcrdc.ac.uk/) and the first author's personal web page (http://www.statanalysis.co.uk/).It is provided as an installer executable file, along with a short manual and data examples.Once it has been installed a meta-analysis menu option will always be available on the menu bar (Figure 3).The first step in creating a meta-analysis workbook is to create and format the worksheets appropriately.This is accomplished with the Format Sheets command which creates five worksheets for a single meta-analysis.All data entry is done on the first sheet: the remaining four sheets are 'locked' and are where the results and graphs are output.
After the data has been input, obtaining the results is an one click process, using one of the two available options: (i) include all primary and secondary outcomes in the meta-analysis models or (ii) only include secondary outcomes where a study has no primary outcomes.Studies frequently collect, analyse, and report data on a number of outcomes relevant to a given meta-analysis.Usually a well-designed study provides a precise description and definition of objectives and outcomes.This allows the outcomes to be classified as either primary or secondary: the former address the focus of the study while the latter allow for the investigation of subsidiary questions (Gebski, Marschner, and Keech 2002).If the study is not clear about which outcomes were regarded primary, then they should all be treated as secondary.
In addition, the reported heterogeneity measures help the user select between the fixed-effects and a random-effects model.Details on the characteristics and usage of the measures have been provided in Sections 5 and 6.
Since copying and pasting the results from Excel to a text editor is not always fully successful, graphs and blocks of cells can be exported as Graphics Interchange Format (GIF) files using the Export graph command.The code for this command, which can be used independently to export any graphs and cell ranges, was obtained from various websites and authors (Staff 2004;McRitchie, Bullen, and Peltier 2008) then edited and integrated into the add-in.However, the code was created in the Microsoft Office 2003 suite and it has not yet been tested with other versions.

Conclusions
In this article we have described an efficient way to perform meta-analyses in Excel.Some of the advantages of our module over established statistical and meta-analysis software packages include: Ease of use.
Time saving, since effect sizes and standard errors are calculated automatically from whatever statistical parameters are available, without the need for prior transformation to a common metric.
More robust calculation of the effect sizes and SE , since the 'best' method is automatically selected.
The extracted data from each study are easily accessible, they can be quickly edited or corrected and the analysis repeated.
Includes a choice of seven meta-analysis models (instead of the usual two).
Provides three advanced methods, maximum likelihood, profile likelihood and permutation, not currently available in other software packages.
Effect sizes and standard errors can be exported for use in other MA software packages.
Provides a descriptive forest plot allowing multiple outcomes per study.
Provides the final forest plot with the estimated mean effect and confidence interval.
Reports a wide range of heterogeneity measures.
The graphs can be easily edited and exported.
It is free (provided Microsoft Excel is available).
We will aim to provide additional tools with future versions of the module (e.g., a funnel plot, Mantel-Haenszel method for binary data), as suggested by user feedback.
MetaEasy: A Meta-Analysis Add-In for Microsoft Excel Needed NIa, NCa, OR, CI95 %(OR) Step 1 Calculate 'absolute' confidence intervals for Standardised Mean Difference: Step 2 SE effect = 1 NIa + 1 NCa Step 3 SD = SE diff /SE effect Step 4 effect = MD/SD Step 5 CI95 %(effect) = effect ± 1.96 • SE effect Table 6: Method 3 (based on mean difference and its confidence interval): Continuous data.* Instead of the CI95 %(MD) the SE diff (MD) may be provided instead.† Excel function tinv returns the t-value for specific alpha and degrees of freedom.
Figure 5: Forest plot on all outcomes.

Table 2 :
Effect calculation methods by input parameters needed.Method priority is described in the text.
SEdiff is the Standard Error of Difference between the means of the two groups.SD(Ia) is the Standard Deviation for the intervention group.SD(Ca) is the Standard Deviation for the control group.

Table 2
(e.g., D1 refers to the first