Comparison of ELISA and flow cytometry for measurement of interleukin-1 beta, interleukin-6 and tumor necrosis factor- α

Background: Although majority of the previous studies have shown a good correlation between enzyme linked immuno sorbent assay (ELISA) and flow cytometry in terms of cytokines, two laboratory methods usually were compared with the regression analysis and correlation in the literature. This study aimed at comparing the ELISA and flow cytometry assay for measuring cytokines by using two different statistical methods, regression analysis and Bland-Altman plot. Materials and methods: Fifty patients, diagnosed with hypercholesterolemia and expecting high level serum cytokines, and 30 healthy volunteers, expecting normal level serum cytokines, were enrolled in the study. The inter-leukin-1 beta (IL-1 β ), interleukin-6 (IL-6) and tumor necrosis factor-alpha (TNF- α ) were measured using ELISA, and compared with obtained levels using flow cytometric method. Results: Although regression analysis showed that the two methods are compatible with measurements of IL-1 β , IL-6 and TNF- α , they tended to show dissimilarity with measurements of IL-1 β and TNF- α based on Bland-Altman graphs. Conclusion: According to Bland-Altman plot, our results providing evidence of ELISA and flow cytometry assays were compatible with each other for IL-1 β and IL-6 measurements compared to TNF- α measurement. However, our study has a small number of participants, hence this study need to be confirmed by investigations involving more participants.


Introduction
Cytokines are crucial immunoregulators that act an important part in hematopoiesis, inflammation; and they display great importance for the diagnosis and follow-up of many diseases [1]. Cytokines are small, soluble proteins which act as immune mediators and together form the communication network for the immune system. Therefore, cytokine profiling may provide precious mechanistic data for many inflammatory and immune disease states [2]. Typically, cytokines including interleukin-1 beta (IL-1β), interleukin-6 (IL-6) and tumor necrosis factor-alpha (TNF-α) have been measured by enzyme linked immuno sorbent assay (ELISA) for years. This well-developed methodology necessitates significant sample volumes for each analyte, which is labor intensive, time consuming, but limited to single analyte at a time; however, it is generally cost-effective [3,4]. These limitations are circumvented by flow cytometry, an optimized method of intracellular cytokine detection in specific cells [5]. The implementation depends on the capacity of the flow cytometer to detect molecules either within or on the surface of a cell or microbead supports [3][4][5][6][7][8][9]. However, it is possible that every researcher may not have access to this optimized method or may have to choose ELISA for some reasons.
When investigators need to change one method for another one, or assess a new or alternative method, or quite simply they have a concordance problem between two instruments, they are in need of some tools to measure and appraise the differences as well as the causes of these differences [10]. As new technologies for the detection of cytokines continue to emerge, it is necessary to evaluate them carefully for clinical utility [11]. There are many studies comparing ELISA and flow cytometry, which are used for a long time in the measurement of cytokine levels [12][13][14]. For example, Carson and Vignali compared ELISA and flow cytometric methods in terms of 15 cytokines, and they showed that both methods are compatible with IL-1β and TNF-α and flow cytometric method which is more sensitive [12]. Chen et al. found that correlation coefficient (r) of the two methods is 0.97 and flow cytometry has statistically significant lower coefficient of variation values than those of ELISA [13]. Kellar et al. noted that there were significantly high correlations between the two methods in terms of TNF-α [14].
Laboratory methods are usually compared by researchers using regression and correlation analyzes [15]. Indeed, when previous studies on cytokines are examined, it is also seen that great majority of the investigations give the product-moment correlation coefficient between the results of the two measurement methods as an indicator of compliance [1,3,[12][13][14]. However, correlation analysis is aimed for two different variables to determine a nonlinear trend, and r measures the strength of the linear connection between the two variables [10,11]. Moreover, these statistical methods are criticized for producing generally overestimated measures or may give totally erroneous results in some specific cases [14]. Therefore, correlation analysis is criticized for usually generating overestimated measures or may give totally erroneous results in some specific cases [15,16].
An alternative method for comparing these two methods is Bland-Altman method which supplies an intuition to assess whether both can be employed alternatively or not [10,16]. Bland-Altman method depends on visualization of variances in measurements employing a graphical approach to plot the difference against the mean of the measurements [14]. Bland-Altman method computes the average difference between two methods of measurement and standard deviation (SD) of the difference, and calculates '95% limit of agreement' as the mean difference. The presentation of '95% limit of agreement' on Bland-Altman plot permits visual decision of how well two measurement methods are in concordance with each other. Smaller range between the limit might be commented as better compliance [15][16][17][18].
Although majority of the previous studies have shown a good correlation between ELISA and flow cytometry in terms of cytokines, two laboratory methods usually compare the regression analysis and correlation in the literature [4,[12][13][14]. Nevertheless, there have been no studies comparing ELISA and Flow methods, which have been used for many years in cytokine measurement, with Bland-Altman plot. Therefore, we aimed at comparing ELISA and flow cytometry assays for measuring IL-1β, IL-6 and TNF-α levels by using not only linear regression but also Bland-Altman plot, which is based on the quantification of the agreement between two quantitative measurements by studying the mean difference and constructing limits of agreement.

Study groups
The study group included 50 patients aged 18-65 diagnosed with hypercholesterolemia and 30 healthy subjects who appeared at the outpatient clinic of Faculty of Medicine, Erciyes University in Kayseri, Turkey. The study protocol was approved by the Ethics Committee of Erciyes University (05.02.2013-2013/119). LDL limit was determined according to the National Cholesterol Education Program, European Society of Cardiology, and European Atherosclerosis Society, Current Medical Diagnosis and Treatment 2013 [19]. Patients were informed about the research and they signed the "informed consent form" according to Helsinki Declaration. The subjects with renal failure, coronary artery disease, liver disease, immune system diseases, diabetes, systemic disease, hypertension, and receiving treatment via lipidlowering drugs for any reason were excluded from the study. Together with patients with similar age and gender distributions, 30 healthy volunteers who had no systemic diseases, and were nonsmokers, and did not use any drugs and whose body mass indexes were within the range of 18.5-24.9 kg/m 2 were included in study.

Sampling and storage
Blood samples were drawn into one CBC tubes containing ethylenediaminetetraacetic acid; yet, one of the tubes was without anticoagulant by the venipuncture technique after an overnight (≥12 h) fast. Blood specimens were allowed to clot for 30 min. Then, tubes were centrifuged at 2000 × g for 10 min at 4°C and aliquots of samples were stored at −40°C for measurement.

Determination of IL-1β, IL-6 and TNF-α by flow cytometry
Concentrations of IL-1β, IL-6 and TNF-α were determined using the BD CBA Flex Set System and the BD Human Soluble Protein Master Buffer Kit (Catalog number: 558264, BD Bioscience-PharMingen, San Diego, CA, USA). This kit allows the simultaneous measurement of levels of IL-1β, IL-6 and TNF-α in a single sample. The kit included three bead populations with distinct fluorescence intensities. The beads were coated with capture antibodies specific to IL-1β, IL-6 and TNF-α (Catalog numbers: 558279, 558276, 558273, respectively) and mixed with phycoerythrin (PE) conjugated detection antibodies. These capture beads were incubated with recombinant standards or test samples to form sandwich complexes. The cost-benefit analysis was resolved in the FL-3 channel of a FAC-SCalibur flow cytometer (BD Bioscience-Pharmingen), and the results were produced in graphic and tabular formate employing cost-benefit analysis software (BD Bioscience-Pharmingen). The assay sensitivities for IL-1β, IL-6 and TNF-α were 0.8, 12.6 and 3.4 pg/mL, respectively.
The tests were conducted and analyzed in line with the producer's instructions. For determination of IL-1β, IL-6 and TNF-α, the blood samples were run without predilution. Briefly, 50 μL of each sample was mixed with 50 μL of mixed capture beads and 50 μL of the human inflammation PE detection reagent consisting of PE-conjugated anti-mouse IL-1β, IL-6 and TNF-α. The samples were incubated at room temperature for 2 h in the dark. After incubation with the PE detection reagent, the samples were washed once and resuspended in 300 μL of wash buffer before acquisition on the FACSCalibur (Becton Dickinson). The BD FACSCalibur flow cytometer (BD Bioscience) was calibrated with setup beads, and 300 events were gained for each factor and each sample. Individual analyte concentrations were demonstrated by their fluorescence intensities and computed with their respective standard reference curve and FCAP array software [20,21].

TNF-α
The serum TNF-α levels were detected using a commercial ELISA kit (Boster: EK0525) according to the manufacturers' protocols. Boster's rat TNF-α ELISA kit was based on standard sandwich ELISA technology. A monoclonal antibody from mouse specific to TNF-α was precoated onto 96-well plates. Standards and test samples were added to the wells, and afterwards a biotinylated detection polyclonal antibody from goat specific to TNF-α was added, and then the process continued by washing with PBS or TBS buffer. Avidin-biotin-peroxidase complex was added and unbound conjugates were washed away with PBS or TBS buffer. HRP substrate TMB was utilized to visualize HRP enzymatic reaction. TMB was catalyzed by HRP to generate a blue color product that converted into yellow after adding acidic stop solution. The density of yellow is proportional to human TNF-α amount of sample captured in plate.

IL-1β and IL-6
The serum levels of IL-1β and IL-6 were detected using commercial invitrogen brand ELISA kits (catalog number for IL-1β: KHC0011 and for IL-6: KHC0061) according to the manufacturers' protocols. These kits use ELISA based monoclonal antibodies specific to Human IL-1β; and Human IL-6 are coated onto the wells of the microtiter strips provided. Samples, including standards of known IL-1β and IL-6 content, control specimens, and unknowns, were pipetted into these wells. During the first incubation, the IL-1β and IL-6 antigens bound to the immobilized (capture) antibodies on one site. After washing, a biotinylated monoclonal antibodies specific to IL-1β and IL-6 were added. During the second incubation, this antibody bound to the immobilized IL-1β and IL-6 captured during the first incubation. After removal of excess second antibody, streptavidin-peroxidase (enzyme) was added. This bound to the biotinylated antibody to complete the four-member sandwich. After a second incubation and washing to remove all unbound enzyme, a substrate solution was added, which acted upon by the bound enzyme to produce color. We measured the absorbance of each well at 450 nm after having added 100 μL of stop solution. The intensity of this colored product was directly proportional to the concentration of IL-1β and IL-6 present in the original specimen.

Statistical analysis
Statistical analyses were performed using statistics programs with IBM SPSS Statistics 20.0. The normality of the data was assessed by the Shapiro-Wilk normality test and Q-Q graphs. Data were expressed as numbers for categorical variables and mean ± SD or median (25 th -75 th percentile) for continuous variables. Comparisons between groups were performed using the Mann-Whitney U-test (non-normal distribution) and independent samples t-test (normal distribution). Gender comparison was made with the exact method of the χ 2 -test. Accordance of ELISA and the flow cytometric method was assessed by Passing and Bablok regression, intra class correlation coefficient (ICC) and Bland-Altman analysis in MedCalc 12.7.5.0 program. Passing and Bablok regression results interpretation: "Regression equation (y = a + bx) revealed constant [regression line's intercept (a)] and proportional [regression line's slope (b)] difference with their confidence intervals of 95% (95% CI). CI explain if their value differ from value zero (0) for intercept and value one (1) for slope only by chance. Thus, if 95% CI for intercept includes value zero it can be concluded that there is no significant difference between obtained intercept value and value zero and there is no constant difference between two methods. Respectively, if 95% CI for slope includes value one, it can be concluded that there is no significant difference between obtained slope value and value one and there is no proportional difference between two methods" [22]. Statistical significance was set at 0.05.

Results
The mean ages of patients were 46.94 ± 9.11 and of the controls were 46.26 ± 8.11. There was no difference between patients and controls in terms of age, gender, weight, height and BMI values (Table 1). Clinical and laboratory parameters of the study population are given in Table 1.
When measured with ELISA and flow cytometry methods, IL-1β, IL-6 and TNF-α levels in patients were found to be significantly higher (p < 0.001) than those of the control group (Table 2). There was no significant difference (p > 0.05) between male and female gender groups in terms of ELISA and flow methods for cytokines.
When an in-class correlation analysis was performed to examine the agreement between ELISA and flow cytometry in terms of IL-1β, ICC of ELISA and flow cytometry was found to be 0.9975, and it was observed that there was an agreement between the two methods ( Figure 1A). Although both methods are in agreement with ICC result, Passing and Bablok regression analysis results showed that there is a minimal proportional error for IL-1β. Passing and Bablok regression results demonstrated that there is no constant error between the values obtained by ELISA and flow cytometry methods, but a proportional bias is observed in terms of IL-1β. On the other hand, it was found that Bland-Altman graph supports Passing and Bablog results and there is a proportional error in terms of IL-1β. Figure 1B shows that the differences were observed between −3.3 and −0.1. The difference between ELISA and flow cytometry is usually below zero point ( Figure 1B). However, it was observed that as the mean values increased, the difference steadily decreased in general. The IL-1β levels were measured by flow cytometry as 0−2% more than the ELISA method at lower values of IL-1β. At high values of IL-1β, the IL-1β levels were measured by flow cytometry as 2-4% more than the ELISA method. Thus, as a result, a constant error is present between ELISA and flow cytometry in terms of IL-1β.
For measurement of IL-6, it was observed that there is no proportional error between the values obtained by ELISA and flow cytometry methods, but a constant error is observed in terms of IL-6. When an intra-class correlation analysis was performed to examine the agreement between ELISA and flow cytometry in terms of IL-6; ICC of ELISA and flow cytometry was found to be 0.9979, and there was an agreement between the two methods ( Figure 2A). For the same parameter, the differences were between −2.8 and 0.02 ( Figure 2B). Difference between ELISA and flow cytometry is usually below zero point. In this case, the IL-6 levels are measured by flow cytometry in an average of 1.39 units more than the ELISA method. Thus, a constant error is present between ELISA and flow cytometry in terms of IL-6.
For measurement of TNF-α, there is no constant error between the values obtained by ELISA and flow cytometry methods, but a proportional bias is observed in terms of TNF-α. When an in-class correlation analysis was performed to examine the agreement between ELISA and flow cytometry in terms of TNF-α; ICC of ELISA and flow cytometry was found to be 0.9215, and there was an agreement between the two methods ( Figure 3A). For the same parameter, Figure 3B indicates that there were differences between −18.3 and −4 on Bland-Altman average difference graphics. The difference between ELISA and flow cytometry was below zero point. However, it was observed that as the mean increased, the difference steadily decreased in general ( Figure 3B). In this case, the TNF-α levels are  measured by flow cytometry in an average of 11.2 units more than the ELISA method. Thus, as a result, a constant error is present between ELISA and flow cytometry in terms of TNF-α.

Discussion
In current study, we compared the ELISA and flow cytometry assay for measuring the levels of IL-1β, IL-6 and TNF-α using regression analysis and Bland-Altman plot.
Our results demonstrated that trends for the levels of IL-1β, IL-6 and TNF-α had correlation between ELISA and flow cytometry method. However, Bland-Altman analysis showed that the limits of agreement for IL-1β and IL-6 were small enough to be confident. The availability of methods to measure cytokines, which are crucial for fighting off infections and for other immune responses, is critically important. ELISA is the most widely used and best validated method, but it measures only a single cytokine in each sample [16]. This explains the high reagent costs, excessive technician time, and large sample volumes necessary to produce each result. The capacity to detect and quantitate many cytokines simultaneously on the same sample via a robust flow cytometry would decrease these costs and enhance efficiency [4,23]. Nevertheless, the flow cytometric assay becomes cheaper than ELISA only if six or more cytokines are measured simultaneously, as the number of reagents used for per cytokine assay can be significantly reduced [12].
ELISA kits, used in our study, for IL-1β, IL-6 and TNF-α (from Invitrogen Brand and Boster) are reported to have sensitivity under 1 pg/mL, 2 pg/mL and 1 pg/mL, respectively. Furthermore, flow cytometry kits used in our study, for IL-1β, IL-6 and TNF-α (from BD biosciences) are reported to have enhanced sensitivity at an approximate value of less than 0.3 pg/mL. In the typical double antibody sandwich ELISA, antibody stuck to the bottom of a  well provides not only antigen capture but also immune specificity, while another antibody linked to enzyme supplies detection and an amplification factor. This approach enables accurate and sensitive detection of the antigen, the cytokine of interest [4]. On the other hand, the use of fluorescence in flow cytometry assay, rather than colorimetric measurements, may provide more sensitivity, specificity and a broader range for cytokines quantitation [24]. Medical scientists frequently need to compare two methods of measurement, or a new method with a fixed one, to determine whether these two methods can be employed interchangeably or the new method can substitute the fixed one [15]. Although these differences are known, it has been suggested that good correlations were observed between ELISA and flow cytometry for cytokines in the literature [4,[12][13][14]. However, there have been many researchers who have reported a high correlation coefficient, and mistakenly interpreted the results as evidence of agreement between ELISA and flow cytometry for measuring cytokines [22,25,26]. Therefore, correlation coefficient is criticized for generally producing overestimated measures or may give generally erroneous results in some specific cases [10,16,27,28].
In our study, consistent with previous studies, ELISA and flow cytometry methods have compliance with IL-1β, IL-6 and TNF-α performed with regression analysis. However, Bland-Altman plot, which supplies an intuition to assess whether both can be employed alternatively or not, showed that this compatibility was not so easy. If a regression line was suited to Bland-Altman plot, it was argued that proportional bias occurred if the gradient of the slope substantially differed from zero [10,16]. Nevertheless, Ludbrook claimed that the existence of bias in the analysis was because of some kind of statistical possibility [29]. An approach employing least-products regression to fit the regression line in Bland-Altman plot has been alleged to remove the bias problem in Bland-Altman plots [10]. When it comes to the so-called 95% limits of agreement, it is obvious from the foregoing that only when there is no proportional bias, this technique can be utilized securely to decide whether one method complies well enough with the other so that either can be employed in clinical applications [10,29].
In our study, when an in-class correlation analysis is performed to examine the agreement between ELISA and flow cytometry in terms of IL-1β, IL-6 and TNF-α, it is observed that there is an aggrement between the two methods.
Kellar et al. measured IL-6 by means of both ELISA and flow cytometry; and they found that two methods were compatible with each other in terms of IL-6 measurement [14]. Carson and Vignali reported that the two methods were compatible with each other for measuring IL-6; and flow cytometry method was significantly more sensitive than ELISA [12].
In our study, Bland-Altman analysis demonstrated that the IL-1β levels were measured by flow cytometry as 4% more than the ELISA method. Since there is a proportional error that can be overlooked, it may be suggested that the two methods were high compatible with each other for IL-1β. Bland-Altman analysis also demonstrated that the IL-6 levels were measured by flow cytometry as 3.4% more than the ELISA method. Since the proportional error is low, it may be suggested that the ELISA and flow cytometry assays are compatible with each other for IL-6, but it should be taken into account that the constant error is 1.33. On the other hand, according to Bland-Altman analysis, there was constant bias between ELISA and flow cytometry for measuring the levels of TNF-α. The TNF-α levels were measured by flow cytometry as 31% higher than that of the ELISA method. Since the constant error is too low but the proportional error is high, it may be suggested that ELISA and flow cytometry assays were not compatible with each other for TNF-α measurement.
Bland-Altman analysis demonstrated that 95% of the differences between ELISA and Luminex values were within ±1.96 SD from the mean difference indicated by the 95% limits of agreement for the measurements of IL-6 and IL-8 [30]. They also found that there was no constant bias as a function of inflammation level. Thus, they suggested that the limits of agreement for IL-6 and IL-8 were small enough to be confident that Luminex can be used in place of ELISA.
According to our knowledge of literature, ELISA and flow cytometry assay were, for the first time, compared via this study by using Bland-Altman plot for measuring cytokines levels. In this study, Bland-Altman analysis demonstrated that the limits of agreement for IL-1β and IL-6 were small enough to be confident. As the number of samples increases, even the smallest differences between methods indicate the existence of a constant error. Nevertheless, this study has some limitations. Firstly, the small number of participants may have limited our findings to be generalized. Secondly, we collected only blood samples, while other body fluid samples were absent in the study. Lastly, the lack of limits of agreement knowledge between ELISA and flow cytometry for measurement of other cytokines such as IL-4, IL-10, IL-13, etc., still remains unclear. Therefore, further investigations are necessary in order to make a more comprehensive evaluation between the two methods for the measurement of cytokines.

Conclusion
In the present study, it was found that ELISA and flow cytometry complied with measuring the levels of IL-1β, IL-6 and TNF-α as a result of correlation analysis. Bland-Altman plot, which proposed an alternative analysis, based on the quantification of the agreement between two quantitative measurements by studying the mean difference and constructing limits of agreement, and showed that the TNF-α levels were measured by flow cytometry as 31% more than the ELISA method. According to the limits of agreement for IL-1β and IL-6, they were small enough to be confident, and it may be suggested that ELISA and flow cytometry assays were high compatible with each other for IL-1β while two methods were compatible with each other for IL-6 measurement. However, our study has a small number of subjects and parameters; so, the subject deserves further investigation.