Concordance among methods of nutritional assessment in patients included on the waiting list for liver transplantation

Background The aim of the present study was to determine the extent of malnutrition in patients waiting for a liver transplant. The agreement among the methods of nutritional assessment and their diagnostic validity were evaluated. Methods Patients on the waiting list for liver transplantation (n = 110) were studied. The variables were: body mass index, analytical parameters, liver disease etiology, and complications. Liver dysfunction was evaluated using the Child–Pugh Scale. Nutritional state was studied using the Controlling Nutritional Status (CONUT), the Spanish Society of Parenteral and Enteral Nutrition (SENPE) criteria, the Nutritional Risk Index (NRI), the Prognostic Nutritional Index (PNI-O), and the Subjective Global Assessment (SGA). Agreement was determined using the Kappa index. Area under receiver operator characteristic curves (AUCs), the Youden index (J), and likelihood ratios were computed. Results Malnutrition varied depending on the method of evaluation. The highest value was detected using the CONUT (90.9%) and the lowest using the SGA (50.9%). The pairwise agreement among the methods ranged from K = 0.041 to K = 0.826, with an overall agreement of each criteria with the remaining methods between K = 0.093 and K = 0.364. PNI-O was the method with the highest overall agreement. Taking this level of agreement into account, we chose the PNI-O as a benchmark method of comparison. The highest positive likelihood ratio for the diagnosis of malnutrition was obtained from the Nutritional Risk Index (13.56). Conclusions Malnutrition prevalence is high and prevalence estimates vary according the method used, with low concordance among methods. PNI-O and NRI are the most consistent methods to identify malnutrition in these patients.


Introduction
The presence of protein-calorie malnutrition is common among patients with liver disease. Malnutrition is one of many possible effects of liver disease. 1e3 This malnutrition is associated with the degree of hepatic dysfunction and increased morbidity both before and after transplantation. 4e7 Great variability in the prevalence of malnutrition has been observed, depending on the method used for assessment 5 and on the severity of the disease, which can change the body composition and analytical parameters. 6 Therefore, there is no consensus among authors on which are the most effective methods to assess the nutritional state of these patients.
Because there is not a gold standard, different tools must be used to quantify and classify malnutrition. 2,8 The most frequently used methods to assess the nutritional state in cirrhotic patients include anthropometric parameters and analytical parameters. 1,2,5 Other methods are the Subjective Global Assessment (SGA), 1,9 handgrip strength, 6,10 bioelectrical impedance analysis, 11 or carrying out dietary assessments using tools such as the Malnutrition Universal Screening Tool (MUST) or Nutritional Risk Screening 2002 (NRS-2002). 4 The European Society for Clinical Nutrition and Metabolism recommends evaluating malnutrition in patients with liver cirrhosis through tests, including subjective global assessment, anthropometry, biomedical impedance, and grip strength evaluation. 12 The American Society for Parenteral and Enteral Nutrition advises performing a nutritional screening using the Mastrich Index, Nutritional Risk Index (NRI), and Prognostic Nutritional Index or Index Onodera (PNI-O), among other tests, in the first 24 h after hospital admission for all adult patients. Furthermore, malnourished patients should be assessed using the Mini Nutritional Assessment (MNA) and SGA. 13 The Spanish Society of Parenteral and Enteral Nutrition (SENPE) recommends simple and understandable nutritional assessment methods that include clinical and laboratory parameters in all hospital-admitted adult patients. 14 SENPE criteria include anthropometric parameters (body mass index [BMI] and weight loss) and laboratory values (albumin, cholesterol, and lymphocytes).
Following the recommendations and due to the variability among methods, some authors have tried to identify the best tool to assess the nutritional state of these patients. Villalobos et al 15 11 assessed the nutritional status of patients with cirrhosis through anthropometric measurements, the SGA, hand grip strength, and bioelectrical impedance. The aim of their study was to identify the safest and most effective method for the assessment of the nutritional state.
This study was performed due to the significance of the nutritional state on morbidity and mortality of patients and the difficulty in determining a nutritional assessment method. We aimed to determine the malnutrition degree in patients on the liver transplant list, study the concordance among assessment methods, and explore the diagnostic validity compared to a benchmark that was chosen because it obtained the best overall concordance.

Setting
We conducted a cross-sectional study at University Hospital Complex in A Coruña, Spain performed during the period of January 2012 through December 2014.

Sampling and inclusion criteria
The inclusion criteria were patients over the age of 18 years who were included on the waiting list for liver transplant during the study period and who signed the informed consent.

Sample size calculation
Sample size was limited by both the duration of the study and the number of liver transplants per year. We evaluated 110 patients during the period of this study (2012e2014). This sample size (n ¼ 110) allow us to estimate characteristics related to nutritional state with a precision of ±10% and 95% confidence (a ¼ 0.05), assuming a prevalence of 50% of the variable of interest and 10% losses during follow-up. At the same time, this sample size allows estimation of the pairwise agreement among methods with a precision of ±16.50% and 95% confidence, assuming a Kappa index value around 0.400.

Data collection and measurements
The patients were identified with a code, keeping their personal data confidential. To obtain the necessary information for the study, clinical records were reviewed, a physical examination was performed, and the patients were interviewed. 17 The following variables were obtained by the time of inclusion on the waiting list for liver transplantation: 1. From clinical records: demographic data (age and gender), analytical parameters (albumin, cholesterol, total lymphocyte, bilirubin, creatinine, and international normalize ratio), transplant etiology (alcoholic cirrhosis, autoimmune hepatitis, polycystic disease, liver cancer, primary biliary cirrhosis, and viral cirrhosis) and the presence of hepatic complications (ascites, hepatic encephalopathy, digestive bleeding, bacterial peritonitis, and hepatorenal syndrome) 2. From physical examination: patient weight, height, and BMI 3. From interviews: the SGA 18 The degree of liver dysfunction was evaluated using the Child-ePugh Scale, 19,20 in which higher score indicate greater liver dysfunction. The assessment of nutritional state was performed using validated scales (Table 1): CONUT, 21 SENPE criteria, 14,22 NRI, 8,23 and PNI-O. 16 The CONUT and PNI-O scores were selected because they have been shown to be useful tools to assess the nutritional state in patients with chronic liver disease. 16 Two other method, the NRI and SENPE recommendations, have been selected based on the recommendations of international and national associations.
Finally, the SGA 18 was also performed, which consists of two parts: a review of clinical records (changes in weight and intake, gastrointestinal symptoms, functional capacity, and underlying disease) and a physical exam to assess the loss of fat-mass muscle, ascites or edema, and the presence of tongue or skin lesions. Patients were classified as well nourished, moderately malnourished or at risk of malnutrition, and severely malnourished. The SGA has been the preferred nutritional evaluation method for liver transplantation candidates.
Although some of these methods could be considered as nutritional screening tools (CONUT, NRI, and PNI-O), while others could be considered as assessment tools (SGA), all of them allow us to calculate the risk of malnutrition.

Statistical analysis
Descriptive analysis was performed for all variables. Continuous variables were reported using means and standard deviations (SD) or medians and interquartile ranges, as appropriate. For dichotomous/categorical variables, absolute numbers and percentages were computed, together with their 95% confidence intervals.
Concordance between the different scales that assess nutritional state was studied with the statistical Kappa (K) index. The overall Kappa was estimated, and the homogeneity test was performed to study the agreement amongst methods.
PNI-O was the method of the highest level of overall agreement. Taking this level of agreement into account, we chose the PNI-O as a benchmark method of comparison.
Diagnostic validity was assessed using the area under the curves (AUCs), the Youden index, sensitivity, specificity, positive and negative predictive values, and positive and negative likelihood ratios. The results of the CONUT and the NRI were compared with PNI-O diagnosis of malnutrition as the reference criteria and displayed on a receiver operating characteristic (ROC) curve. For each method, the ROC AUC was computed, and its optimum cut-off point was determined using the Youden index. The Youden index (J) is defined as the maximum vertical distance between the ROC curve and the diagonal or change line and calculated as J ¼ max (sensitivity þ specificity À 1). 24,25 Using the cut-off points determined by assessing the ROC curves, the positive and negative predictive values of the diagnostic tests were calculated. In order to determine the diagnostic accuracy of the tests, their likelihood ratios were calculated. 26 Statistical analysis was performed using SPSS for Windows (version 19.0, SPSS Inc., Chicago, IL, USA) and Epidat version 3.1 (Direcci on Xeral de Innovaci on e Xesti on da Saúde Pública, Xunta de Galicia y Organizaci on Panamericana de la Salud, La Coruña, Spain).

Ethics
The study complies with the principles laid down in the Declaration of Helsinki. Informed consent was obtained from all the participants in the study. Confidentiality was preserved in accordance with the current Spanish Data Protection Law (15/1999). This project was approved by the corresponding ethics review board (Clinical Research Ethical Committee of Galicia, #2010/081).

Results
The mean age of our patients was 56.85 (SD, 8.23) years, and most were men (72.73%). The BMI was !25 kg/m 2 in 58.2% of them. Mean ChildePugh score was 8.37 (SD, 2.12), and 48.6% of the patients showed a significant functional commitment (ChildePugh B). The main etiology was alcoholic cirrhosis (52.7%). The most frequent liver disease complication was ascites (79.1%) ( Table 2).

Prevalence of malnutrition according to the degree of liver dysfunction
The percentage of malnutrition at the time of inclusion in the transplant waiting list (Table 3) changed depending on the method of evaluation. The highest prevalence of malnutrition was estimated using the CONUT (90.9%), and the lowest prevalence was estimated using the SGA (50.9%). Examining the prevalence of malnutrition according to the ChildePugh classification, we found that the higher the hepatic dysfunction, the worse the nutritional state (p < 0.001) (Fig. 1).

Agreement between methods of nutritional assessment
Due to the variability in the prevalence of malnutrition depending on the method used, we calculated the level of agreement among different methods. Pairwise agreement among the methods ranged from K ¼ 0.041 to K ¼ 0.826, with an overall agreement of each criteria with the remaining methods between K ¼ 0.093 and K ¼ 0.364 (Table 4). PNI-O was the method with the highest level of overall agreement. Taking this level of agreement into account, we choose the PNI-O as our reference criterion to study the diagnostic accuracy with the rest of the methods.  16 10 Â serum albumin (g/dL) þ 0.005 Â Total lymphocyte (cells/mm 2 ) Values < 40 points SGA 18 Consists of two parts, medical history and physical examination.

Diagnostic accuracy to identify malnutrition compared to the PNI-O (reference criterion)
The Youden index, corresponding cut-off points, sensitivity, specificity, positive and negative predictive values, and positive and negative likelihood ratios are shown in Table 5. According to the values of Youden Index, the most accurate diagnostic method for malnutrition was the NRI, followed by SENPE and CONUT.
Using the NRI and the CONUT (quantitative methods), the cutoff point for malnutrition in patients with liver disease was found to be higher compared to the general cutoff point. The ROC curves for the methods of malnutrition analysis are displayed in Fig. 2. The AUC showed that the NRI was most accurate in predicting malnutrition (AUC, 0.987), followed by the CONUT (AUC, 0.847), which was moderately accurate.
From the ROC curves, the optimal cut-off points of these tests for diagnosing malnutrition in the total sample were 98.39 for the NRI and 3.50 for the CONUT, with sensitivities of 96.9% and 75.0%, respectively. The positive predictive values for the NRI and the CONUT were 98.9% and 96.0%, respectively. Negative predictive values were 81.3% and 31.4%, respectively. The NRI was found to have the highest positive likelihood ratio (13.56).

Prevalence of malnutrition
It is important to know the nutritional state of the patient, as it is a risk factor for morbidity and correlates with the severity of liver dysfunction, 27 both before and after transplantation. 5,7 To the alteration in analytical parameters due to the disease, 2,3 the   variability in the malnutrition prevalence is added, 2,8,17 as shown in the literature. 16,17 Among the different evaluation methods, anthropometric parameters, such as BMI, are used. In this study, there were no patients with malnutrition according to BMI, and more than half of the subjects were classified as overweight or obese. This result may be due to most of the patients having ascites, which involves a change in body composition. This finding is similar to those of other publications, which conclude that BMI underestimates malnutrition and is not a suitable method to evaluate these patients. 4,16 In the study by Villalobos et al, 5% of hospitalized patients (regardless of pathology) were classified with possible malnutrition according to BMI, a value which is very low in relation to those estimated using other evaluation methods. In conclusion, as BMI is a late indicator of malnutrition, 15 it was not included in this study to assess concordance or diagnostic validity. 16,17 The methods used in this study can be distributed into three groups: a) structured tests (e.g., the SGA); b) methods using only analytical parameters (CONUT and PNI-O); and c) methods using analytical and anthropometric methods (SENPE criteria and the NRI). The SGA and SENPE criteria showed lower prevalence estimates of malnourished patients (50.9% and 64.6%, respectively). The low prevalence of malnutrition estimated using the SGA is similar to that reported in other publications, 10,11,16 and this may be due to the subjective nature of this method, which underestimates malnutrition in these patients. 28 Tanaguchi et al believe that,  although the SGA is effective to assess the nutritional state in patients with other diseases, it is not a suitable screening tool for the nutritional evaluation of patients with liver disease. 28 We found a similar prevalence of malnutrition using the SENPE criteria to that reported by Villalobos et al. 15 The results can be related to anthropometric parameters, such as BMI and weight loss. These anthropometric parameters are affected by the possible presence of hydropic decompensation, and most of our patients had ascites.
The NRI, CONUT, and the PNI-O all showed higher malnutrition prevalence (89.1%, 90.9% and 87.3%, respectively) than SENPE criteria and SGA. This can be explained by the pathology of patients, which affects the values of albumin, cholesterol (which is synthesized by the liver), and lymphocytes (a parameter related to protein depletion and a malnutrition indicator). In addition, in this study, patients with mild malnutrition were considered malnourished, while only moderate and severe malnutrition patients were considered malnourished in other publications. 8,15,16 The association between degree of liver dysfunction and malnutrition is consistent with other publications. 16

Concordance of nutritional methods
This study shows that the concordance among the methods of nutritional assessment is not very high ( Table 2). The concordance obtained between the SGA and CONUT was low, while it was moderate in studies by Ulibarri et al and Hernandez-Escalante et al (K ¼ 0.488 and K ¼ 0.677, respectively). 21,29 The concordance between the CONUT and the NRI was moderate and similar to the findings of Gimeno et al (K ¼ 0.547). But the results of SENPE criteria with the NRI and the CONUT were weak for both of them and lower than the results obtained by Gimeno et al, with K values of 0.609 and 0.593, respectively. 8 Our results show a lower concordance than that reported in other publications. These findings may be related to the inclusion criteria, because we included patients with liver cirrhosis who were not considered in other published articles. 8,21,29 In methods that also include anthropometric parameters (SENPE and NRI), a lower prevalence of malnutrition was observed in comparison with the CONUT and PNI-O because the increase of weight is considered as an indicator of improved nutritional state. This situation is not realistic in our sample because most of the patients had hydropic decompensation (ascites and/or edema).
The method that showed the lowest level of agreement was SGA. This is reasonable, since it is the only method of those considered that does not include anthropometric and/or laboratory parameters. Furthermore, other authors have also demonstrated that SGA is not sufficient as a nutritional screening tool for patients with liver diseases, 28 arguing that SGA does not include any component to assess the capacity of hepatic metabolism. This is the reason that SGA could underestimate malnutrition prevalence.
The overall Kappa values, from highest to lowest, were for the PNI-O, the SENPE criteria, the NRI, and the CONUT. We decided to use the PNI-O as benchmark to determine the diagnostic validity of the other assessment methods because it showed the highest concordance.
Diagnostic validity of the methods in comparison with the PNI-O Although this study was not designed to identify a gold standard for malnutrition assessment in liver disease patients, we explored the results obtained to determine the diagnostic validity of each of the methods considered with respect to the method that showed the highest consistency (the PNI-O). Although the existence of reference bias cannot be ruled out, the results could be useful to identify suitable methods of assessing malnutrition in these patients.
Using the PNI-O as the reference method, the methods that showed a higher diagnostic validity for malnutrition according to the Youden Index were the NRI, SENPE criteria, and CONUT. The methods that can be considered good tools to identify malnutrition, due to having high sensitivity, are the SENPE criteria and the NRI (sensitivities of 100% and 96.9%, respectively). The methods that best identify the nourished patients, due to having high specificity, are the NRI and the CONUT (specificities of 92.9% and 78.6%, respectively).
Likelihood ratios were calculated to assess the accuracy and validity of the diagnostic test and to aid in selecting an appropriate diagnostic test. Moreover, this statistic has advantages over sensitivity and specificity because it is less likely to change with the prevalence of diagnosis.
The highest positive likelihood ratio (LR) for the diagnosis of malnutrition was obtained from the NRI (13.56). According with the level of evidence of Canadian Evidence-Based Medicine, when a test has a positive LR > 10, it is considered a good test to confirm the diagnosis of malnutrition. 30 These findings demonstrate that the NRI is highly accurate in the diagnosis of undernutrition in our sample, being the method with the best agreement with the PNI-O, with a value Kappa of 0.846.

Conclusions
The present study found high prevalence of malnutrition among patients on the waitlist for a liver transplant and variability in the estimated prevalence of malnutrition depending on the method of evaluation. Moreover, the nutritional assessment methods do not show good concordance. The most valid methods for identifying malnutrition in patients with liver disease are the PNI-O and the NRI.

Conflicts of interest
None declared.