Retrospective Study Open Access
Copyright ©The Author(s) 2020. Published by Baishideng Publishing Group Inc. All rights reserved.
World J Clin Oncol. Apr 24, 2020; 11(4): 205-216
Published online Apr 24, 2020. doi: 10.5306/wjco.v11.i4.205
Prognostic importance of lymph node yield after curative resection of gastroenteropancreatic neuroendocrine tumours
Jaseela Chiramel, Adeel Khan, Kok Haw Jonathan Lim, Melissa Frizziero, Angela Lamarca, Wasat Mansoor, Richard A Hubner, Juan William Valle, Mairéad Geraldine McNamara, Department of Medical Oncology, The Christie NHS Foundation Trust, Manchester M20 4BX, United Kingdom
Jaseela Chiramel, Royal Free London NHS Foundation Trust, London NW3 2QG, United Kingdom
Rose Almond, Xin Wang, Statistics Group, Digital Services, The Christie NHS Foundation Trust, Manchester, M20 4BX, United Kingdom
Astrid Slagter, Antoni van Leeuwenhoek/Netherlands Cancer Institute, Amsterdam 1066 Cx, Netherlands
Bipasha Chakrabarty, Department of Pathology, The Christie NHS Foundation Trust, Manchester M20 4BX, United Kingdom
Annamaria Minicozzi, Department of Surgery, Barts Health NHS Trust, London EC1A 7BE, United Kingdom
Richard A Hubner, Juan William Valle, Mairéad Geraldine McNamara, Division of Cancer Sciences, University of Manchester, Manchester M13 9PL, United Kingdom
ORCID number: Jaseela Chiramel (0000-0003-3447-0935); Rose Almond (0000-0003-0872-9325); Astrid Slagter (0000-0002-3020-0180); Adeel Khan (0000-0003-1209-940X); Xin Wang (0000-0001-5192-8093); Kok Haw Jonathan Lim (0000-0002-6544-9218); Melissa Frizziero (0000-0001-5598-8771); Bipasha Chakrabarty (0000-0002-0236-0363); Annamaria Minicozzi (0000-0002-5820-1118); Angela Lamarca (0000-0001-9696-6122); Wasat Mansoor (0000-0002-1477-4934); Richard A Hubner (0000-0002-7081-8823); Juan William Valle (0000-0002-1999-0863); Mairéad Geraldine McNamara (0000-0002-2272-3678).
Author contributions: Chiramel J wrote the manuscript, drafted conception and designed the study; Almond R and Xin W were in charge of statistical analysis of data; Slagter A, Khan A, and Lim KHJ collected and analysed the data; Frizzero M, Chakrabarty B, Minicozzi A, Lamarca A, Mansoor W, Hubner RA, Valle JW and McNamara MG editing and review the manuscript.
Institutional review board statement: This study was approved by the Quality Improvement and Clinical Audit Committee of The Christie NHS Foundation Trust, Manchester, United Kingdom (reference SE17/2065).
Informed consent statement: Patients were not required to give informed consent to the study. We used anonymous clinical data for the analysis.
Conflict-of-interest statement: The authors report no conflict of interest in this work.
Data sharing statement: No additional data are available.
Open-Access: This article is an open-access article that was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Corresponding author: Mairéad Geraldine McNamara, BM BCh, MD, MSc, PhD, Attending Doctor, Senior Lecturer, Senior Researcher, Department of Medical Oncology, The Christie NHS Foundation Trust, 550 Wilmslow Road, Manchester M20 4BX, United Kingdom. mairead.mcnamara@christie.nhs.uk
Received: December 24, 2019
Peer-review started: December 24, 2019
First decision: February 20, 2020
Revised: February 29, 2020
Accepted: April 8, 2020
Article in press: April 8, 2020
Published online: April 24, 2020

Abstract
BACKGROUND

The prognostic significance of lymph nodes (LNs) metastases and the optimum number of LN yield in gastroenteropancreatic neuroendocrine tumours (GEP NETs) undergoing curative resection is still debatable. Many studies have demonstrated that cure rate for patients with GEP NETs can be improved by the resection of the primary tumour and regional lymphadenectomy

AIM

To evaluate the effect of lymph node (LN) status and yield on relapse-free survival (RFS) and overall survival (OS) in patients with resected GEP NETs.

METHODS

Data on patients who underwent curative resection for GEP NETs between January 2002 and March 2017 were analysed retrospectively. Grade 3 tumours (Ki67 > 20%) were excluded. Univariate Cox proportional hazard models were computed for RFS and OS and assessed alongside cut-point analysis to distinguish a suitable binary categorisation of total LNs retrieved associated with RFS.

RESULTS

A total of 217 patients were included in the study. The median age was 59 years (21-97 years) and 51% (n = 111) were male. Primary tumour sites were small bowel (42%), pancreas (25%), appendix (18%), rectum (7%), colon (3%), gastric (2%), others (2%). Median follow up times for all patients were 41 mo (95%CI: 36-51) and 71 mo (95%CI: 63–76) for RFS and OS respectively; 50 relapses and 35 deaths were reported. LNs were retrieved in 151 patients. Eight or more LNs were harvested in 106 patients and LN positivity reported in 114 patients. Three or more positive LNs were detected in 62 cases. The result of univariate analysis suggested perineural invasion (P = 0.0023), LN positivity (P = 0.033), LN retrieval of ≥ 8 (P = 0.047) and localisation (P = 0.0049) have a statistically significant association with shorter RFS, but there was no effect of LN ratio on RFS: P = 0.1 or OS: P = 0.75. Tumour necrosis (P = 0.021) and perineural invasion (P = 0.016) were the only two variables significantly associated with worse OS. In the final multivariable analysis, localisation (pancreas HR = 27.33, P = 0.006, small bowel HR = 32.44, P = 0.005), and retrieval of ≥ 8 LNs (HR = 2.7, P = 0.036) were independent prognostic factors for worse RFS.

CONCLUSION

An outcome-oriented approach to cut-point analysis can suggest a minimum number of adequate LNs to be harvested in patients with GEP NETs undergoing curative surgery. Removal of ≥ 8 LNs is associated with increased risk of relapse, which could be due to high rates of LN positivity at the time of surgery. Given that localisation had a significant association with RFS, a prospective multicentre study is warranted with a clear direction on recommended surgical practice and follow-up guidance for GEP NETs.

Key Words: Well differentiated neuroendocrine tumours, Pancreatic neuroendocrine tumours, Small intestinal neuroendocrine tumours, Ki67, Lymph node ratio, Lymph node retrieval, Lymph node positivity, Relapse-free survival, Overall survival

Core tip: There is no consensus on the optimal number of lymph node (LN) retrieval in patients undergoing resection for gastroenteropancreatic neuroendocrine tumours (GEP NETs). The purpose of this study was to evaluate the LN status and yield on relapse-free survival and overall survival in patients with resected GEP NETs. By using the outcome-oriented approach to cut-point analysis, this study suggested a retrieval of a minimum of eight or more LNs in patients with GEP NETs undergoing curative surgery. The risk of relapse was high in patients who had ≥ 8 LNs retrieved and a high LN yield and LN positivity were seen in small intestinal NETs and pancreatic NETs. The localisation has a significant association with relapse-free survival, necessitating stricter surveillance especially for intestinal NETs and pancreatic NETs.



INTRODUCTION

Neuroendocrine tumours (NETs) are a heterogeneous group, accounting for only 0.5% of all malignancies and 2% of all gastrointestinal malignancies, with an incidence of 5.25/100000/year[1,2]. These tumours originate from the neuroendocrine cells anywhere along the gastrointestinal tract (62%-67%) or lung (22%-27%)[3,4]. The incidence and prevalence of NETs are steadily rising, possibly related to increased awareness and detection of early-stage disease. The most common location for gastroenteropancreatic (GEP) NETs is small intestine (41.8%), followed by rectum (19.6%), appendix (16.7%), pancreas (10.8%), colon (10.6%) and stomach (7.6%)[5]. These tumours can be functioning, i.e. symptomatic due to hypersecretion of hormones and peptides or non-functioning.

In 2010, the World Health Organisation classified NETs into well differentiated NET Grade 1 (G1) (Ki67 ≤ 2%, Mitotic index < 2/10 HPF), well differentiated NET G2 (Ki67 3%-20%, Mitotic index 2-20/10 HPF), poorly differentiated neuroendocrine carcinoma (NEC) G3 (Ki67 > 20%, Mitotic index > 20/10 HPF), mixed adeno NEC (MANEC) and hyperplastic and preneoplastic lesions[6]. The updated version of World Health Organisation classification in 2017 classified pancreatic neuroendocrine neoplasms into well differentiated NET G1 (Ki67 < 3%, Mitotic index < 2/10 HPF), well differentiated NET G2 (Ki67 3%-20%, Mitotic index 2-20/10HPF), well differentiated NET G3 (Ki67 > 20%, Mitotic index > 20/10 HPF), poorly differentiated NEC G3 (Ki67 > 20%, Mitotic index > 20/10 HPF) and mixed neuroendocrine non-neuroendocrine neoplasms[7]. A working group of the European Neuro Endocrine Tumour Society (ENETS) developed and published a proposal for the first Tumour, Node, Metastases (TNM) staging system for neuroendocrine tumours in 2006. In 2009, the American Joint Committee on Cancer staging (AJCC) included the classification of NETs[8]. The current 8th edition of the AJCC TNM staging for neuroendocrine tumours includes well differentiated GEP NETs but excludes poorly differentiated neoplasms[9]. The staging system of the North American Neuroendocrine tumour society is similar to ENETS TNM staging. However, there is some discrepancy in the staging of pancreas and appendiceal NETs.

Advances in the therapeutic management of these tumours have resulted in improvements in survival over the years[10,11]. Prognosis depends on the location of the primary tumour and presence or absence of regional and distant metastases. A Surveillance, Epidemiology and End Result registry (SEER) based data analysis have reported significant differences in survival amongst various primary sites of NETs, including lung NETs. Neuroendocrine tumours of the rectum had the best prognosis, and NETs of the pancreas (PanNET) had the worse prognosis. Five-year overall survival for stage 1 and 2 GEP NETs without metastasis is 95%-98% following curative surgery, and for NETs with regional metastasis is 54%-75%[12]. Recent SEER data revealed a median survival of 33 mo in G1 and G2 NETs with distant metastasis. The 5-year survival rate in metastatic PanNET is around 40%-60%[13,14]. Several studies have demonstrated that prognosis can be extended if regional metastases are resected along with the primary tumour[15-17]. Hellman et al[15] concluded, in a study on patients treated for midgut carcinoid, that patients with resected mesenteric lymph node (LN) metastases survived significantly longer than those with insitu mesenteric metastases (P = 0.05).

The result from an international multicentre study on surgical management of advanced PanNETs has shown that an aggressive approach for locally advanced or metastatic tumours is safe and offers long term survival[18]. Retrospective studies in colorectal cancer have already revealed improved survival rates when a higher number of LNs are examined following curative surgery[19]. This is reflected in colorectal cancer clinical guidelines which recommend evaluation of at least twelve LNs to ascertain LN-status with confidence, as patients with no LN involvement have a favourable prognosis[20-22]. “Adequate” LN clearance is recommended in patients with GEP-NETs undergoing resection of a primary tumour, but there are no clear guidelines about the actual number of nodes that should be resected to achieve favourable survival outcomes.

Several studies in patients with PanNETs have demonstrated that the presence of liver metastases is associated with worse survival[23-25]. Bettini et al[26] reported that the presence of nodal metastases in patients with neuroendocrine neoplasms was significantly associated with increased mortality and had a similar prognostic significance to the presence of liver metastases and Ki67 expression. In addition, a number of studies in patients with PanNETs have demonstrated that positive LNs, total LNs examined and the ratio between positive LNs and total LNs examined are important predictors of recurrence after surgery. Boninsegna et al[27] reported that a LN ratio (LNR) greater than 0.2 (HR = 2.75) and Ki67% > 5% (HR = 3.39) were significant predictors of recurrence following resection for PanNETs.

This retrospective study was conducted to evaluate the association between LN metastases and survival (relapse-free and overall) in patients with resected well differentiated GEP NETs and to attempt to identify the optimal number of LNs that should be harvested in patients with GEP NETS, undergoing curative surgery.

MATERIALS AND METHODS

Data on patients who underwent curative surgery for GEP NETs between January 2002 and March 2017 were identified and analysed retrospectively. This retrospective study was conducted at a European Neuroendocrine Tumour Centre of Excellence tertiary referral centre, The Christie NHS Foundation Trust, Manchester, United Kingdom, with surgery performed in a high-volume specialised surgical unit at Manchester Royal Infirmary, within the Greater Manchester catchment area serving approximately 2.5 million. Patients with Grade 1 and Grade 2 GEPNETs were included and those with Grade 3 (Ki67 > 20%, poorly differentiated neuroendocrine neoplasms) were excluded as this is a completely different clinical entity with more aggressive biology and behaviour. Data were collected from patient case notes (paper and electronic) and post-operative histopathology reports. Demographic and clinical data were collected, including age, gender, Eastern Co-operative Oncology Group performance status, grade, Ki67, TNM staging, serum Chromogranin A, 5-Hydroxyindoleacetic acid, surgical margin, negative (R0) or positive (microscopic positive margin R1) margins, perineural invasion, lymphovascular invasion, tumour necrosis, total number of LNs retrieved, number of involved LNs and localisation of the tumour. Tumour locations were coded as the stomach, duodenum, small intestine, colon, appendix, rectum, pancreas and rectum. This study was ethically approved by the Quality Improvement and Clinical Audit Committee of The Christie NHS Foundation Trust.

LNR was defined as the ratio between the number of positive LNs (with metastases) and the total number of LNs examined. Relapse-free survival (RFS) was defined as the time between surgery and relapse, or date last seen. Overall survival was defined as the time between surgery and death, or the date the death registry was checked, which was on 23rd February 2018.

Statistical analysis

Descriptive statistics were utilised to check data variability. Kaplan Meier curves were used to identify an empirical estimate of the survival curve, and the Log-rank test was used to evaluate how significant the survival rate difference was between the two categories of a variable. Univariate and multivariable Cox proportional hazard models were used to identify the independent predictors of RFS and OS. A P value of < 0.05 was considered statistically significant.

Univariate Cox proportional hazard models were computed for RFS and OS and assessed alongside cut-point analysis to distinguish a suitable binary categorisation of total LNs retrieved associated with RFS. All statistical analyses were computed in R (version 3.4.2), with cut-point analysis using the function “surv_cutpoint” from the R-package Survminer (version 0.4.2)[28-30]. LN cut-point value was determined using the cut-point determination methods in survival analysis, using R[31]. This is an outcome-oriented method providing a value of a cut-point that corresponds to the most significant relationship with survival. Surv cut-point determines the optimal cut-point for each variable.

RESULTS

A total of 217 patients were included in the study. The median age was 59 years (21-97 years) and 51% (n = 111) were male; 77% were G1 and 23% were G2. Primary tumour sites were small bowel (42%), pancreas (25%), appendix (18%), rectum (7%), colon (3%), gastric (2%), others (2%) (Table 1); LN data was not available in 30% [5% PanNETs and 7% small intestinal-NETs (SiNETs)]. Median follow up times for all patients for RFS and OS were 41 mo [95% confidence interval (CI): 36-51] and 71 mo (95%CI: 63-76), respectively, and 50 relapses and 35 deaths were reported.

Table 1 Characteristics of patients who had curative resection of gastroenteropancreatic neuroendocrine tumours.
Variablen (%)
GenderMale111 (51)
Female106 (49)
Age, yrMedian age59
ECOG PSPS 0140 (65)
PS 164 (30)
PS 28(3.7)
PS 35 (2.3)
LocalisationSmall bowel92 (42)
Pancreas55 (25)
Appendix39 (18)
Rectum15 (7)
Colon7 (3)
Stomach5 (2)
Other4 (2)
GradingGrade 1168 (77)
Grade 249 (23)
Surgical marginR0 resection186 (86)
R1 resection30 (13)
NA1 (0.5)
Vascular infiltrationPresent89 (41)
Absent109 (51)
NA19 (8)
Perineal infiltrationPresent64 (29)
Absent133 (62)
NA20 (9.3)
Tumour necrosisPresent10 (5)
Absent183 (86)
NA24 (10)
Lymph nodes retrieved≥ 8106 (49)
< 845 (21)
NA66 (31)
Lymph node involvementPositive114 (52)
Negative37 (17)
Serum CgA Level≥ ULN46(21)
Normal69 (32)
NA102 (47)
Serum 5-HIAA≥ ULN17 (8)
Negative96 (44)
NA104 (48)

The total number of patients with LNs retrieved was 151. Data on LNs retrieved were available for 76 patients with SiNETs and 45 with PanNETs, and the rest were grouped as “others” (stomach n = 3, appendix n = 20, colon n = 5, rectum n = 1 and other n = 1). Eight or more LNs were harvested in 106 patients (49 SiNETs, 32 PanNETs); LN positivity was reported in 114 patients; 70 SiNETS and 24 PanNETs. Three or more positive LNs were detected in 62 cases; 43 SiNETs and 13 PanNETs. There were 29 relapses and 16 deaths reported in patients with SiNETs and 17 relapses and 11 deaths in patients with PanNETs.

Univariate analysis

The result of univariate analysis suggested perineural invasion (P = 0.0023), LN positivity (P = 0.033), LN retrieval of ≥ 8 (P = 0.047) and localisation (P = 0.0049) have a statistically significant association with shorter RFS, but there was no effect of LN ratio (median 1.8) on RFS: P = 0.1 or OS: P = 0.75. Tumour necrosis (P = 0.021) and perineural invasion (P = 0.016) were the only two variables significantly associated with worse OS. Retrieval of ≥ 8 LNs (P = 0.94), localisation (P = 0.44), or surgical margin (P = 0.69) did not significantly affect OS (Table 2). LN cut-point value associated with RFS was 8.

Table 2 Univariate analysis of variables for relapse-free survival and overall survival in patients who had curative resection of gastroenteropancreatic neuroendocrine tumours.
VariableLR P valueLR P value OS
Production of gut hormones0.0120.75
Production of CgA0.090.81
Production of 5 HIAA0.830.098
Genetic predisposition0.0110.71
Multiple NET0.590.25
Other active malignancy0.270.96
Vascular infiltration0.170.97
Perineural infiltration0.00230.016
Tumour necrosis0.660.021
≥ 8 lymph nodes retrieved0.0470.94
Any lymph nodes positive0.0330.78
Gender0.580.87
Localisation0.00490.44
T stage0.0150.22
N stage0.0080.58
M stage0.0890.17
Grade0.0391
Surgical margin0.320.69
ECOG performance status0.75< 0.0001
ACE comorbidity score0.390.044
Localisation0.630.61
Involved groups0.0620.09
TNM stage0.0190.19
Multivariable analysis

A total of 140 patients were included in the final multivariable model. Eleven patients (one influential outlier and no data on perineural infiltration in 10 cases) were excluded. The variables included in the final multivariable model for RFS were the presence of perineural infiltration, eight or more LNs retrieved, any positive LNs and localisation: Pancreas and small bowel. The grade wasn’t included in multivariable analysis, as it was not statistically significant on univariate analysis. Retrieval of ≥ 8 LNs (HR = 2.70, 95%CI: 1.07-6.84, P = 0.036), tumour localisation: pancreas (HR = 27.33, P = 0.006) and small bowel (HR = 32.44, P = 0.005) were independent prognostic factors for shorter RFS on multivariable analysis. LN positivity was not statistically significantly associated with RFS in the multivariable model. (Table 3 and Figures 1-3).

Figure 1
Figure 1 Kaplan Meier curve of relapse-free survival according to lymph node yield in patients with gastroenteropancreatic neuroendocrine tumours who underwent resection of primary tumour and regional lymphadenectomy. Lymph node yield ≥ 8: P = 0.0365; Hazard ratio = 2.65; 95%CI: 1.06-6.62.
Figure 2
Figure 2 Kaplan Meier plot of relapse-free survival according to LN positivity in patients with gastroenteropancreatic neuroendocrine tumours who underwent curative resection. P = 0.0840; Hazard ratio = 2.62; 95%CI: 0.88-7.78.
Table 3 Multivariable analysis of variables associated with relapse-free survival in patients who had curative resection of gastroenteropancreatic neuroendocrine tumours.
VariableHR95%CIP value
Presence of perineural infiltration: Yes1.570.81-3.060.1858
Over 8 lymph nodes retrieved: Yes2.651.06-6.620.0365
Any lymph nodes found positive: Yes2.620.88-7.780.0840
Localisation category: Pancreas (relative to “others”)10.691.95-58.560.0063
Localisation category: Small bowel (relative to “others”)12.172.19-67.690.0043
Figure 3
Figure 3 Kaplan Meier plot of relapse-free survival according to tumour localisation: Pancreatic neuroendocrine tumours, small intestinal neuroendocrine tumours and others. Pancreas: P = 0.0063; Hazard ratio = 10.69; 95%CI: 1.95-58.56. Small intestine: P = 0.0043; Hazard ratio = 12.17; 95%CI: 2.19-67.69.
DISCUSSION

The prognostic significance of LNs metastases and the optimum number of LN yield in GEP NETs undergoing curative resection is debatable. Many studies have already demonstrated that resection of the primary tumour and regional lymphadenectomy results in a high cure rate for patients with GEP NETs. LN positivity and LNRs are independent prognostic factors for survival in patients with resected NETs, but limited evidence is available on the optimal predictive number of resected LNs required[27,32,33]. As per the AJCC TNM staging for NETs, the presence of positive LNs defines stage III disease, regardless of the number of LNs involved. The ENETS guidelines do provide some advice on follow up of patients with GEP NETs post-resection, but to date, there has been no consensus regarding the optimal number of LNs resected, required for the adequate staging of GEP NETs[34].

The main purpose of this study was to identify a cut off value for LN retrieval in resected GEP NETs. By using the outcome-oriented approach to cut-point analysis, this study suggested retrieval of a minimum of eight or more LNs in GEP NETs undergoing curative surgery. The risk of relapse was high in patients who had ≥ 8 LNs retrieved. Previous colorectal studies have demonstrated an increased relapse rate associated with low numbers of LNs harvested[35,36]. Studies in GEP NETs, in particular SiNETs and PanNETs have shown reduced RFS in patients with increased number of involved LNs. The decrease in RFS associated with an increased number of LNs harvested in the current study indicates that by examining more LNs, one increases the chance of finding more involved nodes; thereby staging patients more accurately. Of 151 patients with available information on involved LNs, ≥ 8 LNs were harvested in 70%, and positive LNs were detected in 41%. The majority of LNs were retrieved from SiNETs and PanNETs, and involvement of 3 or more LNs was high in SiNETs and PanNETs.

There are many factors like tumour size, localisation and tumour biology that influence the variability of LN harvest. It has already been reported that the size of the tumour, LN involvement and Ki67 are independent prognostic factors for relapse after potentially curative surgery for NET[37,38]. A relatively large study from the United States neuroendocrine study group identified pre-operative factors, including tumour size ≥ 2 cm, proximal location, moderate differentiation and Ki67 > 3%, as factors predicting LN positivity in resected non-functional PanNETs. LN metastases were reported in patients without these risk factors also, so the conclusion from the study was that routine regional lymphadenectomy should be considered in patients with PanNETs undergoing curative surgery. Pancreatoduodenectomy routinely includes a complete regional lymphadenectomy, whereas distal pancreatectomy should aim to remove ≥ 7 LNs for accurate staging (5-year RFS in LN positive and negative disease was 67% vs 86%, P = 0.002)[39]. A recently published study in PanNETs concluded that a regional lymphadenectomy of at least 8 LNs is necessary for optimal staging of PanNETs undergoing curative resection. The study reported patients with ≥ 4 LN metastases had a worse prognosis compared to patients with 1-3 LN metastases or node negative disease[40].

Another study reported on the prognostic role of LN positivity and number of LNs needed for accurate staging of small bowel neuroendocrine tumours[41]. It emphasised the importance of a thorough regional lymphadenectomy to accurately stage patients undergoing curative resection for SiNETs. This study suggested that the minimum requirement of LNs for evaluation after curative resection of SiNET was eight, concurring with the current study; and four or more positive LNs were associated with reduced 3-year recurrence-free survival. Patients with four or more positive LNs had a worse 3-year recurrence-free survival compared to those with 1-3 or 0 LNs (P = 0.01), and retrieval of > 8 LNs accurately discriminated patients with 4 or more, 1-3 or 0 LNs (3-year RFS 79.7% vs 89.6% vs 92.9%; P = 0.05)[41]. In addition, Martin et al[32] conducted a study involving 16598 patients from the SEER registry, who underwent curative resection for GEP NETs from different primary locations. This study concluded that the extent of LN involvement was associated with survival across most GEP NET primary sites but did not report an optimal LN cut off. However, an LNR of ≥ 2.0 was associated with worse survival.

Compared to these studies the current study included tumours from all gastrointestinal locations and a high LN yield and LN positivity were seen in SiNETs and PanNETs. Associations between LN positivity and RFS might have been found if the numbers of patients included were higher, but ideally, prospective studies should be instituted. Small studies do not have the power to rule out a real difference and avoid a type II error (false negative). The other limitation of this study was that there was variability in the number of LNs resected, and lack of records of LNs harvested in a proportion of patients (30%), highlighting the associated limitations of a retrospective study. The reason that patients with < 8 LNs retrieved had a better RFS may be a reflection of the fact that less pathological LNs were subjectively obvious at the time of surgery in these patients, and thus these patients have a better prognosis with more localised disease. Despite the constraint of study size, this is a relatively large study including patients with resected GEP NETs in a tertiary real-world clinical setting and adds to the limited body of literature in this study area. It does highlight the importance of retrieving adequate LNs during surgery for GEP NETs and indicates the necessity for closer follow up of patients with LN positivity. The current study demonstrated that localisation has a significant association with RFS, necessitating stricter surveillance for small bowel and pancreas primaries, in particular.

In conclusion, this study demonstrated that an outcome-oriented approach to cut-point analysis can suggest a minimum number of adequate LNs to be harvested in patients with GEP NETs undergoing curative surgery. Removal of ≥ 8 LNs is associated with increased risk of relapse, which could be due to high rates of LN positivity at the time of surgery. However, the current study failed to demonstrate an association between LN positivity and LNR with RFS or OS, due to the small study size. It can be concluded that for accurate staging of GEP NETs, the percentage of positive nodes and LNR should be reported following potentially curative resection and incorporated into TNM staging. Given that localisation (pancreas vs small bowel vs other) had a significant association with RFS, a prospective multicentre study is warranted with a clear direction on recommended surgical practice and follow-up guidance for GEP NETs.

ARTICLE HIGHLIGHTS
Research background

The prognostic significance of lymph nodes (LNs) metastases and the optimum number of LN yield in gastroenteropancreatic neuroendocrine tumours (GEP NETs) undergoing curative resection is still debatable. Many studies have demonstrated that resection of the primary tumour and regional lymphadenectomy results in a high cure rate for patients with GEP NETs.

Research motivation

LN positivity and LN ratio (LNR) are independent prognostic factors for survival in patients with resected NETs, but limited evidence is available on the optimal predictive number of resected LNs required. Several retrospective studies in Pancreatic NETs (PanNETs) and Small Bowel NETs (SiNETs) have emphasized the importance of adequate resection of regional LNs in patients undergoing curative resection. The current guidelines (AJCC TNM staging 8th edition and ENETS) for the management of GEPNETs do not provide a recommendation regarding LN yield. The main purpose of the study was to identify a cut off value for LN retrieval in resected GEP NETs.

Research objectives

This retrospective study was conducted to identify the optimal number of LNs that should be harvested in patients with GEP NETS undergoing curative surgery and to evaluate the association between LN metastases and survival (relapse-free and overall) in patients with resected well differentiated GEP NETs.

Research methods

Data on patients who underwent curative surgery for GEP NETs between January 2002 and March 2017 were identified and analysed retrospectively. Univariate Cox proportional hazard (CPH) models were computed for RFS and OS and assessed alongside cut-point analysis to distinguish a suitable binary categorisation of total LNs retrieved associated with RFS. LN cut-point value was determined using the cut-point determination methods in survival analysis, using R. This is an outcome-oriented method providing a value of a cut-point that corresponds to the most significant relationship with survival.

Research results

The result of univariate analysis suggested perineural invasion (P = 0.0023), LN positivity (P = 0.033), LN retrieval of ≥ 8 (P = 0.047) and localisation (P = 0.0049) have a statistically significant association with shorter RFS, but there was no effect of LN ratio (median 1.8) on RFS: P = 0.1 or OS: P = 0.75. LN cut-point value associated with RFS was 8. Tumour necrosis (P = 0.021) and perineural invasion (P = 0.016) were the only two variables significantly associated with worse OS. Retrieval of ≥ 8 LNs (HR = 2.70, 95%CI: 1.07-6.84, P = 0.036), tumour localisation: pancreas (HR = 27.33, P = 0.006) and small bowel (HR = 32.44, P = 0.005) were independent prognostic factors for shorter RFS on multivariable analysis.

Research conclusions

The study has concluded that an outcome-oriented approach to cut-point analysis can suggest a minimum number of adequate LNs to be harvested in patients with GEP NETs undergoing curative surgery. A prospective multicentre study is warranted with a clear direction on recommended surgical practice and follow-up guidance for GEP NETs.

Footnotes

Manuscript source: Invited Manuscript

Specialty type: Oncology

Country/Territory of origin: United Kingdom

Peer-review report's scientific quality classification

Grade A (Excellent): 0

Grade B (Very good): B

Grade C (Good): 0

Grade D (Fair): 0

Grade E (Poor): 0

P-Reviewer: Kupeli S S-Editor: Wang YQ L-Editor: A E-Editor: Liu JH

References
1.  Massironi S, Sciola V, Peracchi M, Ciafardini C, Spampatti MP, Conte D. Neuroendocrine tumors of the gastro-entero-pancreatic system. World J Gastroenterol. 2008;14:5377-5384.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in CrossRef: 107]  [Cited by in F6Publishing: 109]  [Article Influence: 6.8]  [Reference Citation Analysis (0)]
2.  Öberg K, Knigge U, Kwekkeboom D, Perren A; ESMO Guidelines Working Group. Neuroendocrine gastro-entero-pancreatic tumors: ESMO Clinical Practice Guidelines for diagnosis, treatment and follow-up. Ann Oncol. 2012;23 Suppl 7:vii124-vii130.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 307]  [Cited by in F6Publishing: 351]  [Article Influence: 31.9]  [Reference Citation Analysis (0)]
3.  Modlin IM, Lye KD, Kidd M. A 5-decade analysis of 13,715 carcinoid tumors. Cancer. 2003;97:934-959.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 1848]  [Cited by in F6Publishing: 1766]  [Article Influence: 84.1]  [Reference Citation Analysis (0)]
4.  Rosai J. The origin of neuroendocrine tumors and the neural crest saga. Mod Pathol. 2011;24 Suppl 2:S53-S57.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 58]  [Cited by in F6Publishing: 65]  [Article Influence: 5.0]  [Reference Citation Analysis (0)]
5.  Maggard MA, O'Connell JB, Ko CY. Updated population-based review of carcinoid tumors. Ann Surg. 2004;240:117-122.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 342]  [Cited by in F6Publishing: 352]  [Article Influence: 17.6]  [Reference Citation Analysis (0)]
6.  Bosman FT, World Health Organization, International Agency for Research on Cancer. WHO classification of tumours of the digestive system.  4th ed. Lyon: IARC Press, 2010.  [PubMed]  [DOI]  [Cited in This Article: ]
7.  Lloyd RV, Osamura RY, Klöppel G, Rosai J. WHO classification of tumours of endocrine organs.  4th ed. Lyon: IARC, 2017: 209-240.  [PubMed]  [DOI]  [Cited in This Article: ]
8.  Klöppel G. Classification and pathology of gastroenteropancreatic neuroendocrine neoplasms. Endocr Relat Cancer. 2011;18 Suppl 1:S1-16.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 217]  [Cited by in F6Publishing: 232]  [Article Influence: 17.8]  [Reference Citation Analysis (0)]
9.  Amin MB, Greene FL, Edge SB, Compton CC, Gershenwald JE, Brookland RK, Meyer L, Gress DM, Byrd DR, Winchester DP. The Eighth Edition AJCC Cancer Staging Manual: Continuing to build a bridge from a population-based to a more "personalized" approach to cancer staging. CA Cancer J Clin. 2017;67:93-99.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 2341]  [Cited by in F6Publishing: 3164]  [Article Influence: 452.0]  [Reference Citation Analysis (2)]
10.  Yao JC, Hassan M, Phan A, Dagohoy C, Leary C, Mares JE, Abdalla EK, Fleming JB, Vauthey JN, Rashid A, Evans DB. One hundred years after "carcinoid": epidemiology of and prognostic factors for neuroendocrine tumors in 35,825 cases in the United States. J Clin Oncol. 2008;26:3063-3072.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 3022]  [Cited by in F6Publishing: 3036]  [Article Influence: 189.8]  [Reference Citation Analysis (0)]
11.  Dasari A, Shen C, Halperin D, Zhao B, Zhou S, Xu Y, Shih T, Yao JC. Trends in the Incidence, Prevalence, and Survival Outcomes in Patients With Neuroendocrine Tumors in the United States. JAMA Oncol. 2017;3:1335-1342.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 1510]  [Cited by in F6Publishing: 1982]  [Article Influence: 283.1]  [Reference Citation Analysis (2)]
12.  Man D, Wu J, Shen Z, Zhu X. Prognosis of patients with neuroendocrine tumor: a SEER database analysis. Cancer Manag Res. 2018;10:5629-5638.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 71]  [Cited by in F6Publishing: 90]  [Article Influence: 15.0]  [Reference Citation Analysis (1)]
13.  Garcia-Carbonero R, Capdevila J, Crespo-Herrero G, Díaz-Pérez JA, Martínez Del Prado MP, Alonso Orduña V, Sevilla-García I, Villabona-Artero C, Beguiristain-Gómez A, Llanos-Muñoz M, Marazuela M, Alvarez-Escola C, Castellano D, Vilar E, Jiménez-Fonseca P, Teulé A, Sastre-Valera J, Benavent-Viñuelas M, Monleon A, Salazar R. Incidence, patterns of care and prognostic factors for outcome of gastroenteropancreatic neuroendocrine tumors (GEP-NETs): results from the National Cancer Registry of Spain (RGETNE). Ann Oncol. 2010;21:1794-1803.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 257]  [Cited by in F6Publishing: 282]  [Article Influence: 20.1]  [Reference Citation Analysis (0)]
14.  Ekeblad S, Skogseid B, Dunder K, Oberg K, Eriksson B. Prognostic factors and survival in 324 patients with pancreatic endocrine tumor treated at a single institution. Clin Cancer Res. 2008;14:7798-7803.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 277]  [Cited by in F6Publishing: 296]  [Article Influence: 19.7]  [Reference Citation Analysis (0)]
15.  Hellman P, Lundström T, Ohrvall U, Eriksson B, Skogseid B, Oberg K, Tiensuu Janson E, Akerström G. Effect of surgery on the outcome of midgut carcinoid disease with lymph node and liver metastases. World J Surg. 2002;26:991-997.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 207]  [Cited by in F6Publishing: 194]  [Article Influence: 8.8]  [Reference Citation Analysis (0)]
16.  Ahmed A, Turner G, King B, Jones L, Culliford D, McCance D, Ardill J, Johnston BT, Poston G, Rees M, Buxton-Thomas M, Caplin M, Ramage JK. Midgut neuroendocrine tumours with liver metastases: results of the UKINETS study. Endocr Relat Cancer. 2009;16:885-894.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 198]  [Cited by in F6Publishing: 208]  [Article Influence: 13.9]  [Reference Citation Analysis (0)]
17.  Givi B, Pommier SJ, Thompson AK, Diggs BS, Pommier RF. Operative resection of primary carcinoid neoplasms in patients with liver metastases yields significantly better survival. Surgery. 2006;140:891-897.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 131]  [Cited by in F6Publishing: 144]  [Article Influence: 8.5]  [Reference Citation Analysis (0)]
18.  Birnbaum DJ, Turrini O, Vigano L, Russolillo N, Autret A, Moutardier V, Capussotti L, Le Treut YP, Delpero JR, Hardwigsen J. Surgical management of advanced pancreatic neuroendocrine tumors: short-term and long-term results from an international multi-institutional study. Ann Surg Oncol. 2015;22:1000-1007.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 74]  [Cited by in F6Publishing: 75]  [Article Influence: 7.5]  [Reference Citation Analysis (0)]
19.  Swanson RS, Compton CC, Stewart AK, Bland KI. The prognosis of T3N0 colon cancer is dependent on the number of lymph nodes examined. Ann Surg Oncol. 2003;10:65-71.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 443]  [Cited by in F6Publishing: 432]  [Article Influence: 20.6]  [Reference Citation Analysis (0)]
20.  McDonald JR, Renehan AG, O'Dwyer ST, Haboubi NY. Lymph node harvest in colon and rectal cancer: Current considerations. World J Gastrointest Surg. 2012;4:9-19.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in CrossRef: 102]  [Cited by in F6Publishing: 101]  [Article Influence: 8.4]  [Reference Citation Analysis (0)]
21.  Fielding LP, Arsenault PA, Chapuis PH, Dent O, Gathright B, Hardcastle JD, Hermanek P, Jass JR, Newland RC. Clinicopathological staging for colorectal cancer: an International Documentation System (IDS) and an International Comprehensive Anatomical Terminology (ICAT). J Gastroenterol Hepatol. 1991;6:325-344.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 299]  [Cited by in F6Publishing: 275]  [Article Influence: 8.3]  [Reference Citation Analysis (0)]
22.  Sobin LH, Greene FL. TNM classification: clarification of number of regional lymph nodes for pNo. Cancer. 2001;92:452.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in F6Publishing: 2]  [Reference Citation Analysis (0)]
23.  Frilling A, Sotiropoulos GC, Li J, Kornasiewicz O, Plöckinger U. Multimodal management of neuroendocrine liver metastases. HPB (Oxford). 2010;12:361-379.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 106]  [Cited by in F6Publishing: 104]  [Article Influence: 7.4]  [Reference Citation Analysis (0)]
24.  Mayo SC, de Jong MC, Bloomston M, Pulitano C, Clary BM, Reddy SK, Clark Gamblin T, Celinski SA, Kooby DA, Staley CA, Stokes JB, Chu CK, Arrese D, Ferrero A, Schulick RD, Choti MA, Geschwind JF, Strub J, Bauer TW, Adams RB, Aldrighetti L, Mentha G, Capussotti L, Pawlik TM. Surgery versus intra-arterial therapy for neuroendocrine liver metastasis: a multicenter international analysis. Ann Surg Oncol. 2011;18:3657-3665.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 122]  [Cited by in F6Publishing: 140]  [Article Influence: 10.8]  [Reference Citation Analysis (0)]
25.  Saxena A, Chua TC, Sarkar A, Chu F, Liauw W, Zhao J, Morris DL. Progression and survival results after radical hepatic metastasectomy of indolent advanced neuroendocrine neoplasms (NENs) supports an aggressive surgical approach. Surgery. 2011;149:209-220.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 77]  [Cited by in F6Publishing: 60]  [Article Influence: 4.3]  [Reference Citation Analysis (0)]
26.  Bettini R, Boninsegna L, Mantovani W, Capelli P, Bassi C, Pederzoli P, Delle Fave GF, Panzuto F, Scarpa A, Falconi M. Prognostic factors at diagnosis and value of WHO classification in a mono-institutional series of 180 non-functioning pancreatic endocrine tumours. Ann Oncol. 2008;19:903-908.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 172]  [Cited by in F6Publishing: 169]  [Article Influence: 10.6]  [Reference Citation Analysis (0)]
27.  Boninsegna L, Panzuto F, Partelli S, Capelli P, Delle Fave G, Bettini R, Pederzoli P, Scarpa A, Falconi M. Malignant pancreatic neuroendocrine tumour: lymph node ratio and Ki67 are predictors of recurrence after curative resections. Eur J Cancer. 2012;48:1608-1615.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 121]  [Cited by in F6Publishing: 123]  [Article Influence: 9.5]  [Reference Citation Analysis (0)]
28.  Therneau TM  A package for Survival Analysis in S. R Version 2: 37-34. Available from: https://CRAN.R-project.org/package=survival.  [PubMed]  [DOI]  [Cited in This Article: ]
29.  Therneau TM, Grambsch PM. The Cox Model. Modeling Survival Data: Extending the Cox Model. New York: Springer 2000; 39-77.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 152]  [Cited by in F6Publishing: 150]  [Article Influence: 6.3]  [Reference Citation Analysis (0)]
30.  Kassambara A, Kosinski M, Biecek P, Fabian S.   Drawing Survival Curves using “ggplot2” [R package survminer version 0.4.2]. Available from: https://CRAN.R-project.org/package=survminer.  [PubMed]  [DOI]  [Cited in This Article: ]
31.  Mandrekar JN, Mandrekar SJ, Cha S. Cutpoint Determination Methods in Survival Analysis using SAS.  In: SUGI 28th. North Carolina: SAS Institute Inc., 2003: 261-228. Available from: https://support.sas.com/resources/papers/proceedings/proceedings/sugi28/261-28.pdf.  [PubMed]  [DOI]  [Cited in This Article: ]
32.  Martin JA, Warner RRP, Aronson A, Wisnivesky JP, Kim MK. Lymph Node Metastasis in the Prognosis of Gastroenteropancreatic Neuroendocrine Tumors. Pancreas. 2017;46:1214-1218.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 16]  [Cited by in F6Publishing: 16]  [Article Influence: 2.3]  [Reference Citation Analysis (0)]
33.  Hashim YM, Trinkaus KM, Linehan DC, Strasberg SS, Fields RC, Cao D, Hawkins WG. Regional lymphadenectomy is indicated in the surgical treatment of pancreatic neuroendocrine tumors (PNETs). Ann Surg. 2014;259:197-203.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 174]  [Cited by in F6Publishing: 176]  [Article Influence: 17.6]  [Reference Citation Analysis (0)]
34.  Niederle B, Pape UF, Costa F, Gross D, Kelestimur F, Knigge U, Öberg K, Pavel M, Perren A, Toumpanakis C, O'Connor J, O'Toole D, Krenning E, Reed N, Kianmanesh R; Vienna Consensus Conference participants. ENETS Consensus Guidelines Update for Neuroendocrine Neoplasms of the Jejunum and Ileum. Neuroendocrinology. 2016;103:125-138.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 293]  [Cited by in F6Publishing: 291]  [Article Influence: 36.4]  [Reference Citation Analysis (0)]
35.  Caplin S, Cerottini JP, Bosman FT, Constanda MT, Givel JC. For patients with Dukes' B (TNM Stage II) colorectal carcinoma, examination of six or fewer lymph nodes is related to poor prognosis. Cancer. 1998;83:666-672.  [PubMed]  [DOI]  [Cited in This Article: ]
36.  Cianchi F, Palomba A, Boddi V, Messerini L, Pucciani F, Perigli G, Bechi P, Cortesini C. Lymph node recovery from colorectal tumor specimens: recommendation for a minimum number of lymph nodes to be examined. World J Surg. 2002;26:384-389.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 151]  [Cited by in F6Publishing: 160]  [Article Influence: 7.3]  [Reference Citation Analysis (0)]
37.  Slagter AE, Ryder D, Chakrabarty B, Lamarca A, Hubner RA, Mansoor W, O'Reilly DA, Fulford PE, Klümpen HJ, Valle JW, McNamara MG. Prognostic factors for disease relapse in patients with neuroendocrine tumours who underwent curative surgery. Surg Oncol. 2016;25:223-228.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 13]  [Cited by in F6Publishing: 13]  [Article Influence: 1.6]  [Reference Citation Analysis (0)]
38.  Merath K, Bagante F, Beal EW, Lopez-Aguiar AG, Poultsides G, Makris E, Rocha F, Kanji Z, Weber S, Fisher A, Fields R, Krasnick BA, Idrees K, Smith PM, Cho C, Beems M, Schmidt CR, Dillhoff M, Maithel SK, Pawlik TM. Nomogram predicting the risk of recurrence after curative-intent resection of primary non-metastatic gastrointestinal neuroendocrine tumors: An analysis of the U.S. Neuroendocrine Tumor Study Group. J Surg Oncol. 2018;117:868-878.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 34]  [Cited by in F6Publishing: 36]  [Article Influence: 6.0]  [Reference Citation Analysis (0)]
39.  Lopez-Aguiar AG, Zaidi MY, Beal EW, Dillhoff M, Cannon JGD, Poultsides GA, Kanji ZS, Rocha FG, Marincola Smith P, Idrees K, Beems M, Cho CS, Fisher AV, Weber SM, Krasnick BA, Fields RC, Cardona K, Maithel SK. Defining the Role of Lymphadenectomy for Pancreatic Neuroendocrine Tumors: An Eight-Institution Study of 695 Patients from the US Neuroendocrine Tumor Study Group. Ann Surg Oncol. 2019;26:2517-2524.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 24]  [Cited by in F6Publishing: 25]  [Article Influence: 5.0]  [Reference Citation Analysis (0)]
40.  Zhang XF, Xue F, Dong DH, Lopez-Aguiar AG, Poultsides G, Makris E, Rocha F, Kanji Z, Weber S, Fisher A, Fields R, Krasnick BA, Idrees K, Smith PM, Cho C, Beems M, Lv Y, Maithel SK, Pawlik TM. New Nodal Staging for Primary Pancreatic Neuroendocrine Tumors: A Multi-institutional and National Data Analysis. Ann Surg. 2019;.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 31]  [Cited by in F6Publishing: 27]  [Article Influence: 9.0]  [Reference Citation Analysis (0)]
41.  Zaidi MY, Lopez-Aguiar AG, Dillhoff M, Beal E, Poultsides G, Makris E, Rocha F, Crown A, Idrees K, Marincola Smith P, Nathan H, Beems M, Abbott D, Barrett JR, Fields RC, Davidson J, Cardona K, Maithel SK. Prognostic Role of Lymph Node Positivity and Number of Lymph Nodes Needed for Accurately Staging Small-Bowel Neuroendocrine Tumors. JAMA Surg. 2019;154:134-140.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 37]  [Cited by in F6Publishing: 47]  [Article Influence: 9.4]  [Reference Citation Analysis (0)]