Predictive value and accuracy of [18F]FDG PET/CT modified response criteria for checkpoint immunotherapy in patients with advanced melanoma

Purpose Immune checkpoint inhibitors (ICIs) are widely used in metastatic melanoma and dramatically alter the treatment of these patients. Given the high cost and potential toxicity, a reliable method for evaluating treatment response is needed. In this study, we assessed tumor response in patients with metastatic melanoma treated with ICIs using three modified response criteria: PET Response Evaluation Criteria for Immunotherapy (PERCIMT), PET Response Criteria in Solid Tumors for up to Five Lesions (PERCIST5), and immunotherapy-modified PET Response Criteria in Solid Tumors for up to Five Lesions (imPERCIST5). Methods Ninety-one patients with non-resectable stage IV metastatic melanoma who received ICIs were retrospectively enrolled in this study. Each patient had two [18F]FDG PET/CT scans performed before and after ICI therapy. Responses at the follow-up scan were evaluated according to PERCIMT, PERCIST5, and imPERCIST5 criteria. Patients were classified into four groups: complete metabolic response (CMR), partial metabolic response (PMR), progressive metabolic disease (PMD), and stable metabolic disease (SMD). To assess the “disease control rate,” two groups have been defined based on each criterion: patients with CMR, PMR, and SMD as “disease-controlled group (i.e., responders)” and PMD as the “uncontrolled-disease group (i.e., non-responders)”. The correspondence between metabolic tumor response defined by these criteria and clinical outcome was assessed and compared. Results The response and the disease control rates were 40.7% and 71.4%, 41.8% and 50.5%, and 54.9% and 74.7% based on the PERCIMT, PERCIST5, and imPERCIST5 criteria, respectively. PERCIMT and imPERCIST5 showed significantly different disease control rates from that of PERCIST5 (P < 0.001), whereas it was not significant between PERCIMT and imPERCIST5. Overall survival was significantly longer in the metabolic responder groups than in the non-responder groups based on PERCIMT and PERCIST5 criteria (PERCIMT: 2.48 versus 1.47 years, P = 0.003; PERCIST5: 2.57 versus 1.81 years. P = 0.017). However, according to imPERCIST5 criterion, this difference was not observed (P = 0.12). Conclusion Although the appearance of new lesions can be secondary to an inflammatory response to ICIs and indicative of pseudoprogression, given the higher rate of true progression, the appearance of new lesions should be interpreted deliberately. Of the three assessed modified criteria, PERCIMT appear to provide more reliable metabolic response assessment that correlates strongly with overall patient survival. Supplementary Information The online version contains supplementary material available at 10.1007/s00259-023-06247-8.


Introduction
Melanoma is a common skin tumor with increasing incidence, rapid progression, and a high rate of metastasis [1][2][3]. In recent years, the profound understanding of the pathogenesis of metastatic melanoma and the importance of the anti-tumor immune response and regulatory mechanism has created new treatment options for melanoma patients, including systemic treatment with immune checkpoint inhibitors (ICIs), leading to improved survival in patients with advanced stages of the disease [4,5]. According to the latest clinical practice guidelines for the treatment of melanoma from the National Comprehensive Cancer Network (NCCN) and the European Society for Medical Oncology (ESMO), antibodies against programmed death-1 (anti-PD1) alone or in combination with anti-cytotoxic T-lymphocyte antigen-4 (anti-CTLA4) antibodies are considered first-line treatment for unresectable stage III and IV melanoma patients [6][7][8][9].
Despite these dramatic advances, approximately 40-45% of patients do not respond to immunotherapy [1,10]. In addition, the mechanism of action of these agents differs significantly from conventional cytotoxic approaches (stimulation of inflammation rather than direct lysis of tumor cells) [1,11]. This difference in the mechanism of action between immunotherapy and cytotoxic chemotherapy should be considered when assessing response to immunotherapy [12,13].
Different patterns of response to treatment with ICIs have been described [7]. Pseudoprogression is an unusual pattern of tumor response, defined as an increase in lesion size and/ or metabolic activity and/or formation of new lesions followed by a response. Although pseudoprogression has only been reported in 10% of patients [14], a distinctive feature of immunotherapy compared to chemotherapy is the delayed response to treatment with persistent responses that may persist even after treatment is discontinued [2,11,14]. The ICIs also cause side effects that resemble autoimmune reactions and are referred to as immune-related adverse events (irAE), which can raise challenges in interpreting standard imaging techniques [15].
Given these atypical patterns of responses, the efficacy of conventional criteria such as Response Evaluation Criteria in Solid Tumors (RECIST) or PET Response Criteria in Solid Tumors (PERCIST) in the therapeutic evaluation of ICIs is limited [1,5].
[ 18 F]FDG PET/CT, in combination with brain MRI, is the preferred imaging strategy for staging metastatic melanoma [18] and is also considered a relatively reliable tool for immunotherapy monitoring, though, as, with other modalities, it has some limitations in assessing response to immunotherapy [3,5,9,19]. Therefore, several modified scales have been recently proposed to assess response to immunotherapy by [ 18 F] FDG PET/CT imaging, comprising Prediction of Response to Immune Checkpoint Inhibitor Therapy (PECRIT) (2017) [19], PET Response Evaluation Criteria for Immunotherapy (PERCIMT) (2018) [6,13], immunotherapy-modified PERCIST for up to Five Lesions (imPERCIST5) (2019) [20], immune PERCIST (iPERCIST) (2019) [21], and PERCIST up to 5 lesions (PERCIST5) (2022) [22]. Among the above criteria, iPERCIST, like modified morphological criteria, implements the need to confirm progressive disease on follow-up imaging [23,24]. However, PERCIMT, PERCIST5, and imPERCIST5 approaches rely mainly on metabolic changes, require no mandatory follow-up confirmation, and seem to provide a more clear statement regarding disease status after treatment with ICIs. Nonetheless, given recent developments of ICIs in the treatment of advanced melanoma, there is an unmet need to establish standard response criteria in both routines as well as research clinical trials that reliably predict the prognostic outcome.
In this study, we evaluated the predictive value of PER-CIMT, PERCIST5, and imPERCIST5 criteria in response to immunotherapy on [ 18 F]FDG PET/CT for progression-free survival (PFS) and overall survival (OS) of patients with advanced melanoma treated with ICIs.

Patients
In this retrospective study, clinical and imaging findings of 426 consecutive patients with advanced melanoma, who underwent baseline and follow-up [ 18 F]FDG PET/ CT between January 2015 and July 2021, were analyzed. Patients with stage IV metastatic melanoma treated with ICIs according to ESMO guidelines (Supplement 1) [25], including nivolumab or pembrolizumab or the combination with ipilimumab, and who had [ 18 F]FDG PET/CT before and after at least two treatment sessions were included. The exclusion criteria were: metastases confined to the brain, concurrent other treatment [chemotherapy or B-Rapidly Accelerated Fibrosarcoma (BRAF) or MAP/ERK kinase (MEK) inhibitors], second cancer, resectable metastasis, and inadequate imaging parameters at baseline and/or therapy monitoring. A diagram flowchart summarizing the patient selection process is provided in Fig. 1. The patient's demographic and clinical data were recorded and the effect of the ICI regimen and clinical risk factors, including age ≥ 75 years, gender, history of chemotherapy, BRAF mutation, history of treatment with BRAF/MEK inhibitors, and metastasis location, on the patient outcome were evaluated.

Image acquisition and interpretation
Whole-body [ 18 F]FDG PET/CT scans were performed based on a standard protocol [26]. Imaging was performed using two hybrid PET/CT scanners: Siemens 923/Biograph 64 mCT (Siemens Healthineers AG, Chicago, IL, USA) and Philips Ingenuity TF/Gemini TF 16 (Philips Medical Systems, Andover, MA, USA). Acquisition and reconstruction parameters were harmonized between scanners to minimize differences in standardized uptake values (SUV). Crosscalibration of the instrument is performed monthly at our center. Patients fasted for at least 4 h before intravenous [ 18 F] FDG injection. An amount of 4 MBq/kg body weight for each patient was administered. The mean interval between [ 18 F]FDG administration and the start of the acquisition was 60 ± 12 min. A non-contrast-enhanced low-dose CT scan (Siemens: Care Dose 4D, 100 kV, 59 mAs slice thickness 1.2 mm and pitch 1.5, Philips: 100 kV, 33 mAs, slice thickness 1.5 mm, and pitch 0.8) was performed for attenuation correction and precise anatomical location. Slice measuring 3-mm iterative algorithm (Siemens: PSF + TOF and Philips: BLOB-OS-TF) were used for reconstruction in both scanners. PET data were corrected to account for scattering, attenuation, random coincidence, and scanner dead time.
Both PET/CT scanners were EANM Research Ltd./European Association of Nuclear Medicine (EARL/EANM) accredited.
SUV normalized to lean body mass (SUL) was measured by manually drawing a volume of interest (VOI) around the target lesion(s). SULpeak was defined as the highest mean SUL in a 1.0-cm 3 sphere voxel within the selected VOI and was determined automatically using a commercial software provided by scanner manufacture. Lesions with greater uptake than the reference threshold defined on the healthy liver or mediastinal blood pool without a pattern of inflammation or infection were defined as target lesion(s). A preset SUV cut-off window of 0-5 was defined for semi-quantitative and metabolic diameter measurements.

Response criteria
A board-certified nuclear medicine physician reviewed the images while blinded to the patient's previous [ 18 F]FDG PET/CT reports and clinical features.
For PERCIMT criteria, scans were reviewed with a focus on the appearance of new [ 18 F]FDG-avid lesions, specifically in their absolute number and functional size (i.e., > 1.0 cm or > 1.5 cm). Functional size is the maximum metabolic diameter of pathologic lesion (in centimeters) measured on the fused PET/CT images [13]. According to PERCIMT, no new lesions and regression of all metabolically active lesions associated with melanoma were defined as CMR. PMR included no new lesions and remission of some FDG-avid lesions. PMD was defined when more than 4 new pathologic FDGavid lesions, regardless of size, or 3 or more lesions greater than 1.0 cm or 2 or more lesions greater than 1.5 cm were detected. The remainder variations were classified as SMD when neither CMR/PMR nor PMD were seen (Table 1). No extra brain lesion at baseline n=15 No baseline or post-treatment data n=52 Simultaneous BRAF/MEK inhibitors n=2 Advanced second primary tumor n=8 Treatment discontinued n=3 Stage but not received ICIs n=22 Single therapy with ipilimumab n=8 Resectable metastases n=9 Study population n=91 Patients who received ICs underwent baseline and follow-up scans According to PERCIST5 criterion [22], the sum of SULpeak was measured in 5 metastatic lesions (no more than 2 per organ) with the highest SULpeak. The metastatic lesions could be at different locations on baseline and follow-up [ 18 F]FDG PET/CT scans. CMR was defined as complete resolution of [ 18 F]FDG uptake in the metastatic lesions without appearance of new metastasis on PET. PMR was defined as a decrease of more than 30% in the sum of SULpeak. PMD was defined as an increase of more than 30% in the sum of SULpeak in up to 5 prominent lesions or the appearance of new lesions. All other metabolic changes of [ 18 F]FDG-avid lesions that did not fit into the definitions of CMR, PMR or PMD, were considered SMD.
Based on imPERCIST5, the reviewer determined the sum of SULpeak comprised no more than five lesions with the highest SULpeak. The difference in the sum of SULpeak was scored as (sum of baseline SULpeak minus the sum of follow-up SULpeak divided by the sum of baseline SULpeak multiplied by 100) [20]. Using imPERCIST5 criteria, CMR was defined when no new pathologic [ 18 F] FDG-avid lesion was detectable, and all metastases disappeared. PMR included a decrease of more than 30% or an absolute rate of more than 0.8 in the sum of the SULpeak, regardless of a new lesion. PMD included an increase of more than 30% or more than 0.8 absolute rates in the sum of the SULpeak, independent of a new lesion. SMD is classified when the definitions for CMR, PMR, and PMD were not met (Table 1).
We calculated the "response rate" for each criterion separately. For this, the number of patients with CMR and PMR was divided by the number of all patients in the study and multiplied by 100. To assess the "disease control rate," we also considered two groups based on each criterion: the "disease-controlled group" (CMR, PMR, and SMD) and the "uncontrolled-disease group" (PMD). A comparison between the three criteria was performed using these classifications. The predictive value of PERCIMT, PERCIST5, and imPERCIST5 criteria for PFS and OS was assessed. For this purpose, patients with CMR, PMR, and SMD were classified as "responders," and cases with PMD were classified as "non-responders".

Statistical analysis
Data were checked for consistency and normality. Fisher's exact test or Pearson's test was used to analyze cross-tabulations. Generalized linear models with log-normal distributions were used for continuously distributed variables, and LSD tests were used for pairwise comparisons. McNemar's test compared dependent proportions with 95% confidence intervals. Kaplan-Meier analyses were done using log-rank tests based on randomization tests. All reported tests were Table 1 Definition of PET-based modified metabolic response to treatment criteria. PET positron emission tomography, PERCIMT PET Response Evaluation Criteria for Immunotherapy,

Patient characteristics and imaging
Overall, 91 patients with stage IV melanoma referred to our department between January 2015 and June 2017 were eligible to include in this study. The patient's baseline characteristics are summarized in Table 2

Evaluation of response to treatment
Patients were divided into four groups: CMR, PMD, SMD, and PMR, based on each criterion. The CMR patients were the same in all three response criteria. The number and percentage of different classifications in the three approaches are presented in Table 3.
The clinical status of the patients at the end of the follow-up period was also classified with a similar arrangement in four groups (CMR, PMD, SMD, and PMR). One of the 91 enrolled patients had unclear clinical status at the   Table 4.
The PFS values showed a statistically significant difference between responders and non-responders based on all modified response criteria (P values: PERCIMT < 0.001, PERCIST5 < 0.001, and imPERCIST5 0.003), Fig. 3. Based on all three criteria, PFS was significantly longer in patients with CMR than in PMD (P < 0.001). The OS was also higher for CMR compared to PMD using PERCIMT and PERCIST5 criteria (P = 0.007 and P = 0.019, respectively). However, according to imPERCIST5, no significant difference in OS was observed between response groups (CMR, PMD, SMD, and PMR).
On the other hand, the OS was significantly longer in the metabolic responder groups as compared with the nonresponder groups based on PERCIMT and PERCIST5 criteria (PERCIMT: 2.48 versus 1.47 years, P = 0.003; PERCIST5: 2.57 versus 1.81 years, P = 0.017). However, based on the imPERCIST5 criteria, this difference was not observed (P = 0.12). Table 5 shows the overall survival values for the different patient groups.

Discussion
The response patterns obtained after immunotherapy could be different from those observed after usual chemotherapy. From an imaging point of view, immunotherapy-modified criteria have been formulated to help handle these variegate patterns of response and to overcome the limitations of conventional criteria, both for anatomical and metabolic imaging. In the present study, we compared the efficacy of three metabolic response criteria, i.e., PERCIMT, PERCIST5, and imPERCIST5, to define their potentials and limitations.
The distinction of pseudoprogression from true progression is one of the clinician's challenges. We know that 40-45% of patients show no response to immunotherapy [1,10]. On the other hand, the incidence of pseudoprogression does not exceed 10% [27]. Despite it seems the PERCIMT and PER-CIST5 criteria compared to the imPERCIST5 have a more strict approach to classifying patients in PMD, a larger number of patients take place in the PMD based on PERCIMT and, particularly, PERCIST5 criteria. However, the significance of this difference was not assessed because of the small number of patients in each classification. Nevertheless, the significantly higher number of patients classified as having PMD according to PERCIST5 may pose a challenge for treatment decisions in routine clinical practice. On the survival analysis, the patients with PMD based on PERCIMT and PERCIST5 had significantly lower OS compared to the CMR and SMD group (Figs. 3 and 4), whereas the PMD patients according to the imPERCIST5 do not show a statistically meaningful difference in OS compared to other groups. In addition, the non-responder group of the PERCIMT and PERCIST5, which included only the PMD patients, had significantly lower OS compared to the responder group (i.e., CMR, PMR, and SMD), while this difference did not show between these two groups (responders versus non-responders) according to the imPERCIST5 (Fig. 3). In a way, considering the number and functional size of the new hypermetabolic lesion may be more efficient in the diagnosis of patients with PMD than a 30% increase in the sum of SULpeak of the 5 lesions. In the same way, Anwar et al. reported that a cut-off of four newly emerged hypermetabolic lesions based on the PERCIMT criteria has more predictive value for patient outcomes compared to the SUV changes [13]. Differently from our series, Annovazzi et al. reported the PERCIMT criteria had suboptimal performance in the identification of PMD patients (positive predictive value of 52.3%) [4]. Rivas et al. evaluated the metabolic response by PERCIST5 and imPERCIST5 criteria in patients with advanced or metastatic melanoma 3 months after first-line ICI therapy [22]. The authors found similar findings considering the predictive value of PERCIST5 and imPERCIST5 for patient's outcomes. In contrast, in our study, using imPERCIST5 criterion, there was no significant difference in OS between responder and non-responder groups. In   [20]. Kitajima et al. demonstrated patients stratified by imPERCIST5 criteria in CMR and PMR had significantly longer OS in comparison with the PMD and SMD groups (P = 0.001) [28]. The diversity in the results of the latter studies may be related to the in homogenous patient cohorts with various ICI regimes and the duration of the treatment. In addition, they have evaluated either imPERCIST5 criteria or a combination of imPERCIST5 and metabolic PET parameters for assessment of clinical outcome and treatment response. However, to our best knowledge, this is the first study that compared the predictive value of three modified metabolic imaging criteria (PERCIMT, PERCIST5, and imPERCIST5) for clinical outcome in a relative homogenous patient's population regarding clinical stage of the disease, ICI regimes, and the duration of the treatment.
Previous studies have reported response rates of 26-40% for different regimens of ICIs in melanoma patients [29,30]. In the present work, we found response rates of 40.7-54.9% using three criteria, with comparable results between PERCIMT and PERCIST5, but significantly different to those of imPERCIST5 (Table 3). It should be noted that the type of ICI agents could be a source for this difference between the various studies. The accordance between response to treatment and clinical status at the end of the follow-up period based on the PERCIMT, PERCIST5, and imPERCIST5 was 48%, 53%, and 41% respectively. Response estimated by PERCIMT and PERCIST5 appears to be more consistent with clinical status at last follow-up. One of our limitations was the relatively small population used to develop these criteria. It seems more comprehensive studies are needed for verification of the modified criteria. Additional larger studies with prospective designs can serve as a good guide to achieving consensus criteria for response assessment in melanoma patients treated with ICIs. It is also necessary to apply the modified criteria in standardization approaches of clinics. In addition, the PET/CT acquisitions were performed with two different scanners. This may result in non-identical SUV values between two scanner types. Furthermore, the recent EANM/SNMMI/ANZSNM joint practice guidelines for [ 18 F]FDG PET/CT imaging in immunomodulatory treatment recommend the use of spleen-toliver ratio as a sign of the systemic immune response [24]; however, this was not analyzed in this study. Finally, given the retrospective design of this study, we were not able to accurately assess all irAEs and correlate them with imaging and clinical findings. Therefore, it was not possible to make a reliable statement about the possible misclassification of irAEs by the modified response criteria.

Conclusion
The results of this study indicate that although the appearance of new lesions may be secondary to an inflammatory response to ICIs and indicative of pseudoprogression, given the higher rate of true progression, the appearance of new lesions should be interpreted deliberately. Of the three evaluated modified criteria, PERCIMT appears to provide more reliable assessment of metabolic response that correlates strongly with overall patient survival.
Funding Open access funding provided by Paracelsus Medical University.

Data availability
The detailed data generated during and/or analyzed during the current study are available from the corresponding author upon reasonable request.

Declarations
Ethics approval This retrospective study was performed in accordance with the principles of the 1964 Declaration of Helsinki and its later amendments or comparable standards and approved by the routing ethics committee of the province with the trial number of "EK-1012/2021". Consent to participate I Signature of the written informed consent was obtained from all individuals in the setting of performing [ 18 F]FDG PET/CT examinations.

Competing interests The authors declare no competing interests.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.