Advantages of manual and automatic computer-aided compared to traditional histopathological diagnosis of melanoma: A pilot study

Background: Cutaneous malignant melanoma (CMM) accounts for the highest mortality rate among all skin cancers. Traditional histopathologic diagnosis may be limited by the pathologists ’ subjectivity. Second-opinion strategies and multidisciplinary consultations are usually performed to overcome this issue. An available solu-tion in the future could be the use of automated solutions based on a computational algorithm that could help the pathologist in everyday practice. The aim of this pilot study was to investigate the potential diagnostic aid of a machine-based algorithm in the histopathologic diagnosis of CMM. Methods: We retrospectively examined excisional biopsies of 50 CMM and 20 benign congenital compound nevi. Hematoxylin and eosin (H & E) stained WSI were reviewed independently by two expert dermatopathologists. A fully automated pipeline for WSI processing to support the estimation and prioritization of the melanoma areas was developed. Results: The spatial distribution of the nuclei in the sample provided a multi-scale overview of the tumor. A global overview of the lesion ’ s silhouette was achieved and, by increasing the magnification, the topological distribution of the nuclei and the most informative areas of interest for the CMM diagnosis were identified and highlighted. These silhouettes allow the histopathologist to discriminate between nevus and CMM with an accuracy of 96% without any extra information. Conclusion: In this study we proposed an easy-to-use model that produces segmentations of CMM silhouettes at fine detail level.


Introduction
Cutaneous malignant melanoma (CMM) accounts for the highest mortality rate among all skin cancers and has seen an important increase in incidence in the past few decades (approximately of 3-4% per year) [1,2].The gold standard for melanoma diagnosis is histopathological examination, which is necessary for the correct staging of the tumor and for clinical and therapeutic decision-making [3].Regarding the histopathologic examination, dermatopathologists or pathologists usually start the evaluation by observing the so-called "silhouette" of the lesion [4,5].This term, introduced by Ackerman in 1985, describes the morphologic aspects of tumor extension [4][5][6].He further asserted that the importance of silhouette delineation, at first glance, could be superior to that of other cytological details in determining the benign or malignant nature of a neoplasm [4][5][6].Ackerman first proposed these criteria for the histopathological diagnosis of melanoma in defining his pattern analyses method and for many dermatopathologists little has changed since then [4][5][6].Estimates of the diagnostic accuracy of pathologists' interpretations of melanocytic lesions have been reported by several studies and interobserver evaluation discrepancy may range from 8 to 25-26% [7][8][9][10]25].These discrepancies are higher for challenging melanocytic lesions and in particular in CMM diagnosis could lead to the over-or undertreatment of patients [9].Multidisciplinary consultancies are usually performed, whereby clinicians may add pertinent clinical information and send dermatoscopic images to the pathologists [11,12].Second opinion policies are also frequently used by university-based institutions in Italy.A recent study from the US reported that in the absence of a second opinion for pathologic interpretation of melanocytic skin lesions, 16.8% of cases would receive a reference-disconcordant diagnosis, resulting in 16,850 disconcordant diagnoses per 100,000 biopsies in the US each year with health care costs during the subsequent year estimated at $132,301,000 (95% CI, $130,992,000-$133,625,000) [12].However, second opinion pathologic consultancies are not free of cost and may sometimes cause an increase in the time of diagnostic decision making.[12] Therefore, novel advances in technology could direct second opinions for cancer pathologic diagnosis towards the emerging automated or semi-automated computer-based systems [13].
The introduction of whole slide imaging (WSI), defined as the scanning/digitization of traditional glass slides, along with machine learning and deep learning techniques, has proved to be an important step in this direction [14][15][16].Such automation and digitalization significantly decreases the s sample processing time and lowers costs, allowing a more efficient and standardized diagnostic process.Currently, most of the promising computer-aided diagnostic softwares are limited to dermoscopic image analysis, and few of them are able to analyze histological images [15].The difficulty in creating a computer-aided analysis of WSI is due to the large dimensions of the samples involved, which require complex, ad-hoc algorithms and techniques [14][15][16].In fact, in order to develop a robust processing algorithm, it is necessary to take into account multiple variables, such as sample preparation and staining, the type of image acquisition software, and the resolution of the acquired image [14][15][16].These obstacles therefore need to be overcome in order to allow Computer-aided Diagnosis (CAD) systems to become a feasible option as an automated second opinion.This would increase the objectivity in histopathological evaluations by decreasing the amount of interobserver variability [17][18][19][20].One aspect that could especially benefit from these systems is the detection of the above-cited tumor silhouette.As one of the first steps in the melanoma diagnostic process, an increase in objectivity and standardization of silhouette detection could provide an important advantage in terms of time and reproducibility.

Materials and methods
This study was performed in collaboration with the eDIMESLab and the Melanoma Unit of the Dermatology Department of the IRCCS Sant'Orsola-Malpighi Hospital, University of Bologna (Italy).The study was approved by the local medical ethics committee and was conducted from March 2020 to April 2021 (completion of data analysis and manuscript approval by all authors).The histologic glass slides of 50 CMM and 20 benign congenital compound nevi were collected by the Laboratory of dermatopathology of the Dermatology Department of Sant'Orsola-Malpighi Hospital.The WSI were subsequently acquired using a Nano-Zoomer 2.0-RS Hamamatsu scanner with a 40x (0.23 µm/pixel) magnification and autofocusing.All the examined samples were obtained from deep excisional biopsies or wide excisions.The hematoxylin and eosin (H&E) stained WSI were then reviewed independently by two expert dermatopathologists (C.M. and C.R.), both with > 5 years of experience in dermato-oncology.Histological evaluation for CMM cases included Breslow thickness, mitotic rate, presence of ulceration, inflammation, and tumor regression.Clinical data were evaluated for each case, including gender, age at diagnosis, tumor location, and macroscopic diameter of the lesion.A fully automated pipeline for the estimation of the melanoma areas and the detection of the lesion silhouette was developed.Finally, the results of the automated silhouettes were compared with the silhouettes drawn manually by the dermatopathologists.

Computer-aided diagnosis system pipeline
We developed a novel fully automated pipeline for WSI processing that allows the estimation of the area defined as melanocytic lesion.The processing steps of this novel pipeline (Fig. 1) are as follows: (i) Automated identification of the sections in the WSI field of view and removal of possible confounders (such as bubbles, dirt, and pen marks); (ii) Automated identification of the epidermal surface through a thresholding algorithm; (iii) Rotation of the section: using the epidermal surface as a reference to introduce a measurement of depth; (iv) Automated, non-supervised nucleus identification, and the evaluation of morphological features in relation to their shape and the characteristics of their surroundings; (v) Automated selection of the nuclei according to their morphological features, guaranteeing a multi-level overview of the section.In this system, the cells unrelated to the melanocytic lesion, such as inflammatory cells, are filtered out; (vi) Automated spatial statistical analysis of the cells' distribution.A detailed analysis of the software will be presented in a future work.The full pipeline was developed using the C+ + programming language to guarantee a fast elaboration of the images.

Heatmap elaboration
The proposed pipeline was applied to the entire set of available sections using a workstation machine (64 GB RAM memory and 1 CPU i9-9900 K, with 8 cores) with an average computational time of less than 5 min per section.Each WSI includes several consecutive sections of a single sample, each of which was independently analyzed following all the pipeline steps, collecting the spatial distribution of the detected cells.The resulting spatial distribution highlights the density map of the cells in the section.Superimposing this distribution on the original image allows its spatial registration on the WSI.We further organized these distributions into levels of intensity according to nuclei density, with each level being associated to a range of density scores and assigning a different (incremental) color for the clinical visualization (Figs. 1 and 2).In this work, a range of 10 density levels were used, and the distributions taken from each WSI were standardized according to the available sections.The shapes of the resulting density map act as a silhouette approximation of the neoplastic area.For the visualization of the results and the management of the estimated silhouettes, a dedicated plug-in on the Sedeen Viewer software was developed.The Sedeen Viewer software is supported by the Pathology Image Informatics Platform (PIIP) designed by Martel et al. [17] and provides an easy integration of our pipeline into other WSI programs.In our application, we additionally provided the clinician with the possibility to turn on/off each level of the density distribution, thereby guaranteeing an incremental accuracy of the density score and providing the expert with the option of choosing the desired level.

Results
We included 50 patients (28 males and 22 females, mean age 67 and 64 years, respectively) with a diagnosis of superficial spreading CMM and 20 benign congenital compound nevi (11 males and 9 females, mean age 42.2 years).Clinical and histopathological results are summarized in Table 1.

Automated silhouette definition
The spatial distribution of cells allows a multi-scale overview of the sample at different resolution levels, which are represented by a heatmap overlayed on the WSI.A global overview of the lesion silhouette can be achieved using the low-resolution levels of the WSI.This highlights the spatial distribution of melanocytes in both the epidermis and dermis areas, guaranteeing the detection of a possible melanocyte invasion of the upper epidermis in a pagetoid growth.The qualitative measurement of the melanocytes' structural disorder within the tissue allows a first fast evaluation of the section.A symmetrical, vertically oriented, wedgeshaped silhouette with a smooth, well-defined edge is associated with a benign lesion; an asymmetrical silhouette with a jagged and poorly defined edge is defined as a malignant lesion.The heatmap is an easy-touse visualization for assessing the level of clustering of detected cells.

Comparison of the automated silhouette with the one drawn manually by board-certified dermatopathologists
The WSI were evaluated by the two dermatopathologists as follows: (i) the first expert manually annotated each WSI, outlining the melanoma and nevus silhouettes; (ii) the second expert blindly estimated the agreement between the areas identified by his colleague and the result of the automated algorithm.In 94% (47/50) of melanomas and 100% (20/ 20) of nevi analyzed, the automated silhouette identified by our method is compatible with those manually contoured by the expert dermatopathologist, with a qualitative superiority to the manual scheme in identifying melanocytes.Furthermore, the two pathologists confirmed that the areas with the highest intensities of malignant melanocytes identified by our method correspond to the region of interest for melanoma diagnosis related to the presence of high nuclear density, nuclear atypia, and neo-angiogenesis.This evaluation is a crucial step in testing the robustness of our method, but is even more important for the identification of possible issues and misleading detections.

Diagnostic power of the lesion silhouette
The validated automated silhouettes were submitted to the dermatopathologists involved in this study.Each silhouette was overlayed on the original WSI, and the opacity of the resulting image was adjusted to partially obscure the underlying histological source.During the blind evaluation, the expert was asked to determine the potential malignancy of the underlying lesion in relation to the silhouette shape (Fig. 3).The opacity of the original histological image was reduced to focus the attention of the expert as much as possible on the outlined area.We decided not to completely obscure the histological image source since the spatial localization of the silhouette is a critical indicator for diagnosis, together with its relative position with respect to the epidermis.In Fig. 2. The silhouette identified by our method is defined by a range of density scores.We used incremental colors (from yellow to red) to represent those levels for the clinical visualization.The method is comparable with the silhouette manually contoured by the expert histopathologist (blue line), and in some cases qualitatively superior in the identification of tumoral areas.this way we determined the informative power of the cell distribution for the expert diagnosis.The expert evaluated all the available sections independently, without time limits, and using an HD computer monitor (HP Z27 UHD 4 K, 27 ′′ , 3840 ×2160 resolution).Each section was labeled as "melanoma" or "nevus", and the results of the blind evaluation were compared to the ground truth diagnosis.Considering the total section-by-section analysis (156), 18% of the silhouettes led the expert to an incorrect diagnosis (melanoma-like silhouette vs. a diagnosis of benign nevus and nevus-like silhouette vs. a diagnosis of melanoma).This non-negligible error rate can be reduced by considering multiple (at least 2 to a maximum of 6) sections for each patient in order to increment the statistical population, which adds robustness to this analysis.After adjusting for this consideration by grouping together the sections belonging to each patient, only 1 patient (4% of error rate) was incorrectly classified by the expert.

Discussion and conclusion
In the diagnosis of melanocytic skin tumors, the assessment of the overall architectural pattern, the silhouette of the lesion, is fundamental [4,6,10].The most promising advances in histopathologic diagnosis are supported by new technologies and are based on image processing techniques [10,[20][21][22].Most published studies attempt to exploit deep neural networks in order to assess histomorphological features in H&E slides [24].However, deep neural network models require extensive training and multiple validation sets.Hekler et al. [23] applied a deep learning algorithm for the first time in histopathological melanoma classification, showing a discrepancy between a convolutional neural network and the histopathologist in 19% of the images.These results were comparable to the pathologist interobserver variability described in the literature [7][8][9][10].The goal of our model is to overcome the limitations of supervised approaches, thus allowing it to be applied without relying on manually labeled samples.In the literature, several deep learning models have been proposed to automate the processing of WSI that require manually annotated sets of images in order to obtain reproducible training of the parameters [16,19].This step is normally difficult and time-consuming to perform at the accuracy level required for a precise segmentation model.
Many authors have already proposed automated pipelines for the segmentation of histological samples, mostly focusing on the analysis of small patches of each section, usually at lower resolution than the one allowed by the WSI [17,[20][21][22].This approach guarantees a fast evaluation of the section, but leads to a rough segmentation of the underlying physiological structures.The proposed method performs the analysis on the entire WSI without the need for a subdivision in patches.Therefore, it is possible to obtain segmentations and silhouettes at high resolution that are easy to use and validate by the experts.The developed pipeline requires an elaboration time compatible with clinical application (an average of about 10 min per section).The most time-consuming step (87% of the time) concerns the segmentation and classification of the cells, which requires a high-resolution analysis of the WSI.The pipeline can be further optimized using computation accelerators, such as graphic processing units (GPU).In accordance with Dall'Olio et al. [18], the use of distributed computing for high memory-consuming data should be discouraged, since the best computational performances are reached by a concurrent parallelization framework.
In conclusion, we have shown that the proposed method allows for a quick identification of the most informative regions of interest for diagnosis.The agreement between the human-drawn silhouette and the areas identified by the automated algorithm in our study ranged from 94% in melanomas to 100% in nevi and the diagnostic power of these Fig. 3. Examples of benign (left) and malignant (right) lesions in relation to the silhouette shape.The silhouette contour was filled to obtain a dense heatmap of the segmented area and the underlying histological source was partially obscured to focus the attention of the expert on the segmented area.In the case of nevus silhouette, the shapes are symmetrical, vertically oriented, wedge-shaped, and sharply circumscribed with smooth borders and flat base.An asymmetrical silhouette with a jagged and poorly defined edge is defined as a malignant lesion.
silhouettes was validated by two dermatopathologists.This result shows the importance of the lesion silhouette for the formulation of the dermatopathological diagnosis and the importance of an accurate detection of the lesion area for the discrimination between benign and malignant skin lesions.
Even with these promising results, numerous challenges remain.Among the principal limitations, we noticed that in some cases the automated identification classified tissue characterized by visible inflammation or skin appendages, such as glands and hair follicles, as melanoma-like regions.Both biological structures are characterized by a high level of cellular density and, without a subsequent classification of the nuclei, are difficult to discard by the described method.However, we wish to point out that with further improvements in the WSI processing pipeline, these sources of misclassification can be automatically removed, avoiding false positive detection.Furthermore, the failure to include an estimation of the level of maturation of melanocytes through the lesion depth is a problem that will certainly need to be addressed.Maturation with depth is a known important parameter in lesion classification, and will definitely be included in future models, but it was excluded from the one presented here in order to focus on the predictive capability of the density-based heatmap.Finally, the proposed algorithm will need to be tested on samples from different laboratories to assess the real clinical reliability of the method.
We aim to extend and improve this method for other histological applications involving difficult diagnoses such as dysplastic nevi and different subtypes of melanomas.The use of AI techniques in the future will not aim to reduce the importance of pathologists, but could probably offer "an automated second opinion" that may be used to formulate the final diagnostic decision in a time-efficient and homogeneous manner.

Fig. 1 .
Fig. 1.Heatmap elaboration.a: WSI comprises multiple consecutive of a single sample, each of which was analyzed independently; b: Sample extraction with the removal of possible confounding factors, identification of the epidermal surface and the rotation of the section; c: Automated and unsupervised cell identification and classification based on their morphological characteristics; d1-d2: The resulting spatial distribution highlights the density map of the cells in the section and the superimposition of this distribution on the original image allows its spatial registration with the WSI with silhouette delineation.

Table 1
Clinical and histopathological data of the enrolled patients.