A country-wide teledermatoscopy service in Estonia shows results comparable to those in experimental settings in management plan development and diagnostic accuracy: A retrospective database study

Background Teledermatoscopy accuracy has been examined in experimental settings and is recommended for primary care despite lacking real-world implementation evidence. A teledermatoscopy service has been provided in Estonia since 2013, where lesions are evaluated based on the patient’s or general practitioner’s suggestion. Objective The management plan and diagnostic accuracy of a real-world store-and-forward teledermatoscopy service for melanoma diagnosis were evaluated. Methods A retrospective study analyzed 4748 cases from 3403 patients using the service between October 16, 2017 and August 30, 2019 by matching country-wide databases. Management plan accuracy was calculated as the percentage of melanoma found that was managed correctly. Diagnostic accuracy parameters were sensitivity, specificity, and positive and negative predictive values. Results Management plan accuracy for melanoma detection was 95.5% (95% CI, 77.2-99.9). Diagnostic accuracy showed a sensitivity of 90.48% (95% CI, 69.62-98.83) and a specificity of 92.57% (95% CI, 91.79-93.31). Limitations Matching the lesions was limited to SNOMED CT location standard precision. Diagnostic accuracy was calculated based on a combination of diagnosis and management plan data. Conclusion Teledermatoscopy for detecting and managing melanoma in real-world clinical practice displays results comparable with those in experimental setting studies.


INTRODUCTION
Early detection and treatment of skin cancer are essential to improve prognosis. [1][2][3][4] Teledermatology helps to enable earlier assessment and treatment. 5 Professional organizations suggest teledermatoscopy for pigmented lesions for everyday care, 6 although most research relies on experimental setting studies where the service is provided for a limited time and to a limited patient population in specially controlled environments. [7][8][9][10][11][12][13] Financially selfsustainable teledermatoscopy services have been evaluated 14 ; however, research on melanoma management plan and diagnostic accuracy in these regular clinical practice settings is lacking because of methodological difficulties, such as the need for patient follow-up or excision of all suspicious lesions for histopathology. 15 Here, a different approach was taken using Estonia's long-established country-wide national health information system (NHIS), which was matched with a country-wide teledermatoscopy service database (TDDB). Active since February 8, 2013, the service had helped to evaluate 11,658 cases and had been used by 103 different general practitioners (GPs) and 11 different dermatologists by August 30, 2019.
The study aimed to evaluate the management plan accuracy of a financially self-sustaining storeand-forward teledermatoscopy service for melanoma diagnosis at the primary care level. Because the working diagnosis is not regularly provided as part of the management plan, especially when melanoma is suspected, diagnostic accuracy was calculated based on a combination of diagnosis and management plan data.

Hypotheses
The authors hypothesized that all melanomas (malignant melanoma of the skin -C43 and melanoma in situ -D03) would be correctly managed by dermatologists via the teledermatoscopy service.
The authors hypothesized that the diagnostic accuracy of teledermatoscopy would be similar in real-world and experimental settings. [7][8][9][10][11]13 METHODS This retrospective 2-database study was conducted in accordance with the World Health Organization Declaration of Helsinki, the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) checklist for retrospective studies 16

Index test
The index test was defined as the teledermatoscopy examination that resulted in a management plan (''Excision,'' ''Biopsy,'' ''Visit dermatologist,'' ''Checkup nevus in 1, 3, 4, 5, 6, or 12 months,'' and ''No further action needed'') and a dermatologist diagnosis provided as an International Classification of Diseases (ICD-10) code. The management plan and diagnosis were provided based on dermatoscopic and macroscopic images of the lesion taken at the GP office by a doctor or nurse with an iPhone 6 or higher using a Dermlite DL1 smartphone add-on dermatoscope. The data were retrieved from the TDDB, which also includes lesion-specific information on location and size as well as patients' clinical and baseline demographic data.

Reference test
A positive reference test was defined as a melanoma histopathology diagnosis obtained from the NHIS, including information on the location, time, and date of the lesion's excision. The pathologist analyzing the histology specimen had no access to the photographs of the lesion. A negative reference test was defined as the absence of a melanoma histopathology diagnosis within 1.5 years after the dermatologist's diagnosis.
The SNOMED CT browser Estonian version, 18 the NIH ICD conversion program, 19

Service process and participants
Teledermatoscopy was demanded either by patients or recommended by GPs. The request for evaluation, including patient and lesion anamnesis and images, was sent by a GP to a dermatologist. Management plan and diagnosis were provided approximately within 2 days and reported to the patient by the GP. If indicated, they referred the patients for an excision, which then resulted in the creation of a histopathology diagnosis entry in the NHIS.
All patients who used the service between October 16, 2017 and August 30, 2019 (5389 cases) were included in the study. The final sample consisted of 4748 cases from 3403 patients. Each case was considered independent to determine diagnostic accuracy. 8 The minimum required sample size of 2930 cases for a maximum 95% CI width of 10% was calculated based on the method by Buderer assuming a melanoma prevalence in the sample of 1.85%, 12 sensitivity of 0.83, and a specificity of 0.92 for the detection of melanoma. [22][23][24] Analysis Matching. The index and reference tests were matched if they met the following criteria: (1) the latter was performed within 1.5 years (= 548 days) after the former, (2) they had the same patient ID, and (3) similar location of lesions. Missing or inconclusive data on 1 of the 3 criteria led to an exclusion of the patient before matching.
Generally, if multiple consecutive index tests concerned the same lesion, the later ones were excluded. Only if the first index test contained a management plan recommendation for a checkup and could not be matched with a positive reference test, the first index tests were excluded.
If multiple positive reference tests could be matched with the same index test, the 2 with the least time passed in between, were matched.
If there were still several options to match the index with reference tests, after the rules set out above were applied, then the matching was conducted twice, assuming the worst-and the best-case scenario, in which the primary priority was to achieve either the lowest or the highest number of false-negative cases possible, respectively. For readability purposes, the study presents only the worstcase scenario.
After the matching, inconclusive index tests were excluded along with reference tests without matched index tests. The definition of inconclusive differs between management plan and diagnostic accuracy calculation (see below).
Management plan accuracy. The management plan accuracy was defined as the percentage of any melanomas found by histopathology that were managed correctly with an ''excision'' or ''biopsy'' 25,26 or a ''visit dermatologist.'' A management plan to checkup in 1 month was considered inconclusive and excluded from the calculation; all other management plans were considered incorrect.
Diagnostic accuracy. For diagnostic accuracy calculations, an index test was considered positive, if the diagnosis was melanoma (C43/D03) or if the diagnosis was neoplasm of uncertain or unknown behavior (D48.5 and D48) and the management plan was ''excision'' or ''biopsy.'' Valid inconclusive index tests were those recommending ''visit dermatologist'' or ''checkup nevus in 1 month.'' Invalid inconclusive were index tests, in which images were deemed unusable by the dermatologist. Both valid and invalid inconclusive index tests were excluded from binary statistics but reported and considered for the test yield. 27,28 All other combinations of diagnoses and management plans were considered negative.

Participants and cases
Data retrieval from TDDB produced 5389 potentially eligible index tests. Of these, 5272 contained data for matching criteria (1), (2), and (3), which allowed matching with 23 positive reference tests. Of these 23, one was excluded after matching because it could be matched with the same index test as another similar positive reference test that was performed earlier. Another positive reference test was matched with a valid inconclusive index test in which the management plan recommended visiting a dermatologist and was thus only eligible for management plan accuracy calculation. Therefore, 22 positive reference tests were included for the calculation of management plan accuracy and 21 for the calculation of diagnostic accuracy. Next, 469 index tests were excluded because of 3 different reasons for inconclusiveness, leaving a total of 4748 cases for analysis. Specifics of matching, inclusion process, and exclusion reasons can be seen in Fig 1 and Table I.

Test results
Of the 22 matched diagnoses, 16 were C43 and 6 were D03. Table I shows the overview of melanoma The cross-tabulation is displayed in Table II. The test yield was 92.6% (n = 4748 cases divided by all cases n= 5389 minus invalid cases n = 172 and minus invalid inconclusive results n = 91). Management plan and diagnostic accuracy calculations are shown in Table III.
The mean age of participants counted per case was 39, with an SD of 17.706, range of 0 to 93. Women were predominantly using the service (n = 3126), which is 65.8% of all cases. Additional characteristics referring to the risk of developing melanoma are shown in Supplementary Table 1 Fig 1. STARD flow diagram expanded for case inclusion from NHIS and TDDB databases and matching. *Case matching process steps were as follows: 1. Data cleanup before matching process e cases contained valid information on patient ID, date, and nevus location. Additionally, the melanoma histopathology diagnosis was performed within 1.5 years (= 548 days) after the index test and cases had the same patient ID. 2. Matching process e cases were matched if the locations of the lesions matched. 3. Exclusion after the matching e inconclusive index tests and duplicate diagnoses were excluded. **The reference test was histopathology. If no histopathology diagnosis could be found in the NHIS within 1.5 years after the index test (teledermatoscopy examination), the reference test was considered negative. ***The index test was positive when the diagnosis was C43/D03 or D48.5/D48 with the recommendation to perform ''excision'' or ''biopsy.''

Management plan and diagnostic accuracy
Comparable studies reported management plan accuracy as 95.83% to 100% (1 out of 24 melanomas was misdiagnosed as squamous cell carcinoma), 7 100% (1 out of 48 melanomas was misdiagnosed as solar lentigo but recommended for monitoring), 9 93.75% (of 16 total melanomas, 2 melanomas were diagnosed as activated melanocytic nevus and junctional nevus but excised, 1 melanoma was diagnosed as a multiforme reaction), 10 100 % (5 melanoma correctly managed), 11 and 91.3% (2 out of 23 melanomas potentially mismanaged). 13 Although the reference standard for benign lesions was a face-to-face clinical examination, only 2 studies implemented a follow-up plan to confirm nonmalignancy. 9,11 Hence, melanomas might have been overseen. This study has comparable results with 95.5% (95% CI, 77.2-99.9; 1 melanoma mismanaged out of 22). The potential mismatch of a D03 histopathology diagnosis with a D22 teledermatoscopy diagnosis and a 12-month checkup recommendation was considered mismanagement.
There was only one incident where 2 histopathology diagnoses were registered for one patient (see lesion no. 15 in Table I). This suggests that lesiondirected screening performs similarly to total body examination. 30

Aspects of study design
The statistical prevalence of confirmed melanoma in the sample (0.44%) is similar to that in populationbased screening programs 30,31 and smaller than in prospective studies (eg, 9.38%, 7 37.5%, 9 35.6%, 10 2.13%, 11 and 3.8%. 13 ) In those studies, medical personnel (eg, GPs) (pre) selected the patients who were suspected of having malignant lesions, but in the teledermatoscopy service under evaluation, mostly the patients themselves initiated the teledermatoscopy examination. While prospective studies need a higher prevalence for economic reasons, for this study a natural prevalence sufficed owing to the high sample size of 4748 (5389 before exclusions) recruited during everyday health care provision.
The retrospective study design posed challenges. One limitation was matching the lesion locations because histopathology diagnoses had low SNOMED CT location standard precision (eg, ''entire skin of back''), unlike the TDDB that allowed selecting lesion locations by finger tapping on the smartphone screen. The problem of documenting anatomic body surface locations is generally known in the field of dermatology and is currently at the center of scientific debate. 32 As a result of the low precision, there was ambiguity concerning the matching of teledermatoscopy examinations with positive histopathology examinations in 2 cases. The authors took a conservative stance and highlighted only the results of the worst-case scenario, although the best-case scenario was equally plausible. In the equally likely best-case scenario of management plan accuracy, the recommendation to checkup the one mismanaged melanoma (see lesion no. 7 in Table I) in 6 months was considered mismanagement but might have helped to detect this in situ lesion earlier than the 12 months checkup recommendation in the worst-case scenario. In the equally likely best-case scenario of diagnostic accuracy, 1 of 2 misdiagnosed melanomas (see lesion no. 2 in Table  II) was diagnosed as a D03 with a management plan to excise and thus would have improved the diagnostic accuracy, leaving only one misdiagnosed melanoma.
A diagnosis-reporting protocol cannot be established in studies relying on retrospective data. Because it was common practice for dermatologists not to give melanoma (C43) or melanoma in situ (D03) diagnoses without histopathology confirmation, they often diagnosed a neoplasm of uncertain or unknown behavior: skin (D48.5 or D48) and recommended excision or biopsy. Thus, those cases had to be considered as positive index test results, possibly inflating the sensitivity. This also greatly limits the comparability with other studies of the diagnostic accuracy of teledermatoscopy.
Because the diagnostic accuracy of dermatoscopy depends on the level of the examiner's experience, [33][34][35] it might have differed based on the examiners' various expertise levels in this study and was potentially decreased. The same is true for the persons taking the images. This disadvantage must be considered when comparing the results with those studies with more experimental settings employing only expert dermatologists and experts taking the images.
A major strength of the study was the 1.5 years observation period, which was enabled by the country-wide database NHIS that archived all information on histopathology reports in the examined time frame. If a rapidly growing melanoma was missed, it would likely have caused clinical symptoms in 1.5 years and led to its excision or biopsy. 3 Then, a histopathology report in the NHIS would have been created and a false-negative case could have been detected. Although Wang et al 25 considered a 1-year period to be a sufficient time for missed melanomas to be detected, slow-growing melanoma, however, might be missed by this approach because they quadruple in the area only in 3.5 years. 36 A future follow-up research based on the same data sources would enable to capture of a longer time period and further leverage the opportunity of having a country-wide database of histopathologic reports for analysis.

CONCLUSION
Teledermatoscopy implemented in a regular health care setting as a screening test for melanoma detection shows management plan and diagnostic accuracy comparable with that of teledermatoscopy examined in experimental setting studies.   29 z The prevalence of melanoma in the sample was 0.44 % (95% CI, 0.27-0.68).

Conflicts of interest
x Diagnostic accuracy was calculated based on diagnosis and management plan data.

‫װ‬
Only concerns the management of histopathologically confirmed melanoma (N = 22).