The green toad example: a comparison of pattern 1 recognition software 2

Individual identification of animals is important for assessing the size and status of 14 populations. Photo-based approaches, where animals are recognized by naturally occurring and visually identifiable features, such as color patterns, are cost-effective 16 methods for this purpose. We compared five available programs for their power to 17 semi-automatically identify dorsal patterns of the European green toad ( Bufotes 18 viridis ). We created a data set of 200 pictures of known identity, two pictures for each 22 individual, and analyzed the percentage of correctly identified animals for each software. Furthermore, we employed a generalized linear mixed model to identify 24 important factors contributing to correct identifications. We used these results to estimate the population size of our hypothetical population.

individual, and analyzed the percentage of correctly identified animals for each 23 software. Furthermore, we employed a generalized linear mixed model to identify 24 important factors contributing to correct identifications. We used these results to 25 estimate the population size of our hypothetical population. 26 27

Conclusions 28
The freely available HotSpotter application was the software which performed by far 29 the best for our green toad example, identifying close to 100% of the photos 30 correctly. The animals' sex highly significantly influenced detection probability, 31 presumably because of sex-specific differences in the pattern contrast. Population 32 estimates were close to the expected 100 for HotSpotter, but for the other 33 applications population size was highly overestimated. Given the clarity of our results 34 we strongly recommend the HotSpotter software, which is a highly efficient tool for 35 individual pattern recognition. 36 Keywords Amphibia, picture comparison, population statistics, recapture study, 38 39 Background 40 One of the most basic but also most important tasks in conservation biology is to 41 assess size and status of animal populations. Therefore, mark-recapture techniques 42 in combination with population statistics are used to estimate demographic 43 parameters. In many cases such data are collected in long-term studies and allow 44 prediction of population trajectories. Certain mark-recapture techniques require to tag 45 animals with more or less invasive methods ranging from toe clipping in amphibians 46 (1), inserting PIT tags (2) to adding visual markings (3). Alternatively, genetic 47 identification can be used as a non-invasive method (4). 48

49
The most cost-effective method, however, is to make use of naturally occurring body 50 patterns (5-7). Unfortunately, visually matching pictures of such patterns can be very 51 time consuming and, especially for long-term studies, decreases efficiency (8). 52 Therefore, a number of pattern-matching algorithms have been developed which can 53 cut time of photo comparisons, but in many instances, they can also lead to decreased 54 accuracy (5,9,10). Decreased accuracy, in turn, can cause considerable errors when 55 estimating demographic parameters (11,12). Consequently, deciding on the right 56 algorithm to use for the respective data set is crucial. 57 58 Differences between data sets could include distance to the animals (e.g. photos of 59 giraffes from the distance (13) and close-up pictures of tree frogs (14)), background of 60 photos and quality of cameras used. In long-term data sets picture quality and 61 attributes might vary drastically, therefore, in many instances, scientists prepare the 62 photos before comparing the patterns, for example by cropping the pictures to only 63 show the region of interest (ROI). This can be a quite elaborate task, although it is 64 The cropping types are called a, b and c in this paper, and refer to 'whole animal', 92 'head', 'ROI on head', respectively ( Figure S1). In AmphIdent, APHIS and I3SP+ it is 93 required to define a region of interest for every image after loading. For I3SP+ we 94 defined a ROI similar to the shape of cropping type c. For AmphIdent and APHIS we 95 had to adhere to the ROI selection of the respective software. These are squares and 96 rectangles, respectively, which we placed in the same part of the head as in cropping 97 type c. (see Figure S2 for details). 98

99
Pictures were always compared in the same order. The picture comparisons followed 100 the applications' manuals; however, the general procedure was the same throughout. 101 We first added 100 'reference' pictures for each of the used individuals to the 102 application's database, which simulates that we already encountered each of the 103 individuals (once) before. We did this to ensure that each of the new pictures could 104 find a match in the previous record. We then compared 100 pictures of 'unknown' ID 105 to the application database. Usually the user decides whether one of the first few 106 automatically selected pictures is a match or not, but for the purpose of this analysis 107 we only considered the picture ranked first. If this picture was a true match, we 108 recorded a true detection (1), otherwise a false detection (0). AmphIdent and I3SP+ 109 have functions for manual input to improve the performance, we used them to the 110 following extent. In I3SP+ pixels can be chosen to discern the pattern's fore-and 111 background. Our time effort to discern fore-from background did never exceed three 112 minutes. AmphIdent has the option to only compare subareas of the ROI. If comparing 113 the whole ROI did not yield a true detection, this function was used up to two times. 114 We bootstrapped a 95% confidence interval for the mean detection success for each 115 software using the function boot.ci in the package boot (21) in R (22).

145
Apart of the highly significant effect of the software also the sex of the animals 146 influenced probability of correct identification (Table 1). This might be expected as the 147 patterns of females have higher contrasts between the green and white patches than 148 the male patterns (16). Nonetheless, it highlights potential confounding factors when 149 performing capture-recapture analyses based on pattern recognition. The camera type 150 used for the pictures only showed weakly significant effects or no effects at all, we did 151 not find any interaction effect between the camera used for the first and the second 152 photo. Switching cameras from one to another occasion might not be an important 153 factor for successful identification. 154 155 If we used solely the first hit from the respective software application and no additional 158 manual comparison, the population sizes would be overestimated to different extent 159 (Table 2). For the HotSpotter software, the estimated numbers are still close to the 160 actual N, and only a few animals would be added. However, for I3SP+ the estimated 161 size would be more than doubled, in comparison to the real population size. 162 163 We show that using automatic pattern recognition applications can be a highly efficient 167 tool to identify individual animals. The software which worked best for our example 168 was the freely available HotSpotter (17). Using this software, we identified the majority 169 of the individuals correctly, and this even with our stringent detection criterium. 170 Applying the same criteria for all other tested applications yielded detection rates 171 around 60% and below. HotSpotter also allows to process the pictures without any 172 time-consuming pre-processing, i.e. cropping the pictures did not significantly change 173 the detection probability. Note that for animals lacking well-defined color patterns 174 these applications might not be an efficient solution for individual identification (10). 175 Focusing on other distinct features (e.g. arrangement of warts or the pores of the 176 parotoid gland) might be a viable alternative in these cases, possibly demanding 177 different cropping techniques. Furthermore, we showed that differences in the 178 detection probability between individuals and sexes could bias population estimations. 179 Knowing such potential biases (e.g. detection probability for a certain group) and/or 180 error rates could be used to inform and improve population analyses (25). 181

182
Recently, it has been shown that HotSpotter can be used for identification of animals 183 that were photographed from the distance (26). Therefore, apart from the general use 184 in traditional population analyses, matching individuals from photos could also be used 185 in citizen science projects. In cases where citizens are asked to take pictures of certain 186 animals in a specific area, scientists might be able to determine the individual identity 187 from such photos (27). 188 In conclusion, at least for our green toad example and possibly species with similar 189 patterns the freely available HotSpotter software constitutes a major improvement to 190 other more widely used pattern recognition software, such as APHIS and Wild-ID.

Consent for publication 199
All authors consent for publication of this manuscript. 200

Availability of data and materials 201
Raw data are available in the supplementary of this manuscript. 202

Competing interests 203
The authors declare to have no competing interests. 204

Funding 205
The authors received no funding for this study. 206

Authors' contributions 207
SB and LL conceived the study. SB performed the picture comparisons. SB and LL 208 did the statistical analyses. SB, LL, and GG interpreted the data and wrote the paper. 209

Acknowledgements 210
We thank Christoph Leeb, Eileen Heyer, Christian Wappl and the students who 211 participated in the field courses "Populationsbiologie heimischer Amphibien" and 212