Are extinction opinions extinct?

Introduction

The quality of sighting records of rare species, and particularly those that are approaching extinction, vary considerably. This can lead to confusion, particularly when identifying whether a species is extinct, or identifying when a species went extinct (Roberts et al., 2010). An extinction date for a given species is usually inferred from the rate of sightings by assuming that the rate changes after the point of extinction. Recent models (Solow et al., 2012; Thompson et al., 2013, Lee, 2014; Lee et al. 2014 and Jarić & Roberts, 2014) incorporate uncertain sightings as well, thus recorded sightings might occur after extinction. A review of using sightings records to infer extinction is provided by Boakes, Rout & Collen (2015).

Generally sightings are either grouped as certain or uncertain records by researchers, e.g the ivory-billed woodpecker (Solow et al., 2012). The subjective quality rather than the certainty of a record has been less investigated. To incorporate the varying quality of sighting records, Thompson et al. (2013) and Lee (2014) present a method which allows several different classes of uncertain records, where the classification is determined by subjective quality. Their method is optimal if at least one sighting from each coarsely defined group occurs before the last certain sighting. Any approach requires expert information about the quality of records. Suppose we have high resolution information for the quality of sighting records, that is, we have pooled expert opinions on the quality of each individual sighting record. Does expert information actually improve our inference on extinction estimates? We use the Barbary Lion as an initial test-bed. Collecting pooled expert opinions on individual sighting records is a time-expensive exercise, thus only the Barbary lion sightings currently have this high level of detail. Indeed the primary motivation for this paper is to ascertain whether this cost is necessary.

Lee et al. (2015) provided distributions for 32 alleged sightings of the Barbary lion (Panthera l. leo) which occurred between 1895 and 1956 in Algeria and Morocco. In this paper we use the individual quality score provided by Lee et al. (2015). We also examine the importance of the expert’s prior of the lion being extinct on the results.

The work of Lee et al. (2015) provides several distributions for each lion sighting. One method considers the expert estimates for three different questions relating to the distinguishability of the species, observer competence and verifiability, and pools across experts and questions linearly, while another pools them logarithmically. The distributions from pooling across experts and questions provide a quality distribution for each sighting, which we use in this paper. For clarity, we present results from using the linear pooling distributions only, since, as can be seen from Lee et al. (2015), the distributions are similar, and thus our conclusions will be similar.

We begin by examining these distributions (‘Examining sighting quality’), where it is already implied that individual distributions for the quality of each sighting may lead to counter-intuitive results, and thus expert opinion on an individual sightings should be ignored. However, before confirming this contentious statement, we incorporate these distributions into an existing extinction model to further understand the effect of sighting quality scores on extinction estimates.

For the lion, we consider two methods to include the additional information about the quality of uncertain sightings. Both these methods are extensions of the Bayesian model of Lee et al. (2014), which assumes a constant population prior to extinction. The first method looks for a breakpoint in the sighting quality, where one would assume that the average sighting quality before extinction is higher than the average sighting quality after extinction (when all sightings should be false). The assumption is that this breakpoint broadly coincides with the change in sighting rate. Quality should inform when all sightings must be false, and vice versa. We refer to this method as the ‘quality breakpoint method’. Alternatively, we use the sighting quality as a measure of uncertainty around a sighting record. We refer to this method as the ‘quality as variance method’. To further explore the effect of quality under this method, we also assign random quality measures, such that if results from simulated random quality measures are similar to the results from actual quality measures, then expert quality measures are superfluous.

To further demonstrate our methods on additional data sets we also consider three birds, the Alaotra grebe, Jamaican petrel and Pohnpei starling. Since there are not quality distributions for the individual sightings, for these three birds we use the uniformly distributed sighting qualities provided by Birdlife International (Lee et al., 2014). There are fewer uncertain sightings with the bird species, disqualifying them as a critical tests of the change point method (see Table 1). For the three bird species, only the ‘quality as variance method’ is applied. As with the lion, the model is also run with random quality measures to determine the importance of quality estimates.

As a small addition, we consider the finding of Lee et al. (2014) that the conclusions may depend upon the prior. If one assumes that the prior of extinction is provided by an expert, then perhaps this influence is welcomed. However in our method, for all four species, we use an non-informative prior (Congdon, 2001) effectively integrating over all possible expert’s views. When inferring extinction for a given species it is recommended to always run a model with an uninformative prior. If an expert prior is provided, an additional model with an uninformed prior allows one to observe the effect of the expert’s opinion.

The framework is presented in ‘Model framework’. Within this section we examine the sighting quality, and identify a change point in sighting quality for the Barbary Lion. In ‘The Choice of Prior’ we examine the choice of the expert’s prior, and discuss the influence it has on the outcome, and hence present an alternative, non-informative, prior. In ‘Results’ we discuss our findings: sighting data that consists of certain and uncertain only is the most reliable. Quality is not strongly informative of extinction.

Before discussing the models, it is illuminating to examine the information in sighting quality q itself first. Do any changes make rational sense? We do this with the lion sighting data, looking at the general form of the continuous density assumed for q ∈ [0, 1] (where 1 is certain), and whether q exhibits a change point over time.

Examining Sighting Quality

The elicitation in Lee et al. (2015) was not carried out explicitly under a belief of extinction or non-extinction. Five experts offered a best estimate and lower/upper bounds for three different aspects of sighting quality (in an un-blinded manner) for each sighting at time t. Lee et al. (2015) use the most straightforward way to represent these three points as a probability density, that is as a triangle density. For simplicity we treat experts as exchangeable, ignore any correlation between the best and lower/upper estimates, and also ignore any correlation between the j questions (the differential weights of expert competency does something to adjust for inter-expert correlation as does the exhaustive group elicitation process). The quality density for a given sighting p(Q = q_t) is the result from linear pooling across questions and experts. Note that the distribution resulting from pooling across 15 triangle densities, is not a triangle density. Under the Central Limit Theorem of the sum of identical, independently distributions, one could work in accumulated (normalised) quality measures and thus detect a level change rather than a breakpoint over time as we do herein.

Examination of the raw q_t values is very noisy. A degree of smoothing is needed to see the choice of right density, any pattern (suggesting p(q|E), where E denotes extinction) is or is not equivalent to p(q|notE) and any breakpoint that is informative of extinction. We assume that the first sighting in 1895 (in Morocco) is certain. (Note that no sightings receive a quality score of one, implying no sightings are defined as ‘certain’.) Sweeping across the sightings, sightings are classed as either ‘before’ or ‘after’ the sighting in question, where ‘after’ includes the current sighting. The ‘before’ sightings are combined, as are the ‘after’ distributions, see Fig. 1. Inspection suggests a unimodal distribution like a beta distribution is a sensible choice for the density. A one-sided t-test of the quality data in this way indicates that the before and after distributions first become significantly different to each other in 1929.

Notice that the ‘before’ distribution has a large variance when examining the early sightings, and a similar phenomenon for the ‘after’ distributions for the later sightings (Fig. 1). This is because at these extremes, we have less information. For example, when establishing whether extinction occurred between 1895 and 1898 we are comparing the distribution for the single 1895 sighting with the combined distribution from the 31 other sightings.

The issue with this ‘burn-in’ and ‘burn-out’ is evident when examining the peak for the two distributions. Ideally it would be clear to see that the peak (i.e., the mode) of the distribution for before sightings is initially larger, then a switch occurs around 1929. However the lack of data at the time boundaries makes this more challenging to clearly see from the peaks alone. For example, around 1934 the before and after distributions seem very similar. Further smoothing is needed to see any coherent changes.

So instead we consider the combination of the distributions presented in Fig. 1. We denote Θ_b(year) as the peak of the combined distributions (in Fig. 1) before year. And similarly for Θ_a(year) from combined distributions (in Fig. 1) after year. With this measure it is apparent that a shift in the relationship between these two values occurs around 1930 (Fig. 2), as predicted by the t-test.

The Θ_b(year) and Θ_a(year) allow us to better examine how the quality of sighting changes. The mean of the combined quality of sightings ‘before’ for each sighting year Θ_b(year) increases until 1929, and then the quality remains unchanged, Fig. 2. Conversely, the combined ‘after’ distributions, Θ_a(year), remains reasonably steady and then increases. This phenomenon is unexpected since one would assume that after extinction, around 1930, the quality of sighting would decrease. However, it is likely that this change is due to human factors such as observers still being alive (first hand account), and the increased use of cameras. This already suggests that the quality breakpoint method may be inconsistent with the sightings process.

Lastly, the empirical Bayes Factor (likelihood ratio) for just the quality data alone (below labelled as data_q) is the calculated ratio (1)${\displaystyle B{F}_q=\frac{\ell \left({data}_q|notExtinct\right)}{\ell \left(dat{a}_q|Extinct\right)}=\frac{\ell \left(dat{a}_q|notE\right)}{\ell \left(dat{a}_q|E\right)}.}$ If the species is extant, the ℓ(data_q|notE) is small, giving a log Bayes Factor that tends to negative infinity. At each sighting year we calculate the empirical Bayes Factor and find that before 1929 the log Bayes Factor is indeed approximately zero. After 1929 the behaviour of the Bayes Factor changes. We would expect the Bayes Factor to steadily increase after 1929, however, again we observe how human factors have influenced the before and after distributions to create erratic behaviour. Nonetheless, as in the t-test there is clearly a shift around 1929 where quality increases. Could this be a technological change?

If the lion data is typical, changes in quality may not indicate the breakpoint for extinction, and thus the sighting quality alone is likely to be unreliable to infer extinction. Let us now consider using it in conjunction with the sighting record. Perhaps, together with analysis on sighting rate, quality scores can provide more information than either of sighting rate or sighting quality alone.

Model Framework

The objective is to determine the estimated posterior probability distribution of extinction. By Bayes Theorem (2)${\displaystyle p\left(E|\mathbf{data}\right)=\frac{l\left(\mathbf{data}|E\right)\cdot p\left(E\right)}{l\left(\mathbf{data}|E\right)\cdot p\left(E\right)+l\left(\mathbf{data}|notE\right)\cdot p\left(notE\right)}}$ where p(E) is the expert’s prior on extinction. Let us retain the general form of the existing problem in Lee et al. (2014). That is, we consider the period of observation (0, T) where 0 is the beginning of the sighting record, and T is the length of the sighting record. During this observation period, certain and uncertain sightings occur in parallel. The vector s₁ represents certain sightings (s_1,t) at time t, t ≤ T. Similarly s₂ represents uncertain sightings (s_2,t) at time t, t ≤ T. Our input data comprises of both types of sightings, s = s₁∪s₂. These sightings are used to estimate the posterior probability of extinction and the time at which this extinction occurs. Note that whilst the model does not require an uncertain record s₂, a certain record s₁ with at least two sightings is required. Certain and uncertain sightings are assumed to follow a stationary Poisson process of regular spacing with constant, unknown rates (m₁ and m₂ respectively). Since we include the possibility of false sightings, sightings may occur after extinction, but at a different constant rate to that rate occurring before extinction. This is an offset denoted f₂ as a background for the whole series (0, T). These false sightings by default must only be uncertain sightings. So, given the notation that l means likelihood we obtain the four elements of the model:

• l(s₁|notE) = Bernoulli(1 − e^−m₁)

• l(s₁|E) = Bernoulli(1 − e^−m₁) if the time is before or equal to the time of extinction, with l(s₁|E) = 0 if afterwards

• l(s₂|notE) = Bernoulli(1 − e^{−f₂−m₂})

• l(s₂|E) = Bernoulli(1 − e^{−f₂−m₂}) if the time is before or equal to the time of extinction, with l(s₂|E) = Bernoulli(1 − e^−f₂) if afterwards.

The form (1 − e^−(⋅)) is used for efficient parameterisation. We assume no population decline (see Lee et al. (2014) and references therein). This model of Lee et al. (2014) determines a change point in the sighting rate, which provides an estimate for the year of extinction. The input is two sighting records (certain and uncertain) and the output is a probability that the species is extinct at the end of the sighting record, and a corresponding year in which extinction would have occurred. Our method uses: a uniform distribution from the last certain sighting to the end of the sighting record; a non-informative Jeffreys prior (Beta(0.5, 0.5)) for non-extinction (Congdon, 2001); together with wide uniform prior distributions (range 0 to 100) for m₁, m₂ and f₂ to ensure that there is no bias.

Now consider that our input data comprises of both certain and uncertain sightings, s = s₁∪s₂, and the individual quality of uncertain sightings, q. We interpret individual quality scores as a score for the year in which the sighting occurred. However, we require a quality score for every year, even if no sighting occurred. We take a wide interpretation of what quality is. One could infer that a high quality sighting (e.g., a skin sample that can be tested) implies that the sighting is certain (or close to certain), and conversely, a low quality sighting (e.g., a second-hand verbal account) is less certain to be a true sighting. Note that the quality vector q initially seems to only have a quality assigned during years of uncertain sightings. Later we will discuss how we assign a sighting quality for all other years. We take quality simply to be a subjective attribute of the sighting—no implicit model of its basis is assumed. A method for eliciting the quality measures is provided in Lee et al. (2015).

Let us partition the data into the stochastic sightings, s, and the stochastic quality measure for each of the uncertain sightings (q), measured and stochastically modelled simultaneously. Note that only q ∈ [0, 1) are used as quality measures, which only relate to uncertain sightings. The sighting quality q is thus in a sense a nuisance variable which we take as Beta distributed, which ensures that it is bounded between [0, 1] as required. We use non-informative Exponential(1) and Jeffreys (Beta(0.5, 0.5)) priors for its parameters using the Stroud (1994) method in Congdon (2001). Equation (3) in full would rely upon specifying a general stochastic model for the quality measure p(Q = q) under the alternative hypothesis of extinction versus that under non-extinction.

One would expect that after extinction, sighting quality drops, yet ‘Examining Sighting Quality’ proved this may not be the case. So for simplicity (and to avoid specifying even more unknown priors in the computations to integrate over) we assume herein that the probabilistic generating process for quality attribution is unaffected by whether the species is extinct or not, unless the sighting is deemed as ‘certain’. Formally, l(q|E) = l(q|notE) = l(q). This approximation is reasonable since the alternative requires an estimate for the error process arising from experts assigning quality measures for l(q|E) and l(q|notE), which needs repeated blinded data for a variety of species whose extinction date and status were well known but that investigative adjudicators had little experience about before. A significant practical challenge.

For the quality breakpoint method, we assume that the probabilistic generating process of sightings is unrelated to the generating process of quality attribution, that is, p(data) = p(s&q) = p(s)⋅p(q). The two processes are taken to be independent. So integrating out the uncertainty of the nuisance gives (3a)${\displaystyle l\left(\mathbf{data}|E\right)\to {\int }_0^1{l}_s\left(\mathbf{s}|E\right)\cdot l_q\left(\mathbf{q}|E\right)\mathrm{d}p\left(\mathbf{q}\right),}$ (3b)${\displaystyle l\left(\mathbf{data}|notE\right)\to {\int }_0^1{l}_s\left(\mathbf{s}|notE\right)\cdot l_q\left(\mathbf{q}|notE\right)\mathrm{d}p\left(\mathbf{q}\right),}$ which are the weighted sums of likelihoods over the stochasticity of the quality measure at Q = q, where Q is a particular realisation of q. Conjugacy makes this integration efficient. The likelihoods Eq. (3) can be fed into Eq. (2) to yield the extinction posterior for sightings p(E|s). The same four-part structure of the approach (with subscript 1 indicating certain sightings and subscript 2 indicating uncertain sightings) would now follow but with Beta likelihoods for q. However, the quality measures are for uncertain sightings only, thus:

• l(q₂|notE) = Beta(α + α₂, β + β₂)

• l(q₂|E) = Beta(α + α₂, β + β₂) if the time is before or equal to the time of extinction, with l(q₂|E) = Beta(α, β) if afterwards.

The overall form for estimation over s&q is then comparable to that of Lee et al. (2014) and can be fitted using the same OpenBugs (2012) approach via Markov Chain Monte Carlo (MCMC) integration. Any change point in q will reinforce or conflict with any change point in s in the overall optimistaion for p(E|data). Later work may investigate our assumption that sighting rate and quality scores are independent. For example, perhaps an event occurs (change in IUCN classification or a reward offered) that causes an influx of low quality sightings.

For the quality as variance method we assume that p(data) = p(s|q)⋅p(q) and solve accordingly. In this case p(q) is taken to be a Gamma distributed expansion/shrinkage factor to the variance of the rate of the uncertain (only) Poisson distributed sightings s₂ above. Full calculation detail is given in a later section. Again, later work may test whether it is appropriate to view (s&q) as independent (conditional on extinction) and this Bayesian model extended.

One thus chooses between: the changes in sighting quality inform the inference of whether extinction has occurred by directly affecting the likelihood of the sightings (the quality breakpoint method). Or, that quality is a proxy for the variance around the degree of certainty of the uncertain sightings and so affects the likelihood of the sightings indirectly (the quality as variance method). In either case, integrating over the nuisance modifies the relative sightings likelihoods which arise from the simple approach of assuming all sightings are of similar quality used in Lee et al. (2014).

The Choice of Prior

As previously shown, the method of Lee et al. (2014) is affected by the experts’ prior. This phenomenon is presented for the bird data sets in Lee et al. (2014), and with the Barbary lion data set in Fig. 3. Whilst the three different choices of prior are all initially overwhelmed by the posterior in 1953 (that is, the extinction probability is larger than the prior), the estimated extinction year that corresponds with the data set as it stood in 1953 is different. When the model is run with the data that exists up to 1953 only, with a prior belief of extinction of 0.9, the model infers extinction occurred in 1936; with a prior of 0.5, the model infers extinction occurred in 1931; and with a prior of 0.1, the model infers extinction occurred in 1924, which is before the ‘certain’ sighting of 1925 occurs, so is clearly incorrect.

Using the full data set as it stands in 2016 only includes one additional sighting after 1953, a sighting in 1956. The estimated extinction year does not vary between the three prior choices: a prior belief of extinction of 0.9, 0.5 and 0.1 all predict extinction in 1954. Therefore the model assumes that the 1956 sighting is false. Additionally, the results are less influenced by the prior when the model is run with more data, as one would expect.

To avoid the bias towards the expert’s prior choice, for all four species, we use a non-informative hyper-prior—the beta distribution p(E) ∼ Beta(0.5, 0.5) (Congdon, 2001) and optimise accordingly. This distribution actually slightly favours the extremes equally, when the species is definitely extinct p(E) = 0 or definitely extant p(E) = 1, and all other possibilities are equally likely to each other. The expected value of this density is 0.5 i.e., a 50:50 evens bet of prior ignorance. The model from Lee et al. (2014) applied to the lion data set, with this improvement, provides similar results to when a prior of 0.5 is used, as one would expect, see Fig. 4A. The first year that the likelihood overwhelms the average of the prior, 0.5, occurs in 1953 again, which gives a corresponding year of extinction of 1931. When the model is run on the sighting record as it stands in 2016, the corresponding extinction year is 1954. Although these results are similar to a prior of 0.5, a non-informative prior is preferred to a point estimate as it integrates over all uncertainties appropriately.

Results

Lee et al. (2014) considered the rate of certain and uncertain sightings, and sought to find a change point in the sighting rate, thus the model requires as least two certain sightings. The individual quality of each sighting was not considered. As discussed in the previous section, using this approach the probability that the Barbary lion is extinct is not above the average of the experts’ prior until after 1953 (Fig. 4A), which has a corresponding extinction date estimate of 1931, approximately the same time period that there is a change in the quality of the data (Fig. 2).

When seeking a change in the quality (‘quality as a breakpoint method’), there is low overall information, that is, it is only after an absence of sightings for 13 years after the very last sighting when the likelihood first overwhelms the prior (Fig. 4B). This is inconsistent with the sightings information. The model cannot identify a common change zone for sighting rate whilst simultaneously identifying the change in sighting quality.

Continuing with the lion, we used the sighting quality as a variance method with both expert estimates (Fig. 4C) and random estimates (Fig. 4D) for the sighting qualities. Both choices provide results very similar to the model of Lee et al. (2014), which does not include sighting quality (Fig. 4A). The posterior from all three models first overwhelm the average of the prior in 1953, and at this point, estimate extinction to have occurred around 1931. As before, the effect of the quality is apparent when the full data set up to 2016, is used. The additional sighting in 1956 affects the sighting rate, causing the change point in sighting rate to shift to 1954.

For the bird species, there were not enough uncertain sightings to seek a change point in sighting data, see Table 1. Therefore, we only use the method where quality is implemented as a variance. As with the lion, we find similar results between omitting sighting quality, using the sighting quality estimates, and using random sighting quality estimates, see Fig. 5. Moreover, the Pohnpei starling demonstrates the challenge of trying to infer an extinction date when there is a paucity of data, since the likelihood never overwhelms the prior. This is a similar problem to the ‘quality breakpoint method’ (Fig. 4B) where a long period of no sightings was required.

Reducing the weighting penalty, a, further reduces the effect of the sighting quality, whereas increasing a increases the standard deviation around each annual estimate. When a is increased to, say, a = 10, the change point in the sighting record dissipates into fuzz, and thus cannot be determined. That is, everything becomes uncertain.

For all species, there is a consistent similarity of extinction inferences over all species between using actual quality measures and simulated random quality scores. Since it was shown in ‘Examining Sighting Quality’ that the quality distributions are non-random through time, this points to the lack of sensitivity to expert opinion for the inference of extinction.

Discussion

We have presented a method that considers the quality of each sighting individually. Previous work has initially used sightings that are classed as either certain or uncertain (Solow et al., 2012; Lee, 2014). Further work then sought to divide uncertain sightings further, into several categories (Lee et al., 2014). However, each additional classification provides less information about the rate of occurrence for this particular class of sighting (law of diminishing returns). To avoid continually dividing uncertain sightings into more specific categories we investigated a method that sought to find the change point in the continuous quality of sightings. This required several experts to rate all sightings of a species. As discussed in previous work (Lee et al., 2015), the method by which the experts are questioned is important to the outcome.

There is a change in sighting quality scores at 1930, where sightings after this date have a perceived higher quality. However the change in sighting quality may be picking up on our preference to believe accounts from living observers more than records left by deceased observers. There is a large literature on unbiased Bayesian elicitation methods to help avoid this (see review article Kuhnert, Martin & Griffiths, 2010). Being blinded to date and the historical age of sightings is important—however, this is difficult. Technological changes over time are apparent even when species observations are not explicitly labelled (an old photo is clearly an old photo). Whilst human nuances affect all sighting records, and thus all extinction models, methods which rely more heavily on expert opinion may be more susceptible to these external factors.

Establishing the balance between an extinction model with assumptions that over simplify, and a model that seeks to incorporate everything, is discussed by Caley & Barry (2014), where the authors develop an extinction model that does not constrict the species population to be constant (as assumed here), nor declining. In line with their findings, our work also suggests that a simple model makes it easier to identify the underlying population processes. If quality is to be used, the quality as variance method is recommended.

We have shown that the rate of sightings is the strongest indicator to infer extinction, and too much information about the quality of the sighting can actually be detrimental. Ideally a sighting record would be a list of certain and uncertain sightings only. Using only these two parallel sighting records, a Bayesian model (Lee et al., 2014 or Thompson et al., 2013), with the non-informative prior presented here, could establish a synchronised change point to infer an extinction date. No further classification, nor prior belief about extinction, is required.

The less propensity for human influence in the sighting data the better. As such, an objective method of assessing whether sightings are certain or uncertain is needed. However, even the rate of sightings is susceptible to human influence, such as periods of high interest due to publicity. Future work could try to quantify the effect of publicity on sighting rate.

Supplemental Information