Assessing the efficacy of citizen scientists monitoring native bees in urban areas

With growing human population, urban areas expand, and natural spaces become more fragmented threatening the important ecosystem services provided by bee pollinators. Urbanization also provides a wealth of opportunities in the way of citizens who are willing to engage and develop strategies that improve the quality of urban areas. Knowing that pollinator conservation is a critical issue, we launched Native Bee Watch, a citizen science project to monitor bees in Fort Collins, a fast-growing urban center in northern Colorado, USA. Relying on citizen scientists presents challenges in data accuracy leading to the current study with objectives aimed to develop a protocol for accurate bee identification and determine whether citizen scientists following the protocol collect accurate data on bee diversity. The different genera of bees were grouped into eight morphospecies categories. Citizen scientists received intensive training prior to the start of the biweekly monitoring and researchers monitored during the off-weeks. We had very high volunteer retention rates and our results indicate strong correlation between citizen scientist and researcher data suggesting with intensive training and engagement, accurate data collection by citizen scientists and volunteer retention is possible. We suggest that citizen science can be a plausible option for bee monitoring at the level of morphospecies, but success will depend on the extent of volunteer engagement and training. Detailed taxonomic analyses may be necessary to formulate long term conservation planning for a location. © 2019 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).


Introduction
As the world's human population continues to grow, urban areas expand, and natural spaces become more fragmented leading to dilution and loss of natural resources. In North America alone, over 82% of the population resides in urban areas, a percentage that is expected to grow (Cohen, 2003;United Nations, 2014). Of the organisms that depend on natural resources, pollinators could face significant impacts due to continued habitat loss, fragmentation, and deterioration Hennig and Ghazoul, 2012). Conserving and managing scarce natural resources in and around urban areas are becoming increasingly important with this growing human population. Pollinators are vital to human survival as they facilitate the production of many nutritionally important fruits and vegetables. Even though some plants can self-pollinate, most plants benefit from cross-pollination. Given that roughly 75% of the more than 240,000 species of the world's flowering plants rely on animal pollinators for reproduction, expanding urban areas could reduce the number of pollinators available for successful plant reproduction in ecosystems (Ollerton et al., 2011).
Pollinators and their mutualistic relationship between plants is crucial for the reproductive success of several plant species in the natural environment (Blitzer et al., 2012;Blüthgen and Klein, 2011;Soliveres et al., 2016;Tscharntke et al., 2012). Roughly 35% of global crops including fruits, vegetables, nuts, and other plants that provide food, fiber, drugs, and fuel for humans are dependent on insect pollinators for reproduction (Aizen et al., 2009;Garibaldi et al. 2013Garibaldi et al. , 2014Klein et al., 2007). Economically, insect-pollinated crops have an annual worth of about $14.6 billion of which $2e3 billion is contributed by wild bees (Calderone, 2012;Gallai et al., 2009;Koh et al., 2016;Losey and Vaughan, 2006). In addition, pollinators provide indirect benefits in the form of livestock forage, such as alfalfa and clover. Pollinators also contribute to aesthetics, recreational values, and cultural activities, and they help maintain ecosystem integrity (Niemel€ a, 1999). Subsequently, with continued urbanization, one may expect a reduction in ecosystem services and functions (Banaszak-Cibicka et al., 2018;Cardoso and Gonçalves, 2018;National Research Council Reports, 2007). The widespread services provided by pollinators and the rapid rate at which urban areas are expanding makes understanding the impact of urbanization on insects, such as bees, necessary.
While urbanization is expanding, the human connections to nature and our desire to interact with nature remain steady. One method of understanding the effects of urbanization on ecosystems is to involve residents in exploring the components of the ecosystem around them by engaging, educating, and empowering urbanites through citizen science (Krasny et al., 2014). Citizen science is defined as involving non-scientists in collecting data for a research project and many times contribute to a large database (Cooper et al., 2014;Jue and Daniels, 2015;Theobald et al., 2015;Trumbull et al., 2000). Citizen science has grown dramatically in popularity in the last two decades as a result of more projects being readily available for participants; technological advances such as data entry on portable devices allowing easier data collection and entry by nonexperts; greater recognition that scientists can capitalize on citizen scientist availability and enthusiasm to increase capacity; increased research funding shortfalls; and the need to meet an outreach-related component Silvertown, 2009). The social component of citizen science offers countless tangible and intangible benefits including facilitating behavior change towards environmental issues, developing long-term community-based conservation programs to benefit ecosystem services and habitats, increasing opportunities for adults and children to interact with nature, increasing scientific literacy, bridging communication gaps between scientists and researchers, and providing additional avenues to inform policy makers on conservation and environmental issues (Bell et al., 2008;Danielsen et al., 2005;Dickinson and Bonney, 2012;Kleinke et al., 2018;Schmeller et al., 2009;Schultz, 2011;Theobald et al., 2015;Trumbull et al., 2000).
Data accuracy is one of the main concerns that scientists have with citizen science programs (Danielsen et al., 2005;Law et al., 2017), and several studies have assessed the quality of citizen science data in an attempt to comprehend the underlying problems and develop effective future protocols (Birkin and Goulson, 2015;Burgess et al., 2017;Callaghan et al., 2017;Kremen et al., 2011;McDonough MacKenzie et al., 2017;Rüdisser et al., 2017). In general, the findings indicate a disparity between citizens' self-assessed and actual identification skills, suggesting the importance of evaluating this difference before launching data collection. However, the general consensus is that researchers do not yet fully understand the error potential in citizen science data possibly because data accuracy does not have a reliable definition and there are few consistent metrics on data accuracy . A study assessing citizen science journal articles revealed that only approximately half of the published articles reported having accurate data from citizen scientists but the studies examined did not necessarily define data accuracy (Aceves-Bueno et al., 2017). Burgess et al. (2017), while identifying broad barriers for publishing citizen science data, note that not all projects are suited for citizen scientists and such thoughts on suitability for citizens has also been reiterated by Kleinke et al. (2018) in their study on a pollination services project. Some of the suggested barriers include insufficient awareness among scientists about projects suitable for citizen science, data quality inconsistency, bias among scientists for certain data sources (such as age and educational levels of citizen scientists) and the extent of time commitment required by the project.
Given the importance of pollinators, there are a few well-known citizen science projects, including the Great Sunflower Project (Domroese and Johnson, 2017), Pollination Investigators (Kleinke et al., 2018) and the Urban Pollination Project (Potter and LeBuhn, 2015). Others, including Kremen et al. (2011), reported that citizen scientists can record broad-level data but perhaps not finer details such as genus or species-level data, and Birkin and Goulson (2015) suggest the importance of engaging citizens during the study to maintain interest and commitment. With this background, recognizing the hurdles encountered with citizen science data and knowing that pollinator conservation is an attractive issue for urban citizens, we launched this study known as the Native Bee Watch, a citizen science project on urban bee pollinators specifically to determine whether citizen scientists can collect accurate data on bee morphospecies and abundance and to develop a protocol that yields us accurate data. In addition, we were also interested in determining whether volunteer engagement activities improved retention of citizen scientists throughout the study period.

Study area
The study took place in the city of Fort Collins, a fast-growing urban center along the northeastern Front Range of Colorado. Three public gardens located throughout the city were monitored. The gardens were chosen because they can all be easily accessed by the citizen scientists. The Gardens at Spring Creek is a city-owned, 7.3ha botanical garden located along the Spring Creek corridor. Nix Farm Natural Area is a city-owned, 11.1ha historic site off the Poudre River trail system, surrounded mostly by undeveloped land. The Plant Select ® Demonstration Garden is a part of the Colorado State University (CSU) Annual Flower Trial Garden, a 1.2ha research and public garden near the CSU campus. At each garden, variable length transects were identified along a public walkway that had flowers blooming throughout the season. Transects varied in length due to the size of the gardens and available public walkways.

Citizen scientists and researchers
"Citizen Scientists" were community volunteers interested in monitoring bees. "Researchers" were CSU pollination biology laboratory personnel with bee identification experience and expertise.

Monitoring frequency and duration
Gardens were monitored on a weekly basis from the last week in May until the last week of September in 2016 and from the last week in May through mid-September in 2017. A "Citizen Science Session" consisted of 1e4 volunteers, working in pairs when possible, to monitor the gardens with a researcher on-site. A "Researcher-Only Session" was defined as researchers conducting the monitoring session. Citizen science monitoring occurred on alternating weeks, and researcher-only monitoring occurred during the off-weeks, allowing for paired volunteer and researcher data sets.

Recruiting
Volunteers were recruited through emails to various networks within CSU and the City of Fort Collins, flyers were posted in public places such as libraries and word-of-mouth. Community outreach events on pollinator conservation held each year also helped to recruit volunteers. The target goal was to recruit 30 adult citizen scientist volunteers to participate in bee diversity monitoring. The recruiting goal was set higher than the capacity for the training program because we anticipated volunteer drop out.

Training
Training entailed a 2-h interactive workshop introducing pollinators, learning identification characteristics of bees, flies and wasps, and creating pollinator habitats in backyards. Volunteers received a field guide (Mason et al., 2018), which in addition to details on different bee genera, was geared towards the research project summarizing verbal (Fig. 1) and pictorial ( Fig. 2) keys to identify bee morphological groups and identification characteristics to differentiate bees, wasps, and flies. Each volunteer also examined voucher specimens and took photos to compare with the pictures in their copy of the field guide. A researcher worked closely with the volunteers taking care to point out distinguishing features of individual bee voucher specimens to help them learn the characteristics of each category. In addition to training with voucher specimens, volunteers trained on-site with a researcher during their first monitoring sessions.

Engagement
Biweekly newsletters were emailed to volunteers reminding them the dates they were monitoring during the upcoming week. Newsletters also contained tips to identify bees, interesting bee or insect sightings while monitoring, current research, and other relevant information to keep volunteers engaged in the project. A volunteer appreciation event at the end of the season provided a summary of the summer activities.

Data collection
Each monitoring session lasted up to 2 h from 9AM to 11AM, the most active time for bees (Kearns and Inouye, 1993). Monitoring occurred only on sunny, non-windy days (Kremen et al., 2011). Researchers and citizen scientists selected their monitoring days based on their availability. Up to four volunteers participated in each citizen science monitoring session with an on-site researcher. Volunteers monitored in pairs sharing the responsibilities for observing the bees, starting and stopping a timer, recording the data, and if needed, pausing the timer while referencing the field guide. Volunteers submitted the data sheets to the accompanying researcher. During the researcher-only monitoring sessions, a researcher conducted independent observations repeating the methods of the citizen science monitoring sessions. With the aid of the keys that were used during training sessions (Fig. 1), the photographs (Fig. 2) and the field guide (Mason et al., 2018) provided to each citizen, bees were categorized into morphospecies.
Prior to citizen scientists beginning their monitoring sessions, researchers visited the different gardens together and visually marked out a path that all the citizens would follow to ensure similarity in monitoring. CSU Annual Flower Trial Garden was smallest of the 3 gardens and during the monitoring period, citizens were sometimes able to complete two or more transects. This study does not attempt to compare across gardens and so we are not correcting for the length of the transects. Observational data on bee visitation was collected by using the Focal Plant Sampling Procedure that was modified from other animal behavior studies (Altmann, 1974). The data collectors walked through the gardens on the pre-determined path and stopped at each flowering plant along the way to observe bees on the flowers. All bees pollinating a flower on the plant, as described in the field guide, were recorded for a 2-min period noting the number of bees in each morphological category. Flower visitors, including bees not visibly pollinating the plant, were not recorded.

Statistical analysis
Spearman's Rank Correlation was examined by using the cor.test() function available in base R (version 3.2.5 (RCoreTeam, 2017) to compare the citizen scientist data set with the researcher-only data set for 2016 and 2017, and the researchers-only data with the citizen science leader data for 2017 only. Actual numbers and proportions of bees in the eight morphological categories were used for the analyses. The comparisons were repeated by excluding honey bees from the data as it is essential to understand the morphospecies abundance patterns of non-Apis bees known to be prevalent in urban areas (Tommasi et al., 2004).

Results
Our study was aimed at determining whether citizen scientist volunteers can collect accurate data on the morphospecies diversity and abundance of native bees in urban areas and to develop a protocol for recruiting, training and engaging citizens involved in the project to promote volunteer retention.

Volunteer recruitment and retention
In 2016, 28 volunteers attended the citizen science training sessions, and 22 participated in monitoring and in 2017, 29 volunteers attended the training, and 25 volunteers participated in monitoring. The volunteer retention rate, calculated as the proportion of the volunteers that attended the initial training sessions and then followed through with at least one bee monitoring session during the summer, was 78.5% in 2016 and up to 86% in 2017 (Fig. 3). Returning rate of volunteers, including those that volunteered in 2016 and returned in 2017 was 28%. We continue to have trained volunteers return in the subsequent years, but this information is extraneous to the study objective being presented here.

Citizen science and researcher sessions
The sampling sessions and participation by citizen scientists were similar during 2016 and 2017, resulting in uniform sampling efforts between the three gardens and across the two years when the study took place. There were 50 sampling sessions in total during 2016, of which 27 (Nix Farm: 8; Gardens on Spring Creek: 10; CSU Trial Garden: 9) were by citizen scientists and 23 (Nix Farm: 7; Gardens on Spring Creek: 8; CSU Trial Garden

Categorization of bees observed by citizens and researchers
We observed 3,722 bees in 2016 and 6,499 bees in 2017. The descriptions, common names and scientific classifications of these morphospecies are provided in Table 1. Analyses of the number of bees grouped into the 8 different morphospecies categories indicated a significant correlation between citizen scientist groups and researcher groups ( Fig. 4; Spearman rank correlation, 2016: r ¼ 0.92; p ¼ 0.0013; 2017: r ¼ 0.98; p ¼ 0.00005). During our study, honey bees (Apis mellifera) formed a large proportion (~60%; Fig. 4) of the bees observed. In order to determine whether this similarity between citizen scientist and researcher observations holds when honey bees are excluded from the analyses, we repeated the correlation analyses with only the non-Apis bees. These non-Apis bees, referred to as wild bees are important for pollination of flora in natural habitats (Kleijn et al., 2015;Potts et al., 2016;Tuell et al., 2008) and may be prevalent in urban areas due to small and scattered habitats (Threlfall et al., 2015;Tommasi et al., 2004). While they are suggested to be complementing the pollination activities of honey bees, the pollination services provided by the wild bees are more efficient in some crops including squashes (Artz et al., 2011), sunflowers (Parker et al., 1981) and alfalfa (Bosch and Kemp, 2005) and they are crucial for reproductive success of plants in natural habitats (Brittain et al., 2013;Kremen et al., 2007). There was a highly significant correlation between citizen scientist observations with the researcher observations even when only the non-Apis bees were considered for the analyses ( Fig. 5; Spearman rank correlation, 2016: r ¼ 0.88; p ¼ 0.007; 2017: r ¼ 0.98; p ¼ 0.00005).
We conducted a correlation analyses between morphospecies categories identified by researchers during their observations in the garden and ascribing morphospecies names to taxonomic identifications of bees collected in the same gardens during the same period as a part of another study. The correlation values were significant indicating that the morphospecies categories are a good representation of the taxonomic groups (Table S1).

Engagement
To determine the level of engagement we tracked the number of biweekly newsletters emailed and the number of unique opens using the analytics supplied by the newsletter platform (Constant Contact). All citizen scientists received newsletters and interested members of the community received newsletters if they chose to sign up. The trend seen in Fig. 6 shows not just an increase in the number of citizens signing up for the newsletters but also a continued increase in the number of unique openings during the study period.

Discussion
Overall, our results indicate that with prior training and continued engagement, citizen scientists collect accurate data, comparable to what a researcher would collect. Results also suggest that citizen science data can be used to monitor long term bee morphospecies groups and abundance trends, a potentially valuable conservation tool, when the morphospecies in a given location is correlated with actual species diversity after taxonomic identification of bees. While using morphospecies does not replace taxonomic biodiversity data, there are benefits to collecting data at a morphospecies level. Collecting accurate invertebrate taxonomic diversity data can be challenging because of the financial and human resources required (Derraik et al., 2010;Oliver and Beattie, 1996). Bees are especially challenging as there is a significant lack of keys necessary for identification, especially in the western United States (Koh et al., 2016;Scott et al., 2011), where this study was conducted, and the process of identification is time consuming. For a broad understanding of the diversity of pollinating insects and to develop guidelines for habitat protection in urban areas, morphospecies data can minimize or eliminate the need to collect specimens and when appropriately performed can provide quick and reliable results. This allows data collection over greater spatial and temporal scales. We advocate for using morphospecies as a monitoring tool especially by those with limited taxonomic training, but we caution against relying too heavily on just the morphospecies information for conservation purposes, as bees with similar morphological traits may have different ecological traits, and morphospecies may not accurately reflect these differences (Michener 1974(Michener , 2007. Research has shown that data collection accuracy can be improved with rigorous training sessions initially and having researchers spend additional time with the citizen scientists outside of the initial training sessions during monitoring (Crall et al., 2011;Kremen et al., 2011). Having researchers on site when volunteers monitored bees in our study gardens, provides this continued training, ready access to experts and learning opportunities as volunteers work with professionals in the field. In addition to confirming previous findings that citizen scientists can collect broad-level data on floral visitors (Kremen et al., 2011), our results confirm that citizen scientists can collect accurate data at a finer resolution within bees (Apoidea). It must be noted here that bees have a wide variety of sociality and nesting characteristics that transcend taxonomic boundaries such as family or genera (Michener, 1974). The habitat requirements for conservation of these diverse bee species show substantial variation that may be difficult to capture accurately with this citizen science protocol. It will be immensely important to map the habitat in addition to monitoring bees, but this adds another task to the volunteer group will need to achieve and hence was not included in the current study. Researchers do not always fully understand the potential for errors in citizen science data suggesting the importance of emphasizing consistent metrics when assessing data accuracy (Aceves-Bueno et al., 2017;Dickinson et al., 2010). These metrics can help identify problem areas for improving data accuracy and allow for developing better training protocols. For example, the morphospecies group called "Tiny Dark Bees" in our study had less observations by Note: These are some of the more common bees observed and in no way represent an inclusive list of all bees found in these categories or in Colorado.
citizen scientists in 2016 than the researchers. During training in 2017, we emphasized the "Tiny Dark Bee" category through photos and voucher specimens, and this could have possibly contributed to the more comparable numbers and proportions in 2017 (Figs. 4 and 5).
Our study conducted over a two-year period also provides evidence for high retention rates of the citizen scientists through the season and beyond. Our results support the premise that when volunteers stay engaged, their data accuracy tends to improve suggesting a combination of training and engagement as ways to obtain reliable data from citizen scientists. It has also been suggested that volunteer motivations should be taken into consideration when developing a citizen science project (Bruyere and Rappe, 2007;Guiney and Oberhauser, 2009). Pollinators are a popular topic among citizens which could have played a significant role in the high and continued motivation we were able to informally observe during the study, but we would like to clarify that our study was not designed to evaluate the extent and causes of motivation.
Maintaining volunteer engagement and interest throughout the season is critical to ensure that they stay engaged and motivated to produce high quality work (Dickinson and Bonney, 2012;Crall et al., 2011;Birkin and Goulson, 2015). Although our study was not designed to determine the efficacy of citizen engagement methods in improving volunteer retention, we speculate that the variety of communication methods we used and the one-on-one time our researchers spent with the volunteers could have played a significant role in the volunteer retention rates we experienced. The biweekly e-newsletters in the summer season and monthly newsletters in the off-season, maintaining a project website, and holding a volunteer appreciation event at the end of the season could have also contributed towards volunteer retention. The number of newsletters sent out progressively increased (Fig. 6) and this expansion in readership beyond just citizen scientists occurred as the attendees of our pollinator-related outreach events subscribed to stay in communication with the project. We maintained communication through the newsletters even when the volunteers were not monitoring during September through April. We predict that these engagement activities played a substantial role in reenrollment of 2016 volunteers for the 2017 season. While there are over 20,000 species of bees described globally and over 4,000 of them are found in North America (Michener, 2007), and at least 946 valid bee species in Colorado (Scott et al., 2011), the simplified arrangement of classifying bees into morphospecies (Figs. 1 and 2), allows for non-taxonomists to record bee abundance and morphospecies diversity. Future research determining the relation between the diversity measures calculated from morphospecies groups and taxonomic species groups is a crucial metric to strengthen the validity of the data contributed by citizen scientists towards bee conservation.
Volunteer motivations should be taken into consideration when developing a citizen science project (Bruyere and Rappe, 2007;Guiney and Oberhauser, 2009). Future research recommendations to expand on this study could survey volunteers to understand their motivations, values, norms, and attitudes towards pollinator conservation and citizen science (Burgess et al.,Fig. 5. Frequency distribution of bees excluding honey bees (Apis mellifera) reported during citizen scientists (Open bars) and researchers (Closed bars) sessions.