Quantifying the contribution of zoos and aquariums to peer-reviewed scientific research

,


Introduction
We are facing a global biodiversity crisis in the Anthropocene, with escalating extinction rates and biodiversity losses (Dirzo et al. 2014;Pimm et al. 2014).Addressing and trying to reverse this crisis is a monumental task, currently undertaken by a diverse portfolio of organizations that include inter-governmental panels, federal agencies, research institutions, and nonprofits (e.g., Armsworth et al. 2012).Among these organizations are zoos and aquariums, which have transitioned over time from primarily public or private menageries to organizations actively involved in biodiversity conservation (Hutchins and Smith 2003).In fact, conservation and research are listed as key components of the accreditation standards of the Association of Zoos and Aquariums (AZA, aza.org).Founded in 1924, AZA is a nonprofit organization dedicated to the advancement of zoos and aquariums through institution accreditation, animal care initiatives, education, and conservation programs.
Modern zoos and aquariums have immense capabilities for education and outreach because millions of people (>700 million) visit annually (Gusset and Dick 2011).Other conservation-related activities of zoos and aquariums include creating and participating in Species Survival Plans ® (Hutchins and   Conway 1995; Gippoliti 2012), conducting biodiversity conservation-relevant research (Stoinski et al. 1998;Anderson et al. 2008;Fernandez and Timberlake 2008;Maple and Bashaw 2010), and directing funds to in situ conservation, with $154 million USD contributed by AZA members in 2014 to projects benefitting animals in the wild (Larson 2017).However, evaluating the effectiveness of biodiversity conservation action in a quantifiable and systematic manner remains extremely difficult (Ferraro and Pattanayak 2006;Mace and Baillie 2007;Caro et al. 2009).Although funding is a major driver of biodiversity conservation action (McCarthy et al. 2012;Waldron et al. 2017), in some cases this investment apparently fails to confer significant conservation benefits (Bernhardt et al. 2005;Roni et al. 2008).Accordingly, it seems appropriate to also evaluate other metrics measuring the contributions of zoos and aquariums to biodiversity conservation, such as research productivity in the form of publication of scientific knowledge.
Publication in peer-reviewed scientific literature can be quantified and compared over time and among organizations, serving as a metric of participation in, and value added to, broader communities in biodiversity conservation and ecology (Grant et al. 2007;Livingston et al. 2016;Keville et al. 2017).Nevertheless, we acknowledge that research productivity does not necessarily translate into conservation action (Knight et al. 2008).For example, not all scientific publications by zoos and aquariums may be conservation relevant (although we expect that the majority intend to be), and those aspiring to be conservation relevant may not result in implementation or uptake of recommended actions.However, we anticipate that publication in peer-reviewed scientific literature may serve as a proxy for engagement of organizations within the field and correlate with level of overall activity in biodiversity conservation, as has been proposed for other types of organizations like universities (Lawler et al. 2006;Grant et al. 2007).More importantly, research productivity by zoos and aquariums contributes to the scientific research on which evidence-informed conservation action and management is based, and without which the effectiveness of conservation actions is likely to be lower (Sutherland et al. 2004;Arlettaz et al. 2010;Wilson et al. 2016).Yet, to date, no reviews have quantitatively summarized the overall magnitude and character of zoo and aquarium research productivity in terms of publishing in the peer-reviewed literature, although some past work has focused more narrowly on publishing patterns in individual journals (Anderson et al. 2008).
As such, we report here the first study characterizing the contribution of zoos and aquariums to scientific research that results in the production of peer-reviewed scientific literature.We specifically sought to (i) evaluate the overall magnitude and trend of research productivity by AZA members in aggregate, (ii) characterize the journal outlets and subject areas where AZA members publish most frequently, and (iii) determine the factors (e.g., age, organization type, asset size, and mission statement) associated with inter-organizational variation in research productivity.Cumulatively, this work represents the first comprehensive census of peer-reviewed publishing practices of AZA member zoos and aquariums and identifies needs, opportunities, and mechanisms for these institutions to further engage the broader biodiversity conservation community through their research productivity.

Magnitude and trend of publishing by AZA organizations
We quantified research productivity by AZA member institutions as publications and their citations indexed by the Thomson Reuters ISI Web of Science (WoS) database from 1993 to 2013, reflecting a recent 21-year period since the publication of the first World Zoo Conservation Strategy  (IUDZG and CBSG 1993).Publications included peer-reviewed journal articles, book chapters, and conference proceedings; citations for each publication were then extracted from the WoS, which was also used in previous studies of publishing productivity for other organization types (Grant et al. 2007;Livingston et al. 2016;Wilson et al. 2016;Keville et al. 2017).Our reliance on the WoS inevitably excludes some publishing outlets or publication types produced by AZA members that are not indexed in the WoS, but nonetheless it provides a standardized source for our comparison among zoos and aquariums, assuming that the proportion of research productivity that is not published in WoS sources is roughly constant among institutions.
As of 2014, there were 228 AZA-accredited zoos and aquariums (Table S1).Some AZA members are multiple branches of a single parent organization, e.g., Bronx Zoo, Central Park Zoo, New York Aquarium, Prospect Park Zoo, and Queens Zoo are all affiliates of the Wildlife Conservation Society (WCS).Data extractions were done separately for each AZA member, and not the parent organization as a whole, but publication counts were subsequently summed in regression analysis for comparison with parent organization data (see below).Related institutions, such as the Monterey Bay Aquarium Research Institute, which registers as a separate nonprofit with the US Internal Revenue Service (IRS) from the AZA-accredited Monterey Bay Aquarium, were also not included in the extraction process.
For each AZA member, we searched the institution name, including abbreviations, with asterisks (e.g., "aqua*" instead of "aquarium") and alternate spelling and names (e.g., "Sea Life" and "Sealife"), in the "Address" field using "Basic Search" in all databases within the WoS Core Collection from 1993 to 2013.We saved all records for each publication in the search results and collated outputs for all institutions.Only publications with primary affiliations at AZA institutions were retained; we excluded publications where the corresponding author had moved to an AZA organization ("Present Address") but had conducted the research elsewhere.Redundant publications were removed for analyses addressing research productivity of all AZA institutions aggregated, otherwise publication output was considered per individual organization (below).In addition, we collated WoS journal subject categories for all publications in our data set to determine the primary topic areas where AZA members publish.Publications were thus classified according to the subject category of the journals they were published in.Subject categories may overlap, as each publication can be assigned to more than one category.It is, of course, possible that individual publications include or are relevant to topics other than the subject area of their journal, but this standardized subject area classification provides a coarse perspective of publishing topics by AZA members and is consistent with similar approaches for other organization types like universities (e.g., Keville et al. 2017).

Explaining research productivity at AZA organizations
We sought to identify organizational characteristics associated with research productivity for AZA members using linear regression on a number of zoo or aquarium traits or predictors.For each AZA member we identified the year the organization was founded per their website or other online sources and calculated their age in 2013.We anticipated that older organizations might have had more time to develop capacity and support for research, and that the relatively few (∼10%) organizations founded after 1993 would not have the full study period to accumulate our measures of research productivity.We also identified whether or not organizations were for-profit, nonprofit, or government run.Many AZA members are partnerships between government-run zoos or aquariums and nonprofit "friends" or support groups, which were coded separately as joint government and nonprofit organizations.The majority (214 of 228) of AZA organizations are located in the US, whereas 14 are located in seven other countries.As such, our analysis has a primarily US focus, although preliminary analyses including US or other countries as regression covariates found that research productivity was not associated with nation-level location of AZA members.For US organizations that were exclusively or primarily nonprofits, we estimated their size based on 2012 or 2013 net assets (USD) from IRS 990 Forms extracted from guidestar.org.Alternative measures of size that would relate to research capacity were highly correlated with net assets, e.g., number of employees (Pearson correlation coefficient = 0.76) and expenditures on salaries (Pearson correlation coefficient = 0.79).Where a parent organization (e.g., WCS) was registered singly with the IRS but individual zoo or aquarium branches (e.g., Bronx Zoo) were registered separately with AZA, publications of branch organizations were summed for comparison to parent tax return data.Organization size estimates were not feasible for for-profit or government zoos and aquariums owing to unavailable or inconsistent reporting of financial data.Organization size was only considered for the last 2 years of our analysis due to data availability, although we recognize that size of organizations inevitably varied through time.Nevertheless, we anticipate that rank orders of organization size were likely conserved over the time period of our analysis, with small regional zoos remaining small and large national or international organizations remaining large (Larson et al. 2014).
We also sought to determine whether inclusion of "research" in organizational mission statements was associated with research productivity by AZA organizations.Institutions that have already identified research as part of their mission may be more likely to pursue research than those that have not.As such, we extracted current mission statements from AZA member websites or their IRS 990 Forms and identified those that included research or research-affiliated terms (e.g., "generates and shares scientific knowledge," "scientific discovery," "the study of zoological natural history," etc.).All variables used in regression analyses are listed in Table 1, and all mission statements are included in Table S1.
Total publication and citation numbers to publications were regressed against the above organizational characteristics using linear models in R (R Core Team 2017).For the first analysis we considered the full data set of AZA members and included as predictors the age of the organization relative to 2013; whether or not the organization was for-profit, nonprofit, government-run, or a combined government and nonprofit hybrid; and whether research was included in the mission statement.All categorical variables were entered as 0/1 dummy variables.Organization type had four categorical levels, and the combined government and nonprofit organizations were coded as the 0 "reference" variable in all cases.Publication and citation response variables were log+1 transformed for the analysis.
A second analysis focused on US-based nonprofit organizations (n = 93), for which we could compare log-transformed net assets as a measure of size or organizational financial resources.Regression analysis followed the same procedure as in the full data set but removed the government, for-profit, or nonprofit distinction because all organizations were nonprofits.Size by net assets ranged by nearly four orders of magnitude, from roughly $120 000 USD to $680 000 000 USD. Distribution of net

Case studies of top-publishing organizations
We also sought to explore the characteristics of the most research-productive AZA members in terms of WoS publication outputs in greater detail.We assembled lists of top organizations by total publications and h-index, a citation index defined by number of publications per organization with at least h citations (Hirsch 2005), and assessed their organizational characteristics.For example, did they include research in their mission statements?Were they for-profits or nonprofits, and if they were nonprofits, were they generally larger organizations?We manually calculated the h-index from the publications and their citations for each top-publishing organization.To qualitatively complement our lists of rankings, we requested brief email responses from leaders of the majority of these most research-productive zoos and aquariums characterizing some of the organizational attributes they felt contributed to their organization's research culture and success (Supplementary Material 3).

Magnitude and trend of publishing by AZA organizations
According to the WoS data as of September 2016, from 1993 to 2013 AZA members produced 5175 publications with 81 342 total citations.The number of publications produced annually increased over time, from 114 publications in 1993 to 437 publications in 2013 (Fig. 1).AZA members published most often in the Journal of Zoo and Wildlife Medicine (500 publications), Zoo Biology (330 publications), the American Journal of Primatology (170 publications), Marine Mammal Science (117 publications), and Biology of Reproduction (98 publications).AZA members produced 30.9% and 23.8% of the total articles in Zoo Biology and Journal of Zoo and Wildlife Medicine, respectively, whereas AZA member representation in the other three journals with a wider audience base was smaller, ranging from 0.5% to 7.5%.Considering the top five subject categories, most AZA publications (31.9%) were classified under the category "zoology" (Fig. 1).The next category, "veterinary science," accounted for 31.6% of all AZA publications, followed by "ecology" (13.3%), "marine and freshwater biology" (10.5%) and "biodiversity conservation" (7.3%).

Explaining research productivity at AZA organizations
For all AZA members, both total publications and citations were significantly associated with whether or not the organization included research in their mission statement (Table 2).Age in 2013 was also weakly associated with these two research productivity responses, with older organizations publishing more (Table 2).Compared with other organization types, strictly nonprofit organizations published significantly more and were cited more overall (Table 2).Fitted models for both total publications and total citations had comparatively low explanatory power (adjusted R 2 = 0.15 and 0.13, respectively, Table 2).
When we repeated the above analyses focused on US-based nonprofits for which we could measure organization size as net assets, we found that larger organizations consistently published more and were cited more overall (Fig. 2, Table 3).This pattern was insensitive to the measure of organization size used, with consistent results found when replacing net assets with either total number of employees or expenditures on employees in regression analyses (Table S2).Constraining these regression analyses to only publications produced between 2010 and 2013 (inclusive) to better match the time period at which organization size was estimated similarly did not appreciably affect our results (Table S2).As documented for the full data set, the inclusion of research in the mission statements was associated with increased total publications (Fig. 2) and total citations, but organization age did not have a significant effect on either publishing response metric (Table 3).Fitted models for both total publications and citations had higher adjusted R 2 values (0.43 and 0.40, respectively) than for the analysis including all AZA members (Table 3).

Case studies of top-publishing organizations
The Smithsonian National Zoological Park ranked first in both total publications and h-index, followed by Mote Marine Laboratory, and the Chicago Zoological Society.In decreasing rank by total  publications, the other seven institutions were as follows: San Diego Zoo, New England Aquarium, Lincoln Park Zoological Gardens, Zoo Atlanta, Saint Louis Zoo, Disney's Animal Kingdom, and Alaska Sea Life Center (Table 4).Monterey Bay Aquarium, Cincinnati Zoo, and Vancouver Aquarium Marine Science Centre had higher h-index rankings than Disney's Animal Kingdom (h-index rank = 13), but fewer publications (Table 4).
Seven of the top performing institutions for total publications were US-based nonprofits.All top nonprofits by total publications ranked above the 60th percentile (>$30 million USD) by net assets.In addition, the majority of top performers for total publications mentioned research in their mission  4).The top performing AZA member, the Smithsonian National Zoological Park, is an affiliate branch of a major research organization, the Smithsonian Institution, whereas Mote Marine Laboratory and Aquarium started as a dedicated research laboratory that subsequently added an AZA-accredited outreach and education facility.Qualitative comments from leaders of many of these zoos and aquariums on their organization's culture as related to scientific research and publishing productivity are available in Supplementary Material 3.

Discussion
We provided a summary of the current state of scientific publishing by zoos and aquariums, while offering guidance and role models for zoos and aquariums aspiring to increase their research productivity.We found that AZA-accredited zoos and aquariums have produced a large volume of research outputs from 1993 to 2013, with a trend of increasing publishing activity over time.The trend appears consistent with increased research investment by these organizations as reported by AZA over recent decades.Per Larson (2017) and AZA annual reports, AZA members have increased spending on conservation and research from $77 million USD in 2003 to $183 million USD in 2013 (aza.org/annual-report-on-conservation-and-science).Our study supports that increased monetary investment by AZA members in conservation and research has contributed to increased research productivity in the form of WoS-indexed publications and citations.
A large proportion of zoo and aquarium publications were placed in specialist journals focused on zoo biology and veterinary science, demonstrating a major research focus of zoos and aquariums (Lawson et al. 2008).AZA members also authored a substantial proportion of the articles in these journals, suggesting that scientific discussion in these journals is largely internal among zoo and aquarium professionals.Instead of focusing on zoo-specialized journals, more active engagement with biodiversity conservation-focused publication outlets might increase the visibility of zoos and aquariums to similar organizations and facilitate knowledge-sharing between diverse organization types engaged in biodiversity conservation.Zoos and aquariums might consider placing more of their scientific products in biodiversity conservation (or other environmental science) journals to potentially reach a broader audience across the field.Still, we acknowledge that these journals might also do more to welcome papers from organizations historically associated with ex situ biodiversity conservation.
For both sets of regression analyses, AZA members that specifically identified research themes in their mission statements had greater research productivity than peers that did not.This is likely due to the establishment of an institutional culture that prioritized scientific outputs and investment in dedicated research staff in organizations that emphasized research, a common theme that surfaced from informal queries of the high-publishing organizations in our study (Supplementary Material 3) as well as from similar past interviews with zoo and aquarium staff (Anderson et al. 2010).When we considered the net assets of US-based nonprofits, larger organizations had greater research productivity than smaller organizations.These results support the hypothesis that larger organizations have more opportunity and flexibility to do research, whereas limited financial resources constrain smaller AZA members from this form of engagement in biodiversity conservation (Larson et al. 2014).However, these results simultaneously show that organizations have the capacity to choose to do more research irrespective of their size, as research productivity was higher for organizations with research in their mission statements across all organization sizes (Fig. 2).Although some of the absolute smallest zoos and aquariums will always be size-constrained to primarily focus on animal care, education, and guest services, many zoos and aquariums in our study have the choice and flexibility to increase their focus on research.Finally, future research might seek to relate more resolved measures of organizational structure or executive leadership to research productivity of zoos and aquariums than we were able to use here (e.g., Anderson et al. 2010).
We note that some larger nonprofit organizations appear to underperform in terms of research productivity relative to their size, owing to the specifics of our data extraction process.For example, the disconnect between registration processes at the IRS and AZA resulted in our relating publications of WCS zoos and aquarium (total publications = 85) to the considerable assets of this largest organization in our data set (∼$680 million USD), whereas WCS as a total organization produced 1989 publications from 1993 to 2013.Similarly, we considered only publications of the AZA member Steinhart Aquarium (n = 4) as related to the size of the nonprofit component of the California Academy of Sciences (∼$550 million USD), which actually had a substantive footprint of 1209 publications between 1993 and 2013.We also acknowledge that some large, but young, organizations were founded after the onset of our study period and consequently did not have the full 20-year time period for proper comparison with older organizations.For example, the Georgia Aquarium (∼$350 million USD) had reasonable research productivity (52 publications) despite having only been founded in 2005.
We recognize that not all contributions by zoos and aquariums to the field of conservation science are captured by the metrics of research productivity we used and that peer-reviewed scientific publications do not guarantee effective conservation outcomes (Knight et al. 2008).Zoos and aquariums are important ambassadors for the natural world to a broad public audience (Miller et al. 2004), many of whom leave zoo and aquarium visits with strengthened feelings of connection to nature and support for biodiversity conservation (Falk et al. 2007; but see Smith et al. 2008).Further, zoos and aquariums make substantial financial and other contributions (like staff expertise) to in situ conservation programs (Gusset and Dick 2011) while also participating in ex situ conservation activities like captive breeding of imperiled species (e.g., Gippoliti 2012).We recognize, in particular, that scientific publishing is an imperfect metric for conservation effectiveness, owing in part to the often large implementation gap between science and action in this field (Knight et al. 2008;Gossa et al. 2015;Rose 2015).Papers do not inherently equal results.
How might the role of zoos and aquariums in biodiversity conservation be further investigated?In the context of our focus on research productivity, we suggest that more intensive analyses working with subsets of the large publication data set we built here might quantify the types of research conducted by zoos and aquariums in more detail than the WoS subject area approach we used.For example, how relevant is the science published by zoos and aquariums to biodiversity conservation?Further, subsequent studies could seek to follow through with interviews or retrospective analyses to determine the uptake of published research by zoos and aquariums into actual biodiversity conservation action.Of conservationrelevant science published by zoos and aquariums, what proportion ultimately sees implementation?And finally, what biodiversity conservation action or research by zoos and aquariums is not reported in the peer-reviewed literature?These are more ambitious undertakings than our study here proposed to address, but it is our hope that this initial analysis provides a foundation for subsequent investigation into the roles of zoos and aquarium in biodiversity conservation science.In summary, our study found support for a large and growing role for zoos and aquariums in research in general, while also identifying opportunities for even more engagement with the broader field of biodiversity conservation.

Fig. 1 .
Fig. 1.Trends in the total number of publications and the five most common Web of Science journal subject areas published in (categories may overlap) by members of the Association of Zoos and Aquariums between 1993 and 2013.

Fig. 2 .FACETS
Fig. 2. Regressions of total publications vs.net assets (both log+1 transformed) for nonprofit Association of Zoos and Aquariums member organizations with (blue) and without (red) research included in their mission statements.

Table 1 .
List of factors used in regression analyses of research productivity for Association of Zoos and Aquariums (AZA) members.

Table 2 .
Regression tables for factors related to research productivity for all Association of Zoos and Aquariums members as of 2013 (n = 228).a a Response variables (total publication and citations) were log+1 transformed.b p < 0.05.

Table 4 .
Top ten most research-productive Association of Zoos and Aquariums (AZA) members by total publications.If two or more AZA members had the same h-index the lower (better) rank was given to the institution with more publications.Monterey Bay Aquarium (77 publications, h-index = 24, rank 10), Cincinnati Zoo (160 publications, h-index = 23, rank 11), and Vancouver Aquarium Marine Science Centre (93 publications, h-index = 21, rank 12) placed above Disney's Animal Kingdom for h-index. a