The Perceived Value of Acquiring Data Seals of Approval

The Data Seal of Approval (DSA) is one of the most widely used standards for Trusted Digital Repositories to date. Those who developed this standard have articulated seven main benefits of acquiring DSAs: 1) stakeholder confidence, 2) improvements in communication, 3) improvement in processes, 4) transparency, 5) differentiation from others, 6) awareness raising about digital preservation, and 7) less laborand timeintensive. Little research has focused on if and how those who have acquired DSAs actually perceive these benefits. Consequently, this study examines the benefits of acquiring DSAs from the point of view of those who have them. In a series of 15 semistructured interviews with representatives from 16 different organizations, participants described the benefits of having DSAs in their own words. Our findings suggest that participants experience all of the seven benefits that those who developed the standard promised. Additionally, our findings reflect the greater importance of some of those benefits compared to others. For example, participants mentioned the benefits of stakeholder confidence, transparency, improvement in processes and awareness raising about digital preservation more frequently than they discussed less laborand timeintensive (e.g. it being less laborand time-intensive to acquire DSAs than becoming certified by other standards), improvements in communication, and differentiation from others. Participants also mentioned two additional benefits of acquiring DSAs that are not explicitly listed on the DSA website that were very important to them: 1) the impact of acquiring the DSA on documentation of their workflows, and 2) assurance that they were following best practice. Implications and future directions for research are discussed. Received 31 January 2017 ~ Accepted 19 September 2017 Correspondence should be addressed to Devan Ray Donaldson, Ph.D., 1320 E. 10 th St., Wells Library 019, Department of Information and Library Science, School of Informatics, Computing and Engineering, Bloomington, IN, USA, 47405-3907. Email: drdonald@indiana.edu The International Journal of Digital Curation is an international journal committed to scholarly excellence and dedicated to the advancement of digital curation across a wide range of sectors. The IJDC is published by the University of Edinburgh on behalf of the Digital Curation Centre. ISSN: 1746-8256. URL: http://www.ijdc.net/ Copyright rests with the authors. This work is released under a Creative Commons Attribution Licence, version 4.0. For details please see https://creativecommons.org/licenses/by/4.0/ International Journal of Digital Curation 2017, Vol. 12, Iss. 1, 130–151 130 http://dx.doi.org/10.2218/ijdc.v12i1.481 DOI: 10.2218/ijdc.v12i1.481 doi:10.2218/ijdc.v12i1.481 Donaldson, Dillo, Downs and Ramdeen | 131


Introduction
We live in a world where exorbitant amounts of data are being produced every day. Digital repositories serve as storehouses for a broad range of data from a variety of academic disciplines. Similar to the data they house, digital repositories also vary, in particular, regarding their propensity to effectively make data accessible and preserve data over the long term. Anyone can say that a digital repository is a Trustworthy Digital Repository (TDR) in accordance with the Open Archival Information Systems Framework (Consultative Committee for Space Data Systems, 2012b). However, claims that a digital repository is trustworthy should be verifiable. For this reason, members of the digital curation community have come together to define a TDR based upon the attributes and capabilities it possesses and the services that it provides. For example, the RLG/OCLC Working Group on Digital Archive Attributes (WGDAA) (2002) defines a TDR as "one whose mission is to provide reliable, long-term access to managed digital resources to its designated community, now and in the future" (p. 5).
Digital repository trustworthiness is one of the most pressing issues raised in digital curation research (Science International, 2015). To address this issue, researchers have developed standards for what constitutes a TDR, including, but not limited to: the Data Seal of Approval (DSA) (Dillo and de Leeuw, 2015), Criteria for Trustworthy Digital Archives (DIN 31644) (Deutsches Institut für Normung, 2012), Audit and Certification of Trustworthy Digital Repositories (ISO 16363) (Consultative Committee for Space Data Systems, 2012a), and CoreTrustSeal. 1 Developers of these standards claim that there are multiple benefits to audit and certification of TDRs. For example, they claim that knowledge about a repository's certification status may lead to sustained or increased funding, or may attract additional funders (e.g., Dillo and de Leeuw, 2015). While these are certainly valuable benefits, little research has examined the perceived benefits of audit and certification by those who are actually certified. It is important to understand whether those who have undergone audit and certification actually think it is worth the effort and have experienced tangible benefits.
The purpose of this paper is to present findings of a systematic, empirical investigation aimed at understanding the perceived value of audit and certification from staff at digital repositories that are certified. As an example of this phenomenon, this study focuses on basic certification by interviewing staff who are responsible for digital repositories that have acquired Data Seals of Approval (DSAs). In particular, this study focuses on the DSA because at the time of this study it had the most certified repositories out of all of the repository trustworthiness standards that exist. The main research question this study addresses is: How do repositories benefit from having DSAs?
The remainder of this paper is organized as follows. First, the background section explores repository certification programs and their accreditation procedures. It also examines the benefits of audit/certification of TDRs as well as existing research on the topic. Second, the methodology section describes the methods that were used to collect and analyze the data. Third, the findings section describes the benefits of acquiring DSAs from the perspective of staff at repositories that have acquired them. Fourth, the discussion section describes the contributions of this research to the digital curation research literature, and provides recommendations to the DSA board for revising current Some of these efforts evolved independently (e.g., DSA certification). Other efforts were influenced by one another (Dale and Gore, 2010). For example, to a certain extent, WDS drew upon DSA criteria, although the DSA primarily centred on repositories for the Humanities and Social Sciences, while WDS was primarily associated with Earth and Space Sciences repositories and data centers. Often these criteria are related. For example, DSA criteria are in accordance with NESTOR, Digital Repository Audit Method Based on Risk Assessment (DRAMBORA), and TRAC criteria (Dillo and de Leeuw, 2015). Most, if not all of these standards and accreditation procedures, are based, at least in part, on the Open Archival Information Systems (OAIS) Framework (Consultative Committee for Space Data Systems, 2012b). These standards and accreditation procedures correspond to one of three different types of certification: basic certification, extended certification, and formal certification (Giaretta, Harmsen, and Keitel, 2010). Repositories that acquire DSAs attain basic certification status. Repositories that acquire DSAs and also perform structured, externally reviewed and publicly available self-audit based on ISO 16363 or DIN 31644 attain extended certification status. Repositories that acquire DSAs and also obtain full external audit and certification based on ISO 16363 or DIN 31644 attain formal certification status. Basic certification is less rigorous and requires fewer resources than extended certification, and extended certification is less rigorous and requires fewer resources than formal certification. In many respects, basic certification is seen as a stepping-stool to reach more comprehensive forms of certification at a later date. On the other hand, the different levels of certification offer digital repository staff members the option to choose which level of certification is appropriate and feasible for their repositories.
More recently, the DSA and WDS have come together to create a set of core requirements for TDRs (Edmunds, L'Hours, Rickards, Trilsbeek, and Vardigan, 2016). This set of core requirements builds on the strengths of DSA and WDS and supersedes each group's lists of basic repository trustworthiness requirements. The rationale for creating the DSA and WDS partnership was four-fold: 1) to increase efficiency in accreditation procedures, 2) to simplify assessment options, 3) to stimulate more certifications, and 4) to increase impact on the community. 3 Launched in September 2017, CoreTrustSeal (CTS) data repository certification represents the culmination of this work. Data repositories can seek certification against the CTS data repository requirements from the WDS or the DSA. CTS will ultimately offer a unique entry point for certification by the end of 2017. 4 Comparing standards for TDRs highlights their similarities and differences. Specifically, standards and accreditation procedures for TDRs vary with respect to the domains they cover, the communities they serve, and their comprehensiveness. For example, the NESTOR Seal of Approval is a standard for TDRs developed and maintained within a German governmental framework. As another example, DRAMBORA focuses on risk assessment. Regardless of the level of certification (e.g., basic, core, extended, or formal), all certification standards and accreditation procedures stress the importance of organizational infrastructure, digital object management, technical infrastructure, and security in order for digital repositories to attain "trustworthy" status.
Comparing estimates of the number of digital repositories that exist worldwide against those that are certified as trustworthy suggests that, at present, few repositories are actually certified. For example, re3data.org 5 includes a registry of 1,500 research data repositories, "making it the largest and most comprehensive registry of data repositories on the web." According to repository66.org 6 , there are just over 3,000 digital repositories worldwide. In contrast, as of October 2017, 63 repositories have acquired WDS certification (basic certification), 7 50 repositories have acquired DSAs (basic certification), 8 seven repositories have both DSA and WDS certification (basic certification), 9 six repositories have TRAC certification (formal certification), 10 four repositories have CTS certification (core certification), 11 and two repositories have acquired NESTOR seals of approval (extended certification). 12 Furthermore, as of October 2017, no certification bodies have been accredited to certify a repository as ISO 16363 compliant. Given these numbers, it seems as though relatively few digital repositories have been certified as trustworthy out of all that exist.
There are several possible reasons why few repositories are certified at this current point in time. The standards and accreditation procedures are fairly recent; there has not been enough time for the standards to be adopted and for certification boards to be established on a broad scale. Related to this point, perhaps some digital repository staff members are unaware of the various certification standards and accreditation procedures. In cases where digital repository staff members are aware of the various certification standards and accreditation procedures that exist, perhaps they feel they do not have adequate time or resources to devote to audit and certification. Or, perhaps digital repository staff members do not think it is worth the effort to engage in audit and seek certification. Given the small number of certified repositories, despite the potential for audit and certification to improve digital repository trustworthiness, it is worth investigating the value of audit and certification from the perspective of digital repository staff members.

Benefits of Audit and Certification
TDR standards and accreditation procedures typically have accompanying websites that list specific benefits of audit and certification. For example, the DSA website lists seven main benefits of acquiring DSAs: 1) Stakeholder confidence, 2) Improvements in communication, 3) Improvement in processes, 4) Transparency, 5) Differentiation from others, 6) Awareness raising about digital preservation, and 7) Less labor-and timeintensive. 13 Table 1 lists each of these benefits and their descriptions. Having the Data Seal of Approval signifies to funders that the data they have invested in will continue to be available for reuse. Data producers can be confident that the data they have worked hard to create will be protected, and data consumers can be sure that the data they are using have been managed optimally.

Improvements in communication
Preparing for the self-assessment prompts a repository to communicate internally about their overall mission and goals in ways not always present in day-to-day interactions.

Improvement in processes
Conducting the self-assessment stimulates a repository to improve its processes and procedures and move to a higher level of professionalism, with an incentive to improve its operations over time.

Transparency
The DSA is designed to provide an open statement of repository evidence enabling anyone to evaluate the repository's operations and policies.

Differentiation from others
There are a growing number of options for depositing data. Having the DSA sets a repository apart from others and enhances its reputation, showing in an easily recognized way that the repository is following good practice.
Awareness raising about digital preservation In this age of instant communication, people often focus on access to digital resources but do not consider the importance of preserving data for future reuse. Complying with the 16 DSA guidelines shows Benefits Description a commitment to ensuring that data will remain usable for new generations.

Less labor-and time-intensive
The 16 Guidelines of the DSA are the entry level of the European Framework for Digital Certification, in contrast to the 34 criteria for DIN31644 or over 100 metrics in ISO16363. There is no site visit as the assessment is conducted online through an efficient tool.
In contrast to the DSA (which provides guidelines for their own certification), the Primary Trustworthy Digital Repository Authorisation Body (PTAB) provides training towards certification according to ISO 16363. The PTAB describes benefits of preparing for and undergoing audit and certification on the ISO 16363 website. According to PTAB, certification provides reassurance to those who fund repositories or who deposit their valuable resources. Certification "gives some comfort that someone besides the repository managers can tell them that the repository has (or has not) been doing a good job." 14 Certification also provides assurance that "digitally encoded information will be usable into the future." 15 For repository managers, certification provides: 16 1) something to show to funders and users, and 2) advice on where improvements are needed.

Research on Perceived Value of Audit and Certification
To date, very few studies have taken into account the perspective of digital repository staff members regarding the value of audit and certification. For example, Giaretta and Schrimpf (2012) identified four main reasons for undergoing audit and certification: 1) learning from the process, 2) having their processes and documentation reviewed, scrutinized, and approved by external professionals, 3) demonstrating to management and reviewers a willingness to undertake external, independent, international evaluations in order to reach the highest standards in digital preservation, and 4) contributing to the success of standards and their associated auditing processes. In a similar study, Waterman and Sierman (2016) surveyed staff at 18 repositories that had acquired DSAs. The survey asked participants to rate the importance of the seven benefits of acquiring DSAs that are listed on the DSA website. Participants rated the transparency benefit the highest, followed by awareness raising about digital preservation, stakeholder confidence, and differentiation from others. Additionally, participants rated the impact of DSA certification on various aspects of their own organizations and repositories. According to the participants, DSA certification had the greatest impact on their digital preservation policies and technical digital preservation practices. They also reported that having DSA certification had a considerable positive impact on their organizations' professional reputations.

Critique of Existing Research and Motivation for Study
Although previous research on digital repository staff members' perceived value of audit and certification is useful for identifying reasons/motivations for undergoing audit and certification, there are two potential threats to the internal validity of the previous research: data collector characteristics and data collector bias. Given the data collectors' involvement with the development of the standards and repository auditing processes, they may have subconsciously concluded that their study participants saw great value in audit and certification, whether or not the study participants actually provided multiple overt, explicit statements in support of that particular theme. Data collector characteristics may influence the nature of the data collected if the characteristics are related to the variables that are the focus of a study (Wallen and Fraenkel, 2001). Data collector bias refers to the unconscious distortion of data during the data collection process (Wallen and Fraenkel, 2001). Data collector bias may have affected the data collectors' interpretation of their study participants' responses in prior research on this topic. Given the newness of the standards and their auditing processes, the relatively little research investigating digital repository staff members' perceived value of audit and certification, and the threats to internal validity that may have impacted previous research on this topic, there is a need to more closely examine digital repository staff members' perceived value of audit and certification in a way that takes into account potential threats to internal validity more carefully in the research design.

Methods
This study investigates the value of acquiring DSAs from the perspectives of those who have them. To avoid data collector characteristics and bias that could potentially affect what data were collected and how they were analysed, only those without prior experience in the development of the DSA were selected to be a part of the research team that handled data collection and analysis. Specifically, this study included a data collector who is knowledgeable about repository standards but yet did not play an active role in the development of any of them. The data collector also does not serve as a formal third-party auditor, a contrast from existing research on this topic. Approval for conducting this research was received from the Indiana University Human Subjects Office.
Our findings are drawn from data collected during interviews conducted between August 2015 and February 2016. All participants were at organizations whose repositories successfully acquired DSAs. We selected these individuals because only digital repository staff members at institutions that successfully acquired DSAs would be able to speak from experience about the actual benefits of having them. The list of 64 acquired seals on the DSA website constituted the sampling frame. 17 The first author recruited participants by emailing representatives from each of the repositories that acquired the DSA and inviting their participation. The first author sent follow-up emails on two separate occasions to try to increase participation. As a result of these efforts, we successfully recruited 15 representatives from these repositories to participate in this study, with a response rate of 23%.
The primary purpose of conducting the semi-structured, 30-minute interviews was to understand the value of the audit process and certification from the perspective of actual digital repository staff members. The first author asked respondents to discuss: how they learned about the DSA certification, how they decided to undergo audit, how they prepared for it, what the process was like, any lessons learned, the perceived value of the audit process, and the perceived value of certification since attaining it. No incentives for participation were provided. All interviews took place by telephone or via Skype and were audio recorded. Afterwards, all interviews were transcribed. Transcripts were then coded using NVivoa qualitative data analysis software tool. Prior to analyzing the transcripts, the first author developed a codebook based primarily on the list of benefits on the DSA website. 18 The first author also remained open to identifying additional themes as a result of analyzing the transcripts. The first author and a hired graduate student coded the transcripts. We calculated inter-rater reliability using Cohen's Kappa. We achieved a score of 0.87; thus, on average, we agreed on codes 87% of the time.
Some participants played an active role in the development of the DSA, either as past or present DSA board members. This was seen as a potential threat to the validity of the data. In particular, DSA board members could subconsciously or consciously over-report the benefits of acquiring DSAs based on their knowledge of and experience with the standard. For this reason, additional data analyses were performed. Specifically, a series of Mann-Whitney U tests were performed to detect whether any statistically significant differences existed between DSA board members and non-DSA board members regarding the frequency with which they reported benefits of acquiring DSAs. All statistical analyses were performed using IBM SPSS Statistics 24.
Donaldson (2017) contains: 1) the coded interview transcripts and description of codes (i.e., codebook) in NVivo for Mac Version 11.3.2 (1888) file format (e.g., .nvpx), 2) the raw dataset that lists the frequency with which each benefit was mentioned by DSA board members and non-DSA board members, and 3) the processed/analysed data from the Mann-Whitney U tests in two different file formats (e.g., .doc and .spv).

Findings
The findings are organized in the following manner. First, demographic characteristics regarding the study participants are discussed. Second, we discuss the benefits of acquiring DSAs; they are rank-ordered by the frequency with which participants mentioned them during semi-structured interviews. Third, we present Mann-Whitney U test results.

Study Participants
The respondents who participated in this study are diverse in terms of their job titles, roles and responsibilities, and years of experience. As shown in Table 2, participants held a variety of different professional positions. Four participants were managers. Four participants were directors, assistant directors, or deputy directors. Three participants were archivists, digital archivists, or lead archivists. Three participants were professors, research fellows, or senior researchers. The organizations the participants represent are also diverse. As shown in Table 3, seven participants worked in archives, four worked in data centers, two worked in data repositories, one worked in libraries, one worked on data services, and one worked on a project. The participants reported having multiple years of experience working with the repositories that they represent. As shown in Table 4, five participants worked at their current repositories for ten years or more; four participants worked at their current repositories between five and nine years; and the remaining six participants worked at their digital repositories between one and four years. Six participants were involved in the development of the DSA by serving as past or present members of the DSA board (DSA Board); nine had no prior experience with the development of the repository certification standard (non-DSA Board). Table 5 lists each participant's status. doi:10.2218/ijdc.v12i1.481 Donaldson, Dillo, Downs and Ramdeen | 139

Perceived Benefits
Altogether, participants provided 432 statements relating to nine different benefits of acquiring DSAs. Table 6 lists each benefit along with the frequency that participants mentioned them. Frequency indicates the importance or relevance of each benefit in this study; we include both positive and negative perceptions. The remainder of this section describes participants' experiences with each benefit.

Stakeholder confidence
Participants most frequently mentioned stakeholder confidence as a benefit of acquiring DSAs. Twenty percent of all coded data pertained to participants' discussion of stakeholder confidence.

IJDC | Peer-Reviewed Paper
Consistent with the DSA board's definition of stakeholder confidence, participants mentioned funders, data producers, and data consumers as different classes of stakeholders with whom they either built confidence with (or hoped that they would build confidence with) as a result of acquiring the DSA.

Stakeholder confidence -funders
Participants were able to point to concrete examples in which they reaped the benefits of stakeholder confidence. For example, their funders provided economic/financial support as a result of acquiring the DSA. This was particularly true for staff whose repositories were part of the Common LAnguage Resources and Technology Infrastructure (CLARIN) (e.g., P03, P04, P13, and P14). The CLARIN federation of language data repositories requires its members to acquire DSAs and provides economic/financial support to those who do.
Participants who manage repositories outside of the CLARIN repository infrastructure were also able to point to the positive impact that acquiring the DSA had on their funders. In some cases, participants mentioned that their funders had heard about the fact that they acquired the DSA and congratulated them (e.g., P08). In other cases, participants directly mentioned the fact that they acquired the DSA to their funders, to which they positively responded (e.g., P11).
Participants reported using information about the DSA in funding applications to signify to potential funders that they are trustworthy repositories. For example, P02 said, "we use it in funding applications. So I think that gives us some credibility." Participants in fairly new organizations viewed having the DSA as critical for helping to engender stakeholder confidence, particularly for funders. For example, P15 made the point that, compared to other cultural heritage organizations in her country, her organization did not have the reputation that they had because her organization was new. Thus, it was much more important to have the DSA to vouch for the quality of her repository, since she could not rely upon a long-standing reputation: 'I think [having the Data Seal of Approval] definitely raises the level of trust among our stakeholders and our funders. They see it as very, very important. It's very important that we have it, and that is the main benefit, especially with a new organization in a climate where funding is very limited. Most of the big players, they don't need [the Data Seal of Approval] as much as we do. They've been around for 150 years. They've built a reputation already based on being there for a long time' (P15).

Stakeholder confidence -depositors
Participants emphasized the importance of the DSA for their depositors. Specifically, participants found that having the DSA encouraged people to deposit their data with them. For example, when asked if he could go back in time, to before he performed the audit, would he still do it, P03 said yes. According to P03, having the DSA provides "a communication to depositors of your repository. It adds some level of trust. I have heard from several depositors that they really liked the fact that we did this and it kind of shows that, you know, we do things the right way." As another example, P06 said: 'We actually use it as a selling point when discussing our repository services with researchers or other clients. I've personally seen that in conversations IJDC | Peer-Reviewed Paper with researchers and seen their positive reaction to the Data Seal of Approval. They really like being able to see something that's really visual and that shows that you that we are following those practices and that they can really trust that their data will be available, that we are going to take care of it, and we are going to be responsible stewards. So I think that it is really useful.' Participants also mentioned that depositors valued the DSA because it is viewed positively in depositors' applications to funders. Depositors told the participants that funders wanted to know that if they provided funds to the depositors for research, that the resulting data would be stored in a Trustworthy Digital Repository. Thus, the findings emphasize the importance of the DSA for depositors when applying for research funding. Mentioning the DSA was something that depositors believed strengthened their funding applications because they could mention the fact that they would deposit their data in a repository that has a DSA to prove to funders that their data would be preserved and accessible for the long term.

Stakeholder confidence -data consumers
Participants disagreed on the perceived effect of the DSA on data consumers. Some participants suggested that the DSA positively affected data consumers' confidence in data repositories (e.g., P03). Other participants said that data consumers did not know or care about the DSA (e.g., P11). And still other participants expressed that DSAs could potentially increase data consumers' confidence in data repositories if repository staff made more of an effort to explain to data consumers what the seal actually means. For example, P12 stated: 'When you've got a nice, highly-attractive, beautifully-designed red seal to put on your website which I would love to see pushed a little bit more to our end users. […] We don't push it massively to our end users. I think possibly we should do more, but I think the fact is that we were a trusted digital repository before we got trusted digital repository status. We can approve that stuff every day. If I was a new archive trying to make an impact, I would probably push it on the user side a bit more.'

Documentation
Participants stressed the importance of documentation during the interviews. In particular, participants expressed that the practice of acquiring the DSA helped them identify gaps in the documentation of their workflows. The process of acquiring the DSA helped the participants to understand where their documentation regarding processes and protocols were lacking, and to enhance their documentation accordingly.
Several participants described the knowledge about their practices and processes as information that people knew, but did not necessarily write down. The process of acquiring the DSA provided an opportunity for the participants to make the tacit knowledge about their digital preservation efforts much more explicit. For example, as P03 pointed out: 'There is a lot of knowledge in people's heads that is not made explicit and it's good to have these things written down also for continuity. You know, should people leave [the organization]. It is often a bad thing if the IJDC | Peer-Reviewed Paper 142 | The Perceived Value of Acquiring Data Seals of Approval doi: 10.2218/ijdc.v12i1.481 knowledge only exists in people's minds and there is no documentation around. So, yeah, we have written quite a bit of that through the DSA Certification and I think that's a good thing.'

Assurance
Participants reported benefiting from acquiring the DSA because it gave them assurance that they were managing their digital repositories in conformance with best practice. Another way participants described this benefit was that acquiring the DSA provided a benchmark by which they could assess their digital repositories. This was critically important to the participants because, without this assurance, or without a benchmark, they felt they had no way of knowing whether they were being effective at digital preservation. According to multiple participants, successfully acquiring the DSA let them know that their repositories were "on the right track" in terms of the scope and quality of the services they were providing, which was very important for them to know (e.g., P01, P07, P08). Overall, participants learned that they were fulfilling the DSA's requirements for digital repositories and were assured that they were not overlooking any major areas related to digital preservation. As P01 pointed out: '[the Data Seal of Approval] is a good tool for us to ensure that at least now, we are on the right track and that there were no large red flags identified. We look at [acquiring the Data Seal of Approval] as a confirmation of our existing policies and practices. To me, that's okay, but we can't just sit on our laurels. We have to keep on top of things as things are changing. We need to evaluate and evolve.' Participants also mentioned the benefit of being able to use the DSA as a means of comparing digital repositories. In particular, participants saw a major benefit in being able to determine whether their efforts were comparable to similar repositories or were lacking. For example, P08 said: 'It's also nice to benchmark yourself in that way because we also looked of course at the self-assessments that were filled out by other organizations because that's one of the nice things about DSA. When you get your seal, the self-assessment that you have written gets published on the Internet. So that is a very big database where you can check how your colleagues are doing. And that also gives you insight into your own position. So, it's also a nice benchmarking tool.'

Transparency
Participants described acquiring the DSA as an opportunity to demonstrate transparency. More specifically, acquiring the DSA forced participants to be more explicit about what they do and how they do it. It also encouraged them to provide this information to various classes of stakeholders. For example, P05 stated: 'I think up until relatively recently, repositories and archives like us have been sort of like black boxes. Nobody knew what happened inside and that's not right. We should all be exposing what we do to our constituencies.' doi:10.2218/ijdc.v12i1.481 Donaldson, Dillo, Downs and Ramdeen | 143

IJDC | Peer-Reviewed Paper
As another example, P15 described acquiring the DSA as an opportunity to "be more transparent about what you can do, what you can't do, what you can promise, and what you can't promise," which she saw as "the right thing" to do with respect to her repository's role in preservation and stewardship -articulating this role to various classes of stakeholders on behalf of her repository.

Improvement in processes
Participants described improvement in processes as a result of acquiring the DSA. For example, P14 pointed to modifications to their workflows and extending their technical solutions in order to pass audit. P10 discussed feedback that she received from reviewers after having submitted the application for DSA, which helped her organization refine some of its processes.
Several participants viewed acquiring DSAs as an opportunity to identify what processes need to be improved. Additionally, some indicated that process improvements are what they need to continually be open to learning more about. For example, P01 said: 'We've come to a mindset that we're willing to have folks review us with the understanding that we will probably learn something -maybe something we need to fix or change. And we have to be big enough to accept that.'

Awareness raising about digital preservation
Participants discussed acquiring the DSA as an opportunity to demonstrate a commitment to digital preservation to their stakeholders. Specifically, they described acquiring the DSA as demonstrating their commitment to preserving data and making data reusable in the future. For example, when asked whether it was worth it to undergo DSA audit, P05 said: 'I think [having the Data Seal of Approval] also gets people thinking more and being more aware of what they should be doing so when people come to our site and hopefully see that logo, they understand that means that we're preserving data to a certain standard and maybe it gets them thinking about it as well.'

Less labor-and time-intensive
All participants agreed that the process of acquiring the DSA was less labor-and time-intensive than the processes necessary for certification through other standards, such as DIN 31644 or ISO 16363. Participants saw this as a real benefit of undergoing DSA and selected DSA over other "more involved" repository certification standards because of it. As P06 pointed out: 'If everybody had to go through a full ISO 16363 certification to be able to show your trustworthiness, then I don't think there would be any repositories to be certified. Right? It is a very kind of daunting certification; whereas the DSA is a lot more approachable.' Even though participants considered audit and certification under DSA as less laborand time-intensive than certification under other programs, participants disagreed on how long it actually took them to prepare for DSA audit. For example, some participants reported that it only took them a few days or weeks (e.g., P03, P05) and/or it did not take much time to complete (e.g., P06, P07, P14). In contrast, other participants reported that it took them a couple of months to prepare for DSA-audit (e.g., P11) and/or that it took them longer than they expected (e.g., P01). The reason for this discrepancy most likely arises from the fact that some participants' repositories had become certified by other more involved certification programs before acquiring the DSA. According to the participants, since the other programs involve similar processes, they were already prepared for audit and certification under DSA.

Improvements in communication
Several participants described improvements in communication about the missions and overall goals of their repositories as a result of acquiring DSAs. Participants mentioned that staff at their repositories gained a better understanding of their repositories as a result of having to communicate for the purpose of preparing for DSAaudit. For example, P08 said: 'Because there were some issues that had to be taken care of [in order to apply for the Data Seal of Approval], people [within my organization] started discussing them in detail with one another. So information experts started talking with the Information Technology guys. So, in the end, everyone involved in this process has a much broader and much more complete insight into the workings of our own repository.' Some participants described the process of acquiring the DSA as an exercise in team building within their organizations, noting that their colleagues benefited from working together and even enjoyed the experience. For example, P09 said: '[Acquiring the Data Seal of Approval] involved many staff who had to look at more than ten different areas of compliance, and made sure that their areas were all up to speed (e.g., information technology, data archiving, etc.). It took a number of staff, working together and it was very successful, cooperative, and collaborative. The effort involved all of the staff working together. Everybody came together, brought their areas of expertise, brought their areas of commitment together, and worked together to achieve the status. I know that helped the work environment at both places for everyone to be going in the same direction. It was a really neat challenge in both environments. I didn't necessarily expect that. Beforehand, it seemed to me that undergoing an audit would be worse than, you know, going to the dentist and getting a tooth pulled. But in this case, it was really helpful for the work environments and achieved team-building.' Since P09 was involved in DSA audit at two organizations, he offered a unique perspective by being able to compare DSA audit processes at both places. He witnessed the benefit of improvements in communication for both repositories, thus twice underscoring the effect of acquiring the DSA on more effective and broader communication about the work that staff who are responsible for various aspects of digital repositories can actually experience.

Differentiation from others
Participants perceived having the DSA as an opportunity to differentiate themselves from other repositories. Specifically, they considered having the DSA as demonstrating that they abide by best practices for digital repositories in ways that other repositories may not (e.g., P02, P05, P13).
Participants were more concerned about being able to join a community of repositories that have acquired DSAs as opposed to separating themselves from those who do not have them. In this respect, the real value for them was being able to say that they were similar to other repositories that were doing a good job at digital curation. For example, P05 stated, "we liked the idea of joining this community of other repositories who had this DSA designation," and P06 stated that having the DSA demonstrates that "you have the same level of quality on the same level of the profession."

Mann-Whitney U Test Results
Mann-Whitney U tests were conducted to evaluate whether there was a statistically significant difference between those with experience on the DSA board (i.e., DSA Board) and those who had no prior experience on the DSA board (i.e., non-DSA Board) regarding how frequently they mentioned the nine benefits of acquiring DSAs that were identified during this study. We tested for the hypothesis that DSA board members would mention benefits more frequently than non-DSA board members because it was logical to assume that they have more experience with and knowledge of the standard. Results of the Mann-Whitney U tests suggest that, while there were differences in how frequently both groups mentioned benefits of acquiring DSAs, none of these differences were statistically significant (see appendix). In other words, from a statistical standpoint, participants who were past or present DSA board members did not mention benefits to acquiring DSAs any more or less than participants who had no prior experience with development of the standard.

Discussion
This study provides two main contributions to the digital curation research literature. First, it adds to the research literature on the perceived benefits of audit and certification of TDRs. In particular, it validates all of the benefits of acquiring DSAs which are currently listed on the DSA website. Participants mentioned stakeholder confidence, transparency, improvement in processes, awareness raising about digital preservation, less labor-and time-intensive, improvements in communication, and differentiation from others as benefits that they experienced as a result of acquiring DSAs. Second, this study identifies two additional benefits of acquiring DSAs that are not explicitly listed on the DSA website: documentation and assurance. The former refers to the benefit of identifying gaps in the documentation of repository workflows and filling those gaps by recording information to make tacit knowledge explicit. The latter refers to the benefit of knowing that staff members are managing their digital repositories in conformance with best practice.
Based upon this study's findings, we propose the following two recommendations to the DSA board. First, some of the descriptions of the benefits of acquiring DSAs need to be revised to more closely align with how actual digital repository staff members describe the benefits of acquiring DSAs. For example, description of differentiation from others should be modified to better articulate why those who have acquired DSAs are interested in this benefit. For them, it is more important to join a community of repositories who have DSAs than to differentiate themselves from those who do not have them. Second, two benefits of acquiring DSAs need to be added to the current list of benefits on the DSA website: documentation and assurance.
Based on the findings of this study, we propose two directions for future research. First, studies similar to this study should be conducted regarding the perceived benefits of audit and certification for standards for TDRs that correspond to extended (e.g., the NESTOR Seal of Approval) and formal certification (e.g., ISO 16363) once more repositories have had the opportunity to become certified by those programs. Such studies will provide insight into the perceived value of those methods of audit and certification. Results of those studies could also be compared with the results of this study to understand how digital repository staff members perceive the value of different audit and certification programs. Second, since this study was conducted, the DSA has been replaced by the DSA-WDS Core Trustworthy Data Repositories Requirements and the CoreTrustSeal (CTS). Consequently, studies similar to this study should be conducted after more repositories have had the opportunity to become CTS certified. Results of those studies should be compared with the results of this study to understand the extent to which benefits of audit and certification of digital repositories transcend both standards.
Based on the findings of this study, we recommend three methodological considerations for all future research on this topic. First, researchers who wish to study the perceived benefits of audit and certification of TDRs should consider not including people who have developed standards for TDRs as interviewers. Their involvement in data collection may introduce interviewer bias, affecting the validity of the results. Second, researchers may want to consider not including people who have developed standards for TDRs as part of research teams who are responsible for data analyses. Their knowledge of the standards and their stake in the success of the standards may affect how they interpret the data. Third, in cases where people who have developed standards for TDRs are study participants, researchers should analyse the data to determine whether statistically significant differences exist between study participants who helped develop or maintain the standards and those who have not. Such analyses help to ensure the validity of the data. This study provides an example of this type of analysis.
The primary limitation of this study is its sample size. Representatives of 16 organizations participated in this study out of a possible 64 organizations that currently have DSAs. The low sample size may have affected our ability to detect statistically significant differences between DSA board members and non-DSA board members regarding how frequently they mentioned the benefits of acquiring DSAs. To address this limitation, we tried to recruit participants on two separate occasions. This increased our response rate to 23%, which is above average when recruiting participants via email (Dillman, Smyth, and Christian, 2014). Future studies could devise more innovative ways to increase response rates during participant recruitment, and compare their results with ours to assess the potential impact of sample size. doi:10.2218/ijdc.v12i1.481 Donaldson, Dillo, Downs and Ramdeen | 147

Conclusion
The results of this study underscore the perceived value of acquiring DSAs by those who have them. Specifically, our findings demonstrate that, as a result of acquiring DSAs, digital repository staff members are able to:  Build the confidence of their stakeholders in them,  Improve their documentation,  Gain assurance that they are following best practice,  Demonstrate their transparency,  Improve their processes,  Raise awareness about the importance of digital preservation,  Spend less time on audit and certification as compared to audit and certification through other programs,  Improve communication among staff members, and  Join a community of repositories who have demonstrated their commitment to digital preservation and following best practice.
Ultimately, this research demonstrates the importance of trust in digital repositories. Many of the perceived benefits of acquiring the DSAs directly pertained to the need to build and maintain trust between digital repositories and their various classes of stakeholders, including co-staff, colleagues at other repositories, funders, data depositors, and data consumers. Further studies are needed to better understand the impact of perceived benefits of acquiring DSAs and other forms of certification on trust in digital repositories as the social, technical, and political frameworks in which these repositories exist continue to evolve. Table 8 lists the Mann-Whitney U tests results in terms of Mann-Whitney U values and significance. As shown in Table 8, none of the significance values are less than .05. This suggests that while there were differences in how frequently DSA board members and non-DSA board members mentioned benefits of acquiring DSAs, none of the differences were statistically significant. In other words, there was no statistically significant difference in how frequently DSA board members and non-DSA board members mentioned the benefits of acquiring DSAs. doi:10.2218/ijdc.v12i1.481 Donaldson, Dillo, Downs and Ramdeen | 151