Improving the Usability of Organizational Data Systems

For research data repositories, web interfaces are usually the primary, if not the only, method that data users have to interact with repository systems. Data users often search, discover, understand, access, and sometimes use data directly through repository web interfaces. Given that sub-par user interfaces can reduce the ability of users to locate, obtain, and use data, it is important to consider how repositories’ web interfaces can be evaluated and improved in order to ensure useful and successful user interactions. This paper discusses how usability assessment techniques are being applied to improve the functioning of data repository interfaces at the National Center for Atmospheric Research (NCAR). At NCAR, a new suite of data system tools is being developed and collectively called the NCAR Digital Asset Services Hub (DASH). Usability evaluation techniques have been used throughout the NCAR DASH design and implementation cycles in order to ensure that the systems work well together for the intended user base. By applying user study, paper prototype, competitive analysis, journey mapping, and heuristic evaluation, the NCAR DASH Search and Repository experiences provide examples for how data systems can benefit from usability principles and techniques. Integrating usability principles and techniques into repository system design and implementation workflows helps to optimize the systems’ overall user experience. Received 18 November 2018 ~ Revision received 2 December 2020 ~ Accepted 11 December 2020 Correspondence should be addressed to Chung-Yi Hou, 127 Haddon Pl., Montclair, NJ 07043-2314. Email: sophie.hou@ronininstitute.org An earlier version of this paper was presented at the 14 International Digital Curation Conference. The International Journal of Digital Curation is an international journal committed to scholarly excellence and dedicated to the advancement of digital curation across a wide range of sectors. The IJDC is published by the University of Edinburgh on behalf of the Digital Curation Centre. ISSN: 1746-8256. URL: http://www.ijdc.net/ Copyright rests with the authors. This work is released under a Creative Commons Attribution Licence, version 4.0. For details please see https://creativecommons.org/licenses/by/4.0/ International Journal of Digital Curation 2021, Vol. 16, Iss. 1, 21 pp. 1 http://dx.doi.org/10.2218/ijdc.v16.i1.592 DOI: 10.2218/ijdc.v16i1.592 2 | Improving the Usability of Organizational Data Systems


Introduction
According to Jakob Nielsen (2012), usability is the "quality attribute that assesses how easy user interfaces are to use" as well as the "methods for improving ease-of-use during the design process." By applying usability principles to user interface (UI) designs and evaluating user experience (UX) through usability techniques, a data repository can enhance and improve its services to provide users with positive and productive interactive experiences.
Usability is especially critical in web interface design. For the commercial sector, UI evaluation and UX management are often emphasized because maximizing the competitiveness of their web systems can affect customer satisfaction, and consequently, have significant impact on the companies' bottom lines (Dray, 1995). For data repositories, even though the missions or goals are not typically focused on increasing monetary transactions, the UI/UX of their websites and their users' satisfaction remain crucial. The data repositories' web interfaces are usually the primary, if not the only, method that is available to their users for searching, discovering, understanding, retrieving, and using the data and related resources that the repositories hold. Studies of scientific data interfaces have shown that sub-par user experiences can present some of the most difficult challenges to overcome when finding and accessing data (Chunpir, Williams, and Ludwig, 2017). Novice users in particular, such as students, can encounter significant roadblocks in downloading data from scientific data repositories (Rood and Edwards, 2014). Usability challenges can discourage people from even attempting to use digital repositories, even when they expect that useful resources can be found therein (Matusiak, 2012). Given that sub-par UIs can have significant impact on reducing the users' ability to access and use data, it is important to consider how repositories' web interfaces can be evaluated and improved in order to ensure useful and successful user interactions.
Getting user input on the operation and effectiveness of information and data systems can be difficult, however, since users are often external and even unknown. Operators of such systems may not know what problems users are encountering unless users provide detailed feedback on the specific problems that occur, and the circumstances in which they arise. Information scholar Michael Buckland noted this tension as it applied to library services in a book published in 1991: '...it is the responsiveness of the users, more than the responsiveness of the librarian, that serves to restore stability to the library service. This helps explain a noteworthy cybernetic aspect of library services: libraries, like other public services, can survive with remarkable stability even in the absence of effective management -survive, that is, not excel' (Buckland, 1991).
Direct interaction and feedback from the users are therefore essential to the development and iterative improvement of information and data services. While there is general acceptance of the need for digital repositories to be built in a user-centered fashion, the application of user-centered design and assessment methodologies to such systems is limited (Chapman, et al., 2015). Usability techniques provide a platform of established methods for gathering such feedback in a systematic and operational fashion. Many such techniques exist, and when applied systematically, can complement each

IJDC | General Article
Chung-Yi Hou and Matthew S. Mayernik | 3 other to provide many kinds of valuable insight into digital repository usefulness (Meyerson, Galloway, and Bias, 2012). This paper discusses how usability techniques are being applied to improve the functioning of data repository interfaces at the National Center for Atmospheric Research (NCAR). At NCAR, a new suite of data system tools is being developed and collectively called the NCAR Digital Asset Services Hub (DASH). The usability of the DASH services has been a priority since the beginning, and usability evaluation techniques have been used throughout the design and implementation cycles in order to provide positive and productive results for the DASH's intended community. By applying user study, paper prototype, competitive analysis, journey mapping, and heuristic evaluation, the NCAR DASH Search and Repository demonstrate examples for how data systems can benefit from usability principles as well as optimize the systems' overall UI/UX by integrating usability techniques into their workflows.

Background and Rationale
In 2014, using a community driven and informed model (Hou, Mayernik, and Worley, 2017), NCAR formed the Data Stewardship Engineering Team (DSET). 1 The DSET's key goal is to support the needs to discover and access NCAR's scientific products more efficiently and effectively. To this end, the DSET has created the DASH 2 , which is a set of openly available and publicly accessible web-based systems that are dedicated to providing data services in the following four major areas. Three of these services are currently operational, and the fourth (DASH Repository) is under active development with the goal of becoming operational in 2019. It is important to note that the DSET membership includes one to two representatives from each of NCAR's seven labs plus additional representatives from the NCAR Library and University Corporation for Atmospheric Research (UCAR) Community Programs (UCP). The membership reflects the scientific, technical, and data diversity of the NCAR community, and therefore, has been vital in facilitating communication from and participation by different stakeholders.
 DASH Consulting: A collection of resources and tools (hosted on the DASH's Drupal 3 -based website) that can help the NCAR researchers in learning about data management best practices as well as the DASH's services. The DASH Consulting website also provides information about how researchers can set up in-person consultation with NCAR's Data Curation and Stewardship Coordinator;  DASH Metadata: A workflow that focuses on assisting the NCAR researchers in creating and submitting metadata records for the scientific products produced by their research. The metadata records, which are compliant with the ISO 19115 geospatial metadata standard 4 , are part of the DASH Search and Repository processes and can be created using a CatMDEdit 5 -based metadata editor tool;  DASH Search 6 : A CKAN-based 7 , public metadata registry that allows anyone to search, locate, and access NCAR's scientific products;  DASH Repository: A new repository that supplements NCAR's existing, domain-specific repositories in order to enhance the preservation and sharing of scientific products produced by the NCAR research community. When developing and implementing the DASH services, the DSET relies heavily on a variety of open source software tools (e.g. Drupal, CKAN, and CatMDEdit), many of which are new to NCAR and its web environments. A common challenge in using open source software is that the out-of-the-box UI configurations are often not optimized for any specific community. Consequently, it has been necessary for the DSET to customize and update the DASH service UIs to make them more "useful," i.e. having the features that are needed and easy to use to the NCAR community (NIMIT, 2013).
A key strategy for the DSET to learn about how to produce useful DASH service UIs is by working with the other NCAR data management groups who have had experiences with the open source software. The DSET has also leveraged lessons learned from peer organizations outside of NCAR. In particular, external guidance has been helpful in understanding how different system components need to work seamlessly together to present a consistent interaction experience to data service users. Additionally, the DSET has employed a user-centered approach to further enable the DASH services' usefulness. In other words, by choosing design selections and implementation decisions that are based on users' overall goals instead of situation-specific tasks, the DASH services are less likely to become outdated as users' needs evolve rapidly over time (Cooper, Reimann, and Cronin, 2007). Ultimately, in order to best meet user needs from NCAR's diverse community, it is crucial that the DASH services' designs and implementations are guided by actual community members' feedback and participation. The DSET has therefore determined that in order to ensure that the DASH's UIs and their services are compatible, efficient, and scalable for its user community, user interactions need to be captured and analyzed iteratively throughout the DASH's development by using UI/UX methodologies.

Methodology
This paper focuses on usability evaluation methodologies that were used to improve the DASH Search and the DASH Repository systems (shown as the bottom two components in Figure 1). When determining the appropriate techniques to be applied to the DASH services in order to obtain effective usability improvements, the DSET first evaluated the types of usability issue that the team would like to address according to Nielsen's (2012) usability quality components:  Learnability -How easy is it for users to accomplish basic tasks the first time they encounter the design?
 Efficiency -Once users have learned the design, how quickly can they perform tasks?
 Memorability -When users return to the design after a period of not using it, how easily can they re-establish proficiency?
 Errors -How many errors do users make, how severe are these errors, and how easily can they recover from the errors?
 Satisfaction -How pleasant is it to use the design?
The DASH's services are new to the NCAR community. The goal with these services is to allow users to discover, identify, access, and archive scientific products expeditiously. It is unreasonable to assume that scientists and other stakeholders with limited time would want to go through intensive training sessions in order to learn the capabilities of the DASH services. As a result, when selecting usability techniques, those that could help with enhancing the DASH's learnability, efficiency, and error prevention were emphasized. Additionally, during the initial design and development phase of DASH and its services, it was vital that the usability evaluations were done with minimal disruptions to the project's timeline. At the same time, the evaluations still needed to produce meaningful results, so that quantifiable improvements could be included in the subsequent design and implementation iterations. Consequently, usability techniques that would be suitable for testing early instances of a web systems were also prioritized.
In addition to reviewing Nielsen's usability quality components, the DSET also leveraged the community experience from the Usability Cluster 8 of the Earth Science Information Partners (ESIP). 9 The ESIP Usability Cluster has been working with ESIP's community members, including the National Oceanic and Atmospheric Administration (NOAA), the National Aeronautics and Space Administration (NASA), Data Conservancy, and the Biological and Chemical Oceanography Data Management Office (BCO-DMO), to study and understand UI/UX challenges as well as potential solutions for data services, especially those that are serving the Earth science domain. Given that the DSET considers the ESIP community members as its peer organizations, the experience and the lessons learned from the ESIP Usability Cluster were helpful in assisting the DSET in recognizing common UI/UX issues for data services, and providing the opportunity to discuss how to avoid these pitfalls for DASH and its services.
The first UI evaluation featured in this paper focused on the DASH Search. As discussed above, the DASH Search aims to be the "go-to" starting point when a user would like to search, locate, and access NCAR's scientific products. The DSET also wants the DASH Search to serve as many different user types as possible, including researchers, students, and educators. As a result, it is crucial for the DASH Search to have designs and functions that are easy to understand and quick to learn. Additionally, the DSET wants the DASH Search to be able to guide its users as needed in order to minimize potential user errors and frustrations. Equally important, it is vital that the DASH Search presents accurate and consistent information to the users, so that they can trust the system and make confident decisions regarding whether they have found the scientific products that they are looking for. All of this user experience needs to be delivered to all user types regardless of their research experience level or whether they have prior experience with a domain specific search and discovery system. Focusing on these goals, the DSET decided to use user study, paper prototype combined with user study, and competitive analysis to help in evaluating the DASH Search during the initial design and development phase.
During the process of designing, evaluating, and implementing the DASH Search, the DSET also kicked off the design requirement gathering process for the DASH Repository. The DASH Repository is aimed to complement existing NCAR repositories by archiving datasets that do not readily fit into the extant repositories. The DASH Repository will also support researchers in meeting data sharing obligations, including open access requirements from funding agencies and publishers. Since the DASH Repository is being built from the ground up, the DSET has an invaluable opportunity to identify and understand clearly its users' needs, and use this knowledge to guide the selection and implementation of the corresponding solutions. In order to optimize the DASH Repository's ability to assist its users in successfully achieving their goals, the DSET decided to apply journey mapping to the DASH repository's design requirement gathering process. Table 1 in the Appendix summarizes the selected usability techniques and how they were applied to the DASH services.

DASH Search
The home page of the DASH Search facilitates direct searches and casual browsing of the DASH Search's content, i.e., datasets, publications, software packages, and other digital assets managed and provided by NCAR. Additionally, as the users continue to work with the DASH Search and until a desired product is identified, they can choose from additional functions, such as filtering and sorting of the returned results as well as viewing the metadata information of the scientific research products via the dedicated landing pages. Because all of the web pages under the DASH Search can potentially receive significant amount of user interaction, it is crucial for the whole DASH Search to be evaluated for usability.

User study
In order to enhance the DASH Search's usability, user studies were conducted on the DASH Search's beta version (i.e., a preliminary but operational version). Specifically, the

IJDC | General Article
Chung-Yi Hou and Matthew S. Mayernik | 7 "discounted" version, that is, a simplified version of the full usability testing setup, was adopted. By requiring a minimal amount of resources, the "discounted" user study technique allows several study sessions to be performed quicker and iteratively while still involving actual users (Nielsen, 2009). In the DASH Search's case, existing, available conference rooms were used as "test labs"; the NCAR Data Curation and Stewardship Coordinator with usability experience acted as the test administrator, and a total of five users from the NCAR community were included in each of the studies (the users were solicited and invited by NCAR DSET members to ensure different user participation for each session). The user study sessions were also designed to last at most one hour, but have several specific tasks included to allow users to interact with as many areas of the DASH Search as possible. This way, the testing sessions could be conducted expeditiously but with focus. In turn, the test results could be incorporated in the subsequent design and implement cycle while the test sessions themselves would not introduce delays to the project timeline. In the Appendix, List 1 has the sample tasks that the users were asked to perform, and Figure 1 includes the screenshots of the web pages that the test users saw.
After conducting the testing sessions, the overall result showed that the test users encountered a range of issues. However, a majority of the issues converged and could be considered as the "low-hanging fruits". These issues were resolved first because they would take fairly low engineering effort to correct, but would result in impacts that were highly desired by the users. Samples of these "low-hanging fruits" issues are included in Table 1 below. Besides "low-hanging fruit" issues, issues that would require additional design discussions and engineering effort became opportunities for DASH to enhance the DASH Search. By consolidating and prioritizing these issues, DASH was able to build an improvement roadmap that was appropriate for its project timeline and available resources. Most importantly, when the identified usability improvements were implemented after each study session, subsequent user studies showed that the increase in the DASH Search's ease of use was evident. In other words, users became increasingly more comfortable with interacting with the DASH Search and indicated willing to return to use its services in the future. Usability Improvement Potential Usability Impact Add a short description of the website and the key functions on the DASH Search's home page Users could quickly and easily understand the purpose and the available services of the system as well as how to get started.
Remove "Log In" from the DASH Search's home page since a user account is not required Help minimize confusion by removing extraneous functions that are not needed by users.
Hide any features that are still under-development or not yet fully operational Related to the usability improvement above, allowing users to interact with functions that are fully operational helps in building users' confidence in and trust for the system.

Provide version information of the DASH Search system
Being able to view any changes that were made to the system gives users the opportunity to understand if their established workflow with the system is impacted and to provide feedback to the system administrators as needed.
Enable the DASH Search system to be mobile friendly Users would have flexibility and not be restricted to using specific platform(s).
Use consistent layout for the landing page Being able to find and recognize the desired functions consistently helps in reducing users' learning curve and the amount of mental processing load when using the system. Subsequently, it could also help in retaining users' familiarity with the functions when returning to the system.

Paper prototype (combined with user study)
From the user study sessions, another major finding indicated that the landing pages (pages that display metadata information for particular scientific assets) in the DASH Search had significant impact on the user satisfaction. In particular, it was revealed that there were two major aspects of a landing page that would especially receive attention from the users:  The top portion of the landing page.
• This is the area that first "greets" the users when they arrive at a land page.
As a result, this is the natural point for the users to start the examination and understanding of the layout and content of the page.
 The "Additional Info" table.
• As the users scan the landing page and build the mental map of the page's layout, they would quickly learn that the "Additional Info" table provides the details of a specific scientific product. If the users would like to determine whether a product is suitable for their uses, they would then need to spend additional time in reviewing the information in this table.
Between these two areas, it was clear that the "Additional Info" table should receive priority for further evaluation. Particularly, the test users often relied on the information in the table to make the final decision regarding whether they had found the scientific research product that met their needs. Structuring the information in the table more clearly could therefore help the users in learning about, understanding, and selecting a product.
In order to assess the most effective style and format that should be used to display the metadata information via the "Additional Info" table, three style and format issues were prioritized based on the user feedback. For each of these issues, two alternative designs were created using the paper prototype technique. The style and format issues and their design options are summarized in Table 2 in the Appendix.
Paper prototypes allow the various design options to be manifested quickly without having to affect the current, operating system (Berkun, 2000). Equally important, the paper prototypes can be integrated easily with other usability methods in a cost-effective manner. In this case, the paper prototypes were combined with the discounted user study method so that actual users could again be involved in testing the different designs.
Since the test users were still able to interact and think aloud comfortably with the paper prototypes, the testing with the paper prototypes confirmed that the users continued to prefer the designs that incorporated the most usability principles. For

IJDC | General Article
Chung-Yi Hou and Matthew S. Mayernik | 9 example, for the "number of tables to use" issue, the "multiple tables that are expandable and collapsible" option was the most desirable one because with this option, users could control when they would like to see the information. At the same time, when the tables were separated as "Scientific Information" (which would always be shown by default), "Contact Information," and "Citation Information," users experienced lower cognitive load from each table while being able to understand the purposes of the tables quicker by scanning the headers of the tables. This finding was in line with Loranger's discussion (2014) regarding how users would typically like to focus first on information that was considered to be the most relevant (in this case, the "Scientific Information" table), and yet still retain control over if and when they would review the other available information. Likewise, when evaluating the "indication of metadata fields with missing information" issue and its design options, the proposal to keep the metadata fields in the table but provide "N/A" as the value was considered to be the most user friendly. This design option kept the presentations of the tables consistently while allowing users to discern easily the metadata fields that did not contain information. Figure 1 through 4 in the Appendix show the design improvements selected, and Figure 5 is the comparison between the original and the current land page design.

Competitive analysis
During the user study sessions, users also voiced the desire to have temporal and geospatial search/filtering capabilities added to the DASH Search. These are common features provided by scientific data repository systems. DASH Search users thus expected that the DASH Search would provide analogous, if not exactly the same, features. Since many data services from NCAR's peer organizations have already implemented temporal and geospatial search/filtering capabilities, the DSET decided to use the competitive analysis technique to help in determining the kinds of temporal and geospatial search/filtering capabilities that would be appropriate for the DASH Search. The competitive analysis technique, or a method to assess the relative strengths or weaknesses of competitor products or organizations (Usability.gov, n.d.), was used to review and assess similar functions with six other Earth and geoscience-centric search and discovery systems. Based on this assessment, paper prototypes were created to assist in evaluating how the DASH Search could present temporal and geospatial search/filtering capabilities in a clear and concise manner. As discussed above, paper prototypes enable physical representations of the system-under-evaluation's potential functions and can be created rapidly with minimum required resources (Nessler, 2016). It was an effective method to generate and visualize the DASH Search's design options quickly for the temporal and geospatial search/filtering capabilities. By combining the paper prototypes with user studies, the DSET was again able to complete multiple iterations of geospatial and temporal search features. These studies were accomplished within a two-week sprint in order to integrate the user feedback and confirm the final design for implementation. Figure 2 and 3 below show respectively the current implementation for the DASH Search's temporal and geospatial search/filtering capabilities.

DASH Repository
When deployed operationally, the DASH Repository will be another high visibility data service for the NCAR community. NCAR researchers often need to fulfil funding agencies' and publishers' requirements to archive and share data within a designated time frame. The NCAR community is therefore expecting the DASH Repository will be easy and friendly to use, and to offer both self-help and dedicated support to allow the dataset deposits to be completed efficiently with high quality.
Currently, the DASH Repository is being built to serve two critical user types:  A dataset provider who would like to deposit with the DASH Repository the datasets that underlie a submitted paper in order to meet the publisher's open access requirements;  A dataset consumer who is unfamiliar with NCAR's organizational structure but would nevertheless like to locate and access datasets produced and hosted by the DASH Repository.
Goals for these two user types were created by compiling and analysing the user stories submitted by the DSET members on behalf of their communities. These user stories were documented using the following format: As a < type of user >, I want < some goal > so that < some reason >

IJDC | General Article
Chung-Yi Hou and Matthew S. Mayernik | 11 The user stories provided information regarding how different types of user would like to interact with the DASH Repository. After reviewing and consolidating the user stories, the frequency for each unique user type was tabulated. Once the top user types were determined from the user stories, the most common goals could then be determined.
After the two critical user types and their goals were constructed, the NCAR Data Curation and Stewardship Coordinator used the journey mapping technique to develop the desired user experience for these users. Leveraging the information from the user stories, the journey map technique allowed the Coordinator to create a visualization that emphasized the experience and the key expectations of these two user types when interacting with the DASH Repository. The map also included the steps that these users might take, including the system capabilities that the user might use, in order to achieve their specific goals (Grocki, 2014). When compared with the other usability techniques, the journey map highlighted the "pain level," or how happy or irritated the users would feel, at each step the process of using the DASH Repository. The map for each critical user type was finally validated and enhanced through interviews with the DSET members. Figure 4 shows the journey map for the dataset provider user type.
By using the journey mapping technique, the DSET was able to gain insight into these two target user groups' workflows as well as their emotional experiences when interacting with the DASH Repository. This knowledge enabled the DSET in understanding the strengths and the weaknesses of the DASH Repository. Consequently, the maps also assisted the DSET in selecting the system features and developing the technology development roadmap that would support the desired user experience.

Conclusion
By applying user study, paper prototype, competitive analysis, and journey mapping techniques to the DASH Search and Repository, the DSET was able to observe firsthand the usability issues and evaluate the potential solutions to improve the corresponding UI/UX. The majority of the user experience issues found could be categorized by using usability heuristic principles from Nielsen's list (1995). For example, when reviewing metadata information of the scientific products via the DASH systems' paper prototypes, users made it clear that the terminologies used within the systems, including the keywords and metadata element names, were not intuitive to understand. This feedback was important for the DSET to keep in mind since the DASH's target users, including the research community, would typically be unfamiliar with the DASH's ISO 19115 based metadata schema. Due to the ISO 19115 schema's complexity, it would be difficult to expect the DASH's target users to invest time to master the schema's terminology and structures. Consequently, "matching between system and the real world" became a key heuristic principle to the DSET. By applying this principle to all subsequent design and development cycles, the DSET kept its focus on the users and made sure the DASH services would minimize the use of jargon. Instead, the DASH services would use terminologies that the users would naturally use.
With the competitive analysis, the DSET also recognized that Nielsen's "consistency" principle would be another helpful heuristic to uphold with the DASH's services. In the case of adding temporal and geospatial search/filtering functions to the DASH Search, the DSET observed that many of the other Earth and geoscience-centric search and discovery systems already have similar features. Since all the existing features share common characteristics and users are familiar with them, the DSET made the decision to implement analogous functions within the DASH Search. This way, the DASH Search's temporal and geospatial search/filtering behaves consistently with other systems that the users could also be using. In turn, this allows users to recognize the features quicker, and not requiring them to have to learn a new searching function. This design decision is also in line with the usability heuristic principle of minimizing memory overload. Finally, with the user studies, because actual users were involved, several heuristic principles were observed simultaneously, including making sure that the DASH Search and Repository could prevent user errors by having built-in checks, such as for invalid values and entry formats, while also giving users sufficient control and freedom to explore the systems' capabilities.
The different usability techniques described above work together to reveal diverse UI/UX issues and complementary improvement solutions. Based on the DASH Search and Repository experiences, the DSET sees benefits in continuing the integration of user-centered strategies when developing data services. Going forward, it is crucial that the DSET continues to recognize that the stages of the UX design process, i.e., User Research, Ideation, Prototyping, and Evaluation, are compatible with the software development life cycle. This recognition will allow the team to give equal emphasis on UX and software design and implementation. In turn, this balanced emphasis will enable the team to organize and prioritize its resources appropriately, and consequently, be able to complete concurrent requirements effectively. Additionally, while several different usability techniques are applicable for each UX design stage (Komninos, 2020), the DASH project has also shown that positive impact on the users' experience can be created by using just one technique per stage. In the case of the DASH project, using at least the following techniques is recommended for each of the UX design stage: Finally, as the team's user-centered culture matures, it is helpful for the team to foster additional, specific usability skillsets. For the DASH project, the NCAR Data Curation and Stewardship Coordinator (the lead author) has usability experience, so she was able to help start the user-centered design process as part of her responsibilities. However, adding dedicated usability expertise, such as graphic design, content strategy, and customer experience, will certainly further enable the team in optimizing the overall experience for all of its data services.

IJDC | General Article
Building on current lessons learned, the DSET will continue to apply these as well as other techniques to the DASH services. For example, cognitive walkthrough, survey, and card sorting are additional qualitative methods that can help to better understand the users of the DASH services and their interactions with the services. Equally important, the DSET would like to apply quantitative techniques to measure the performance of the DASH services over time. An example would be the benchmarking technique, where a system's current ease of use can be quantified as the baseline and be measured against the system's future usability performance (Berkun, 2003). Given that the DSET uses an Agile software development methodology, the DSET would also like to consider how to include lean UX practices as part of the sprints. Moreover, going forward, it will also be critical to evaluate and improve alternate access modes for machines, such as via Application Programming Interfaces (APIs), and for humans, such as through alternative contents and color schemes. Ultimately, the DSET aims to continue to focus proactively on and integrating user-centric approaches, so that the DASH and its services can have positive impact on as many different data user types as possible. -The use of paper prototypes as part of the discounted user study (as described below) gave the users the opportunity to interact with the design options and provide feedback as they normally would in a user study.
-The users were selected to reflect different experience levels of working with data services and in the scientific research environment.
-Each test session took about 1 hour to complete.

IJDC | General Article
Chung-Yi Hou and Matthew S. Mayernik | 17 -The sessions were not audio/video recorded.

User Study (discounted version)
A user study involves an actual user in performing tasks with the system-undertest while asking the user to think aloud in order to express the thought process behind his/her actions.

DASH Search's Search and Browse
-The "discounted version" refers to the use of a total of five test users and one test administrator, and that a conference room acted as the impromptu test area instead of setting up a formal test lab.
-The users were selected to reflect different experience levels of working with data services and in the scientific research environment.
-Each test session took about 1 hour to complete.
-The sessions were not audio/video recorded.