Accessibility in Online User-Testing

The current COVID-19 crisis has revealed the crucial role of online communication technologies in providing unique opportunities to carry out qualitative research in online user-based testing. The ability to provide a shared common space for participants living in different parts of the world and to record discursive data in text format accurately, makes these tools crucial in gathering qualitative data for research studies (Turney & Pocknee, 2005). Although the accessibility of the online communication platforms is improving, they still present significant challenges for all users, especially when running synchronous meeting sessions with participants in remote settings (Dodds & Hess, 2020).


Introduction
This article analyses problems and discusses solutions specifically related to accessibility in online user-based settings for research purposes. The article will first provide a brief overview of the current legislative framework involving media accessibility, along with the definition of the terms "accessibility", "usability" and "user experience". Secondly, it will provide a description of the different steps to be considered when carrying out an accessible online user-based testing, with special focus on user recruitment and ethical procedures. Thirdly, it will outline the two main approaches for conducting an online user-based testing, namely synchronous and asynchronous. Finally, it will analyse the accessibility features and compliance levels of the latest online communication platforms according to the different standardisation agencies. Conclusions will be used to support the decision-making process for selecting the most suitable online platform to conduct accessible online user-based testing in a research project.

Digital Accessibility Legal Framework in Europe
One of the basic pillars when carrying out user-based testing with a diverse number of users is to plan an accessible testing session from the onset of the project. Considering accessibility at the earliest stage of the design process of any user-based testing, helps to remove any potential participation and interaction barriers for all types of participants.
Last decade the Council of Europe signed and ratified United Nations Convention of Rights of Persons with Disability (UN CRPD, 2006) 1 . This has resulted in three pieces of legislation: European Web Accessibility Directive (2016) 2 , Audiovisual Media Directive Directive (updated 2018) 3 and the European Accessibility Act (2019) 4 . This European legislative framework should ensure full and democratic participation for all citizens in the new Information Society. The recent adoption of these laws and policies at the EU and international levels put accessibility at the front line requiring that any software, web content, documents and hardware can be accessed in a way that all people regardless of their capabilities can use and interact with it. In order to allow all types of users to evaluate any Information and Communications Technology (ICT) product, service or tool, the userbased testing session should be carried out with accessibility in mind.

Accessibility, Usability and User Experience
According to article 9 of the United Nations Convention of Rights of Persons with Disability (UN CRPD, 2006), ICT technologies, products, services and tools should be easy to use for all people. Adapting a Universal Design for All approach at the earliest stage of the design and development process maximizes accessibility and usability of any ICT service and product for all potential end-users (Oncins & Orero, 2020).
Accessibility is a term for which there is a wide range of different definitions. International Organization for Standardization (ISO) 9241-171 (2008) defines accessibility as "the usability of a product, service, environment or facility by people with the widest range of capabilities." Usability has also been variously interpreted. The ISO 9241-11 (2018) defines "usability" as "the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use." Finally, user experience (UX) has even more interpretations. ISO CD 9241-210 (2019) defines user experience as a "person's perceptions and responses resulting from the use and/or anticipated use of a product, system or service" including user's emotions, beliefs, preferences, perceptions, physical and psychological responses, behaviours and accomplishments that occur before, during and after use. UX evaluation is the process that is done in order to improve a product. Product development is often a hectic process and the resources for UX evaluation are scarce. Evaluating at an early stage of developments is recommended, as the earlier the evaluations can be done, the easier it is to modify the product accordingly.
All these definitions mean that for a product to be usable (effective, efficient and satisfying) and accessible, users should be able to use it (technical accessibility) to achieve their goals in an acceptable amount of time and be satisfied with the results as part of the user experience.

Online User-Based Testing Procedures
User-based testing has been broadly studied (Dumas, 2003;Dumas & Redish, 1993;Karat, 1997;Lewis, 2006a;Rubin, 1994;Tullis & Albert, 2008). In user-based testing participants are invited to perform specific tasks with a product, or to explore it freely. Their behaviours are observed and recorded in order to identify design flaws that cause user errors or difficulties.
In order to conduct a user-based testing, there is a set of standardised techniques. According to Bastien (2010, p. 13) the implementation of a user test generally goes through a certain number of steps such as: • the selection of tasks participants will have to perform; • the creation and description of the task scenarios; • the choice of the measures that will be made as well as the way data will be recorded; • the preparation of the test materials and of the test environment (the usability laboratory); • the choice of the tester, and the design of the test protocol per se (instructions, design protocol, etc.); • the design and/or the selection of satisfaction questionnaires, the data analyses procedures; • and, finally, the presentation and communication of the test results.
For the purpose of this paper, a brief mention will be made of user sampling and recruitment, and ethical considerations; special attention will be paid to conducting online user-based testings and the further evaluation of the accessibility features in online communication platforms.

User Sampling and Recruitment
Main challenges in any qualitative research involving users are first to define and then find the appropriate sample of participants. This is especially relevant when the user-based testing session is aimed at including all types of users. Yet, the number of participants to include in any experimental research remains a subject of debate among researchers and some studies have led to the conclusion that testing with a large number of users does not yield more significant results than testing with a fewer number of users; most usability problems will be detected by the first users (Lewis, 2006b;Orero et al., 2018;Turner et al., 2006). Research results show that five participants are enough to achieve the maximum benefit-cost ratio for testing when comparing different users testing the same ICT product, service or tool in fairly similar ways (Nielsen, 2012). Exceptions apply depending on the type of research method and tool (i.e., surveys to get statistics or eye-tracking studies).
In terms of recruitment, participants with disabilities are hard to reach and effectively engage in research ; they often feel vulnerable, intimidated and have a lack of trust in the research process (Dodds & Hess, 2020). To establish a close connection with various communities of end-users, it is crucial to enlist different types of users and be aware of their needs, as accessible communication in all steps of the testing process has to be ensured.
When moving to an online environment, accessibility features of the platform to be used by the participants have to be previously tested and tailored to their needs. Apart from these features, issues related to various capabilities such as digital skills of the participants can prove to be problematic. A usability approach with an additional focus on capabilities, in which the needs of different types of users are considered, could prove to be effective when conducting a user-based testing (Agulló et al., 2018;Tor-Caroggio & Orero, 2019).

Ethical Considerations
The increasing use of online methodologies raises some ethical issues which are unique to this type of research in addition to more traditional ethical concerns (Brownlow & O'Dell, 2002). When involving participants with different needs in user-based testing, special attention should be paid to ensuring equal communication channels for all (e.g., when necessary provide sign language interpreters and also consider the needs of any people with speech, hearing, visual or cognitive disabilities). According to Brownlow & O'Dell (2002, p. 13), the "process of 'giving voice' via online methodologies will largely be ignored if the individuals concerned are not given the opportunity to reinterpret texts and guide the researchers, giving informed consent for their narratives to be used in research". Therefore, proper ethical procedures for human subject research in the social sciences have to be followed ). An information sheet should be provided to participants detailing what is expected from them and what they can expect from the testing session. This information document has to be provided in written form or read out, and should come along with a consent form that has to be signed by all participants. Allowing enough time for understanding and completing any necessary forms is crucial. Obtaining consent is important as both personal data and potentially sensitive information might be collected. Therefore, two issues become crucial from an ethical perspective: the confidentiality of the information and the anonymisation of personal data. Special care should be taken when dealing with user activities involving participants from vulnerable groups. According to the European textbook on ethics research (European Commission, 2010, p. 53), vulnerability is a very complex concept. The following indicators could be used to define vulnerable groups: • Subjects who lack competence will be unable to protect their interests by choosing to give or withhold consent. • If the voluntariness of consent is compromised, this may similarly prevent the subjects from choosing to give or withhold consent in a way that would protect their interests. • The physical (or psychological) condition of some subjects leaves them especially liable to harm, for example as a result of frailty through age, disability, or illness.
In these cases, special consent forms and information sheets might be drafted to ensure that the rights of these subjects are guaranteed. Additionally, different legislations may apply depending on the country in which the user-based testing session is conducted.

Conducting Online User-Based Testings
Online user-based testing refers to a situation in which the researcher conducting the testing and the participants are not in the same room or location. Using the Internet to conduct online user studies with particular groups has enormous potential for qualitative social research (Turney & Pocknee, 2005). In traditional face-to-face user-based testings there are the costs associated with travel, time and accommodation. Online user-based testing is cost effective and time saving, especially for travel expenses when participants are recruited in different regions of a given country. The adoption of new communication platforms, such as the ones detailed in section 4.2, allow remote participation and eliminate the costs associated with the travel and loss of time necessary when working in traditional face-to-face environments. Still, accommodation should be ensured in order to allow full participation and interaction of all users.
Two approaches to online user-based testing can be defined: synchronous and asynchronous. Each approach uses specific methods and tools. Asynchronous approach includes methods such as surveys and questionnaires, which are mainly delivered to participants before and after the testing session via email or other online channels. The synchronous approach includes methods such as interviews, focus groups and user observation with the use of think-aloud protocols. These are conducted during the session and involve real-time interaction between the participants and the researcher.
These qualitative data collection methods in both approaches are particularly important to gain depth, insight and understanding of particular issues which cannot be obtained only through quantitative research. They have become increasingly popular among social researchers over the last few decades (Adams & Cox, 2008;Carey, 2016;Creswell, 2013). What is often lacking is an understanding of how the research method design fits with the research question of the testing session (Creswell, 2013;Orero et al., 2018), and how to appropriately utilise these different approaches for specific Human-Computer Interaction (HCI) needs (Adams & Cox, 2008). Creswell (2013) provides a comprehensive analysis of the different quantitative and qualitative methods and how they can be mixed and matched to improve the quality of research.

Asynchronous Approach
In the asynchronous approach, communication with participants is mainly done through email or other web-based platforms, when data is needed from a large number of disparate users (Adams & Cox, 2008). It does not involve observational data, and video or audio recordings of spontaneous verbalisations from users. Researchers do not have access to the data in real time, and there is no researcher interacting with participants during the session. The qualitative data is mainly gathered through online surveys, questionnaires or self-report forms. All of them are in a written format. Thus, it can be both cost-effective and easier to analyse than other methods which imply verbalisation. On the other hand, one of the main problems when using surveys and questionnaires are the dropouts, as users may decide at any time to quit the survey without notification; moreover, there is no control over participants, as research is carried out under uncontrolled conditions (Birnbaum, 2004).
Therefore, knowing what questions to ask is crucial when constructing a questionnaire . Triangulating between qualitative and quantitative methods can also help to avoid possible bias in responses due to uncontrolled online conditions (Adams & Cox, 2008).
The asynchronous approach can be useful in gathering both quantitative and qualitative data for specific purposes, e.g., to see what people do in a particular situation and/or to analyse their preferences (i.e., which types of software they use and how frequently, or what their preferences are in the use of specific software).
In terms of accessibility, when using an asynchronous approach, it is important to make sure that documentation provided to the participants is made accessible and can be accessed in an alternative format. Some online and offline tools already contain built-in accessibility settings and checking options that can be used to assess whether provided documentation is also accessible for assistive technologies such as screen readers.

Synchronous Approach
The synchronous approach allows the capture of qualitative data in real-time. Traditional face-toface user-based testings have some disadvantages, particularly when dealing with participants that are geographically dispersed or hard to reach, and when dealing with sensitive topics (Birnbaum, 2004). Online user-based testings therefore have the potential to address these issues while also offering researchers the opportunity to avoid the costs of finding an ideal location with the proper accommodation to conduct their testing sessions (Kite & Phongsavan, 2017). The researcher/facilitator conducts the session in real time with one or more participants who are connected remotely. This approach requires video communication platforms that allow interaction and, if necessary, sharing the computer screens so that the researcher/facilitator can see what is happening on the screens of the participants. There are three main types of synchronous online userbased testing methods, namely focus groups (FG), interviews and user observation with the use of think-aloud (TA). If the main aim is to identify why something has occurred, a questionnaire will provide less valid responses than FG, interviews, or TA, because these last three methods allow participants to express and verbalise their opinions in context.
In FG, a facilitator addresses a group of participants through a set of questions related to a particular topic and participants are expected to verbalise and share their opinion in an unstructured form. FG are the most common method used in research to investigate the opinions and experiences of a defined group of people (Carey, 2016;Krueger & Casey, 2009).
In interviews, a researcher/facilitator addresses a single participant. Any particular interview can fall somewhere between unstructured and structured. The unstructured interview is more of an open conversation, in which the interviewer may contribute as much as the interviewee and the communication is based on a list of topics with questions that do not have a clear structure. On the other hand, a structured or a standardized interview "entails the administration of an interview schedule by an interviewer" (Bryman 2012, p. 210). The middle ground would be covered by a semistructured interview in which the order of the questions can be adapted, and the questions are more open than in structured interviews.
Finally, in the TA method, a single user is asked to think aloud while interacting with the evaluated system. It is a method used to understand cognitive processes based on tasks performed by participants, both within usability tests and in the broader study of HCI (Donker & Markopoulos, 2002). This method ranges from open and unstructured questions to structured interviews that can be adjusted according to the participant behavior and responses. TA makes it possible to obtain direct verbal information from the users about their thoughts.
Depending on the type of user-based testing, asynchronous and synchronous approaches can be used together. It might be useful to start with a questionnaire or survey and then follow up with a focus group, interview or TA in order to explore some aspect of the issue under study in more detail.
In all cases, one of the main challenges is to design the session with accessibility in mind. The information and documentation provided for the session has to be prepared in accessible alternative formats and made available to participants in their preferred formats. In the case of the synchronous approach, there is the added challenge to find an accessible online video conference platform that enables full participation and interaction with all types of users.

Evaluating Online Video Conference Platforms
When selecting an online meeting platform to conduct the focus group, interview or TA, it is important to consider which accessibility services are needed and what way they will be presented.
In face-to-face meetings, three main basic accessibility services are usually provided to allow participation for all users, namely captions/subtitles, audio description and sign language. When moving to an online environment, other related technological aspects will also be crucial for the selection of the online video conferencing platform. In some cases, this can create a complex process in which separate systems have to be set up to allow users to participate and interact.

Online Meeting Platforms According to Accessibility Standards
Ensuring access for all people in the different phases of the user-based testing session remains a major challenge. Designing testing procedures with accessibility in mind already in the initial stages of the project allows researchers to avoid possible accessibility issues that can obstruct the online testing session, such as a blind user not being able to login onto the testing platform because the selected software is not compatible with a screen reader.
At the global level there are four main agencies for accessibility, namely IEC (International Electrotechnical Commission), ISO (International Organisation for Standardisation), ITU (International Telecommunication Union) and W3C (World Wide Web Consortium). All of them provide standards, guidance and recommendations related to accessibility Oncins & Orero, 2021).
ITU has a technical report (ITU, 2015) with guidance to ensure that participation is accessible to remote participants, including people with disabilities and those using assistive technologies. It also provides recommendations for the preparation and management stages. The first part of the document lists three sets of requirements: essential, important and additional. As for essential requirements the following aspects are outlined (ITU, 2015, p. 3-11): • Ensure that information and registration processes are accessible to all; • Provide real-time captioning; • Provide audio streaming from the meeting room; • Provide sign language interpretation when needed; • Provide a way for remote participants to ask to make a comment or ask a question; • Allow interventions by voice; • Allow interventions by text; • Enable communication directly with the remote participation moderator and/or technical support; • Provide information in advance on how to participate remotely; • Ensure that the remote participation tools are accessible and available to as many persons with specific needs as possible including persons with disabilities; • Ensure access to contents of presentations for participants with vision impairments; • Ensure that meeting documents are accessible and available to as many persons with specific needs as possible including persons with disabilities.
All these essential requirements may also apply to online user-based testing, as participants are not present in the same location as the researcher conducting the session and they are participating in the project via audiovisual communication. In order to achieve proper HCI, specific reference is given to the W3C Web Content Accessibility Guidelines (W3C WCAG 2.1), which are also an ISO/IEC standard (ISO/IEC 40500:2012) and have been adopted in the European standard EN301549. ISO also develops specific documents that provide guidance on access services ; a detailed list of ISO standards in relation to accessibility can be found on the ISO website.
At a more technological level, WCAG 2.1 specifies how to make web content more accessible to people with disabilities. These guidelines are organized around four accessibility principles: • Perceivable: users must be able to perceive it in some way, using one or more of their senses.
• Operable: user interface components and navigation must be operable.
• Understandable: information and operation of user interfaces must be understandable (i.e., use of clear and simple language).
• Robust: content must be robust and interpreted by a wide range of user agents, including assistive technologies.
In order to apply these accessibility principles, the Education and Outreach Working Group (EOWG) at W3C develops resources to promote awareness, understanding, and implementation of web accessibility following the WCAG2.1 guidelines. Two main resources have been developed by EOWG to guide researchers and organisations in providing accessible digital content: "How to make your presentations accessible to all" 5 and "Making audio and video media accessible" 6 . In addition, the W3C document "RTC accessibility user requirements" 7 outlines three main conditions for a remote meeting to be accessible. First, the user interface of the software to be used has to be accessible in compliance with WCAG 2.1. Second, the content to be shared with the participants (i.e., documents, presentation slides, pre-recorded multimedia) has to be provided an accessible format. Third, access has to be granted to the live audio and communication with other meeting participants. This last aspect refers to real-time communication involving the use of the software that has to allow access for all users to participate and interact in a remote meeting.

Accessibility Evaluation According to WCAG
This section presents the main accessibility features that were tested on different online communication platforms. The accessibility testing and evaluation was performed on the latest public versions of the most popular videoconference communication platforms, namely Google Meet, Gotomeeting, Jitsi Meet, Microsoft Teams, Skype, Cisco WebEx and Zoom. The analysis covered assistive technologies (screen reader and keyboard navigation, screen magnification) and the following accessibility services: audio description, subtitles/captions, transcripts and sign language. The accessibility testing was carried out manually and following accessibility conformance reports when available. The considered criteria fall within different levels of compliance with the Web Content Accessibility Guidelines (WCAG) 2.1. Table 1 shows the main accessibility services and the most common assistive technologies (ATs) in the tested platforms.
The aim of this analysis was to highlight some points about how the presence or absence of certain key accessibility features can impact usability and users' experience. While accessibility features might be available in a platform, they can still pose problems in terms of usability for users with limited digital skills. In addition to these accessibility features, poor connection performance can make completing the test difficult or cause rescheduling.
The first three features, namely screen readers, keyboard accessibility and screen magnification, allow blind and low vision users to navigate, understand and interact with the online platform. They may help in, for instance, listing headings of a page, using shortcut keys or resizing text. The fourth feature (audio description) involves an additional audio channel as an alternative format if an audiovisual product is shared on the screen of the researcher. The fifth feature (Subtitles/Captions/Transcript) makes it possible for the people who are Deaf and Hard-of-Hearing to use an alternative way to access verbal information. Most of the platforms under analysis include subtitling features, either automatically generated or through a third-party provider. Finally, the use of speech recognition allows people with motor disabilities to navigate and use the platform using voice commands (i.e., access menus, activate commands such as mute or unmute). People with motor disabilities generally face barriers associated with using a mouse or a keyboard to access and participate in a web platform. The videoconference platforms with best accessibility features include Zoom, Google Meets and Microsoft Teams followed by Skype, Cisco Webex, Gotomeeting and Jitsi Meet. This last one is the only open-source video conference platform included in the table, and it is the least compliant with the listed accessibility requirements under analysis. The platforms offering the best accessibility features (i.e., Google Meet, Microsoft Teams, Skype and Zoom) have accessibility conformance reports (VPAT) publicly available on their websites. These documents are basically there for procurement purposes and indicate whether the accessibility level of the ICT software, product or tool is compliant with the current accessibility legislation in a given country (i.e., standard EN301549 in the EU or Section 508 in the US).
A closer look at the available accessibility features shows that screen reader and keyboard accessibility are supported by all video platforms except for Jitsi Meet, which offers partial support. Users report that when entering a meeting on Jitsi Meet, they often get trapped as all participants might be assigned the same name. It should be mentioned that Microsoft Teams allows screen reader users to access shared presentations with the screen reader. In the case of screen magnification, the only platforms not providing the features are Gotomeeting and Jitsi Meet.
The provision of audio description (AD) it is a feature lagging behind in almost all of the video conference platforms. According to WCAG 2.1, AD refers to success criterion 1.2.5. Audio Description (pre-recorded). In its accessibility conformance report, Zoom refers to the service as supported with exceptions, stipulating that the platform does not provide the possibility to select a separate audio track while viewing shared video content. In the case of Teams, Skype and Google Meet, this success criterion is indicated as not applicable in their accessibility conformance report. In the more particular case of Google Meet, it is further specified that the web application does not have video content. The other platforms do not provide any type of information regarding AD. Managing AD in real-time conference platforms is a major problem. Any audiovisual content shared or streamed in a testing session should include the necessary visual information to allow blind and low vision users to access the content.
As for subtitles/captions and transcripts, all platforms support this feature. According to European Federation of Hard of Hearing People (EFHOH, 2020), users of this service prefer closed captions and real-time transcription in the same window as the video conferencing itself, as opposed to a separate window. There are two main ways to deliver this service. One option is to include a human-based real-time subtitling service provided by third parties, which needs to be ordered separately. The second and most commonly used option is the use of the available automatic speech recognition system (ASR) on the platform. This feature might be popular due to the fact that subtitles can be helpful to many participants, such as non-native speakers.
According to the EFHOH report (2020), respondents report that automatic captioning is often not accurate, particularly when the speakers fail to speak directly to the microphone or there is background noise at the speaker's end. Users emphasize their preference for a live captioner to ensure quality, especially for important meetings or workshops. In addition to the sound quality when using ASR systems, there is the additional problem of available languages. While ASR works accurately for meetings held in English, this is not the case for online meetings in other languages, in particular in minority languages (Oncins & Delgado, 2019). In fact, Zoom, Gotomeeting and Cisco Webex platforms only offer automatic captions in English. Google Meet 15 allows users to activate automatic subtitles in 5 languages and dialects. MS Teams 16 supports subtitles in 16 languages and dialects but does not allow its users to include human-based real-time subtitles from third parties. Skype users can activate subtitles in 11 languages and dialects. In the case of Jitsi Meet, it is possible to install Jigasi which uses Vosk 17 , an offline open-source speech recognition toolkit which includes speech recognition models for 18 languages and dialects.
Apart from accuracy, further issues related to subtitles and transcripts displayed on the screen have to do with legibility (i.e., font and size) and readability (i.e., reading speed) principles. Readability varies depending on whether the subtitles and transcripts are human-based or computer-based. In the first case, the real-time subtitler can edit the output text that will be shown on the participants' screen and adjust the characters per second (cps) to a proper reading speed. When using a computerbased system through ASR, no editing is done and in some cases speech rates of the text displayed on the screen might be too high as reported by EFHOH (2020), and therefore would be difficult to follow. In terms of legibility, all platforms offer sans-serif fonts for the subtitles and transcripts which are recommended in Easy-to-Read principles. However, the only platform in Table 1 allowing its users to adjust the font and size of the subtitles displayed on the screen is Zoom.
Subtitles and transcriptions are also important for deaf-blind braille users participating in a meeting, as different technical requirements apply to ensure that braille users can access an alternative format to visual and audio information.
Regarding sign language, one of the main challenges in the use of online communication platforms in any type of meeting is the set-up for sign language interpreters to provide an accessible alternative format to Deaf users. All platforms in Table 1 include the "pin" option which allows their participants to "pin" anyone's video, so that specific video is always shown regardless of who is speaking at a given moment. This feature allows Deaf users to "pin" the sign language interpreter, so they are always present on the screen. Deaf participants need to see the interpreter at any time, because they might be signing their comments, which in turn have to be spoken aloud by the interpreter for the benefit of other participants. The main problems with this feature include being able to provide a reasonably sized window for Deaf participants watching Sign Language (SL) interpreters, and the ability to "pin" different participants at the same time. Depending on the digital skills of the participants, this may cause usability problems.
Finally, the speech recognition feature involves the use of voice commands to allow participants access and interaction with the platform. All platforms apart from Gotomeeting and Jitsi Meet incorporate this accessibility feature.

Conclusions
Online user-based testing makes it possible to collect and analyse data from a more diverse type of user than face-to-fade testing sessions. Temporal and spatial flexibility facilitated by online environments is a clear advantage for both the researcher and the participants. When designing the testing session, ethical considerations should be strictly followed, and human rights have to be safeguarded . Identifying and recruiting individuals for online user-based testing might be problematic and close collaboration with end-users associations should be sought.
There are two main approaches in online user-based testing: asynchronous and synchronous. Asynchronous settings can be more cost-effective and larger numbers of participants can be managed, allowing for a broader understanding of the subject matter. On the other hand, participants might initially agree to take part or even start a survey but quickly drop out. In addition, due to the uncontrolled conditions of this approach, researchers may miss important insight on relevant information that could otherwise be obtained in real-time interaction.
Conversely, synchronous settings allow for more open exchanges, providing in-depth qualitative data that is more oral than written. Yet along with these advantages an accessible set-up could be more difficult. Ensuring a proper set up to allow access for all types of users in a real-time online environment can be complicated due to the accessibility needs and digital skills of the participants. When planning an online user-based testing session, researchers have to make sure that the accessibility needs of the participants taking part in the asynchronous and synchronous online userbased testing are catered for, and documentation and materials are provided in alternative formats. In order to avoid problems related to digital skills, particularly in the case of the synchronous approach, proper instructions and, if needed, assistance have to be provided.
Although the accessibility of video conference platforms is improving, there are still significant challenges for people with disabilities. For people experiencing disability, some aspects of a video conference platform selected for online user-based testing may present various difficulties. According to current accessibility legislation and standards, access should be granted to all in an accessible and understandable way.