ACCEPTANCE TESTING OF AN ELECTRONIC DEVICE FOR A SELF-COMPLETION APPROACH IN SLOVAKIA

In 2021, the Slovak team of the European Social Survey participated in an acceptance testing of electronic devices that enables respondents to respond in public opinion surveys. So far, face to face or telephone survey modes have been used for data collection. With the development of digitization, most of the survey work has moved to the online space. As a result, respondents without internet access were left outside the reach of survey agencies. Additionally, the approach was tested as a response to the Covid-19 pandemic, which in several respects affected the implementation of the fieldwork so far implemented in the form of computer assisted personal interviews. The Electronic Questionnaire Device project (EQD) coordinated by NatCen (UK) tested electronic devices (tablets) and collected the feedback from respondents who have no experience with the use of such devices and do not use the Internet. The aim of this paper was to investigate and describe i) initial participants ́ reactions to the device; ii) issues related to device usage; iii) willingness of respondents to participate in self-completion surveys again. The findings of the study show up evidence for the overall acceptance of the tested approach. This approach aims to enable participation in a survey to those people who might not have been (approach)able to take part under other circumstances. In general, the participants were open to cooperation, and accepted the option of selfcompletion. While an acceptance of the device as a tool for the self-completion approach by respondents with low/uncompleted education remains questionable and uncertain, having some previous experience with ICT seemed to help in decreasing the initial stress and tension of the respondent to accept the self-completion option. These findings will, however, require further verification on larger samples of respondents.


Introduction
The emergence of the presented project was prompted by some specific circumstances. The first is the gradual transformation of social survey data collection methods, where face-to-face or telephone interviewing have dominated, into online research. The advantage of online interviewing is fast and cost-efficient data collection. In addition, online surveys do not require the employment of interviewers, and is flexible in time and space according to the needs of the respondent. Online research also reduces social desirability and produces more realistic data on sensitive topics, when the lack of interviewer presence can remove response bias (Dayan, 2007). Online research also allows for regularly re-interviewing the same respondents, thus enabling longitudinal research that investigates changes over time. The main disadvantage of online research is the limited representativeness. Representativeness of survey samples is one of the basic conditions of multi-national surveys such as the ESS. Only in this way can the results obtained be generalized to the as a whole, and then compared between countries or over time. This means that each individual in the target population has a non-zero probability of being selected (Mohler, 2007). The inequality of chance in being selected is then corrected by weighting. In online research, the risk in this context is the so-called offline population. These are people who do not have access to the Internet or do not have adequate ICT competences and were left outside the reach of research agencies. These are mostly older, less-educated or unemployed people who do not have ICT equipment for online research, or they have it, but they are not willing to participate in online research. The size of this group in individual countries also depends on the internet penetration. Overcoming these limitations are the probability based online panels that also include the originally offline population, but there are only a few in Europe (Blom et al., 2016). Nevertheless, they are a valuable source of information on recruiting strategies and offline population management. These respondents are usually retrofitted with electronic equipment and an Internet connection, as well as trained to complete questionnaires.
The second circumstance influencing the project was the Covid-19 pandemic, which in several respects affected the implementation of the fieldwork, which had been implemented in the form of computer assisted personal interviews (CAPI). Restrictions related to the pandemic disrupted the work of interviewers in the form of travel restrictions as well as their willingness to conduct face-to-face surveys due to the risk of infection (e.g., Sastry et al., 2020). Respondents were also expected to react negatively to the survey for the same reason. Experience from some countries during the pandemic suggests several alternative solutions, such as fieldwork interruption (Kurdija & Vovk, 2021) or change from face-to-face data collection to telephone interviewing . In complex surveys, telephone interviewing due to the absence of visual material is, practically, unfeasible, but can be replaced, for example, by video interviewing. However, this again runs into the limitations of online research. In this context, the possibility of changing the CAPI mode to CASI (computer assisted self-interviewing) using the Drop-off / pick-up (DOPU) method was considered as another alternative. The DOPU method is based on the hand-delivery of the questionnaire by the interviewer, its completion by the respondents, and subsequent collection by the interviewer at an agreed time. Although this solution does not exclude the personal contact of the interviewer with the respondent, it reduces the contact to the necessary minimum while maintaining methodological standards and pandemic measures. The DOPU method is based on the social exchange theory in which the personal interaction of the interviewer with the respondent stimulates exchange and reciprocity, which increases the chances that the respondent accepts and completes the survey (Dillman et al., 2014). At the same time, personal interaction allows respondents to address issues and concerns as well as provide feedback. As reported by Trentelman et al. (2016), DOPU allows one to choose the right respondent in the household, to establish a relationship with him/her, as well as to instruct him/her correctly in completing the questionnaire, and thus achieving a high response rate. The disadvantages include increased travel and interviewer's costs.

Electronic Questionnaire Device
This paper presents selected findings of the "Electronic Questionnaire Device" (EQD) acceptance testing project which was carried out by the Slovak ESS team for the European Social Survey European Research Infrastructure Consortium (ESS ERIC) and coordinated by NatCen. EQD is a tablet computer with no other use besides responding to a survey questionnaire that is uploaded on it. The aim of this project was to test the use of an electronic device for survey questionnaire self-completion by participants with no internet connection and without or very limited ICT skills. Taking part in the project involved two tasks. Firstly, completing a questionnaire on an electronic device. The tablet was dropped off at the participant´s home (DOPU method), and the participant's task was to attempt to complete the questionnaire in his/her own time and pace. Later the interviewer/fieldworker came and collected the device. Delivery and collection was done on the doorstep and the interviewer did not enter the respondent´s home. Secondly, once the respondent finished the questionnaire, he/she was asked to give feedback about their experiences of using the device in a short (10-20 minutes) telephone interview. The aim of this paper is to investigate and describe: i) the initial participants´ reactions to the device; ii) the issues related to device usage; and iii) the willingness of the respondents to participate in self completion surveys again.

Sample
The recruiting of the respondents was based on recommendations from colleagues and students to identify the target participants that meet two testing requirements.
1. People who have no internet access or only rarely use the internet (once a month or less often) 2. People who have no or limited experience using a tablet, computer, smartphone. Then a snowball method was used to identify similar respondents with an opt-in approach (explicit consent). An advance letter with the EQD study information was delivered in person to the identified participants. It also included information about the phone-call debriefing. Incentives (a 15-Euro voucher) were given to the participant on delivery of the device by the fieldworker.handed in by the fieldworker once delivering the device. Limited experience using a digital device 9 30%

Results
The findings described below are based on follow-up telephone interviews with participants after finishing the self-completion of the questionnaire on the device. Taking part in this interview was entirely voluntary and confidentiality was stressed by interviewers as a high priority. Interviews with each respondent lasted no more than twenty minutes.
Firstly, following the first aims of the paper, the initial reactions to the device were investigated by the question: How did you feel when you saw the device for the first time? Several categories were identified. Here is the list of them followed by selected respondent´s quotes.

Unknown thing:
For some participants the device/tablet was simply an unknown thing ("it" or "something"). "It was that I was going to work with something for the first time. It's something I don't work with." "What is it?" 2. Lack of previous experience with the device: The majority of participants mentioned lack any previous experience with the device. "I didn't know it (Tablet). It was completely new." "That I should start using the tablet, I do not use the Internet or the tablet, I leave it (browsing the Internet) to family members." "I've never done anything on a tablet before. But I have experience with using a mobile phone and a computer. I like working with technology." 3. Lack of previous experience as a source of stress: For some of participants the lack of experience was considered the source of stress, tension, uncertainty and/or doubts. "I was scared because I had no previous experience." She was afraid she would not understand what to do without any experience. "I don't know. I was wondering if I could do it." "I was worried that I wasn't technically proficient enough and that I wouldn't help you" One participant was afraid he/she could damage/break the device: "The device looked quite fragile, I was afraid I would break it, but I was pleasantly surprised."

Lack of previous experience as a source of curiosity:
For some of participants the lack of experience was considered the source of curiosity and excitement about new experience. Participant told the interviewer she wouldn't fill it out until the next day, but she was curious. Participant has no previous experience, but he wanted to try it "I don't have that experience yet, I was looking forward to it."

Learning and gaining new experience:
Some participants thought about learning something new. "I realized that I would have to learn to work with it" 6. Tablet as a gift: Some participants hoped that they could own the tablet once completing the questionnaire. "So at first I thought it would be mine, so I was glad to be able to learn something and then I was disappointed that it wasn't really mine, just borrowed." 7. Distrust of technology: Some participants expressed their dislike or distrust. "I don't believe these things." Participant considered it nonsense and didn't trust the tablet and survey.
Secondly, following the second aim of the paper, the respondents were interviewed about the issues they came across when working with the device. Only a few issues were reported when answering the following question: 1. Any issues with the text? Issues with the screen? "…it disturbed me that the screen went dim after a short time, I was still reading the question and it was getting darker, then I tapped the screen to light it up, but then it took me a while to find out where I had stopped reading..." Overall, only two participants reported the text too small and they would welcome a bigger size of letters.

Any issues with moving forward and back in the questionnaire?
More than issues, the process of learning how to tap was mentioned and participants were soon experienced enough (after responding to several questions) "At the beginning it was difficult to find out how to tap properly on the "next" and "back" buttons." "Sometimes I had to press the NEXT or BACK button several times until it worked." Overall, the participants did not report it as a problem: It wasn't a problem, even though the participant was working with the tablet for the first time.

It took a while for her to get used to it, because she didn't have that feeling in her fingers.
Thirdly, following the third aim of the paper, the willingness of the participants to use the device again to complete a survey was investigated. 23 participants out of 30 would agree to use a tablet as a survey tool in the future, while 5 participants said they didn´t know and 2 wouldn't use it. A selection of participants' responses follows: "Yes, I would. My wife laughed at me that after the research, I would finally start learning these things (ICT)." (male, 16-64) "Yes, I would like to return to the tablet, it is a chance to work with the tablet again, it attracts me, it is a new experience, it is more interesting. I have more time to respond, and I can create comfort doing it on my own, more privacy." (female, 65-69) For several respondents it turned out to be a preferred survey mode: "I prefer to work with a tablet rather than face a stranger (interviewer)." (female, 70+) "During Covid pandemics, I would prefer a tablet, but out of it (covid) I would prefer an interview." (female, 65-69)

Discussion
In relation to the acceptance of the device by participants outside of using the Internet, it is necessary to state their specifics in more detail. Recruitment of participants identified two groups: marginalized groups (Roma, homeless people, long-term unemployed) and seniors, or a combination of both. In the group of marginalized respondents, distrust prevailed in relation to the interviewers and the objectives of the project. In the group of seniors were concerns about their own ability to overcome the self-interviewing. Blom et al. (2016) states that older age cohorts are difficult to survey in any mode, and pose difficulties in self-completion modes, in which the respondents receive no help from an interviewer to read out question texts and answer options. For this reason, probability based panels chose an upper age bound to 70. Despite the fact that the offline population is declining every year, it is still a significant part of the population. According to Eurostat (2021), 10% of households with a population aged 16-74 do not have an internet connection in Slovakia. It can therefore be assumed that the number of households with an even older population without an internet connection is in fact higher. Face to face recruitment was therefore an adequate way to obtain respondents who needed to explain the broader context of social research. Personal contact was both an opportunity and a challenge to convince respondents that the use of electronic equipment is easy -which was the biggest obstacle. At the same time, the use of the DOPU method, in addition to establishing a relationship and convincing the respondent to participate in the research, created a slight pressure and a commitment to complete the questionnaire. This was reflected in the fact that almost all respondents who completed the questionnaire did so shortly after the interviewer's visit. DOPU reduces the likelihood of participants giving biased, socially desirable responses since the respondent can complete the questionnaire in private . There is also less opportunity for bias to be introduced by researchers with DOPU than with interviews. The benefits from personal interaction with respondents are similar to those seen with interviews, and DOPU provides more safety for interviewers since they do not have to enter the participants' homes. They also noted that some respondents are more likely to participate given the opportunity to complete a questionnaire at their leisure, rather than needing to take the time to be interviewed while the researcher is at the door. Overall, respondents accepted the electronic questionnaire device and their responses were generally positive.
No major problems with the use of EQD were noted during testing. Respondents emphasized the negative features of working with a tablet, e.g., tapping, screen lockout or typing on a virtual keyboard. However, these shortcomings can be addressed with more detailed instructions. These participants differ significantly from the general population. Differences, in addition to the digital divide, were mainly in the integration of these respondents into society. As a result, they did not understand many questions, what social research was, or were suspicious. The respondents in this group did not sufficiently understand the social context, e.g., migration issues, political overview such as European affairs, or did not watch the media. Facing questions they did not understand (too complex or/and lack of knowledge in certain topics) led several respondents to quit the questionnaire. At the same time, the low digital literacy of people over the age of 65 and their low willingness to learn in this area have been repeatedly confirmed (Velšic, 2020). A particular issue concerns the respondent's desire to keep the device. Due to the fact that several respondents expressed interest in the device, it is necessary to resolve the procedure of returning the device in case the participant does not agree. Although this has not happened, addressing this situation will require a sensitive approach (e.g. in the form of a record of delivery) so as not to discourage respondents from participating in the survey.
The findings of the study show up evidence of an overall acceptance of the tested approach. The approach aims to enable participation in a survey to those people who might not have been (approach)able to take part under other circumstances. In general, the participants were open to cooperation and accepted the option of self-completion, but the acceptance of the device as a tool for the self-completion approach by respondents with low/not completed education remains questionable and uncertain. Having some previous experience with ICT did seem to help in decreasing the initial stress and tension of the respondent to accept the self-completion option. The benefit of the study is to verify the use of electronic equipment in the offline population, which requires a sensitive approach and a more detailed briefing. The limitation of the study is the limited number of participants, as well as the criteria for their selection. These findings, therefore, will require a further verification on larger samples of respondents.