Challenges and acceptance of the use of computer-assisted personal interviews technology for verbal autopsy / social autopsy child mortality survey in urban slums of Karachi , Pakistan

Background: For collecting data regarding child mortality, the verbal autopsy/social autopsy (VASA) surveys involves lengthy questionnaires. Use of computer-assisted personal interview (CAPI) technique is now preferred over paper-and-pencil interviews (PAPI) due to several factors. This study identifies the level of acceptance of CAPI over PAPI among participants; challenges in implementing this technology; and the persuading factors for improving the acceptance of and getting quality data from CAPI based VASA surveys. Methodology: This qualitative study involved a total of 20 face-to-face VASA interviews conducted from mothers of deceased under-five children in selected urban slums of Karachi, followed by semi-structured focus group discussions (FGDs) and in-depth interviews (IDIs), enquiring about their acceptance towards CAPI and discussing the challenges faced during the use of PAPI versus CAPI. Total of 8 IDIs were conducted from different stakeholders for strengths, weaknesses, opportunities, and threats (SWOT) analysis on the CAPI implementation. Thematic analysis was conducted. Results: Out of 20 female respondents (median age 28.2 years and median education 5th grade); and 10 female interviewers (median age 30.1 years and median education of 11th grade), 78.1% and 83.7% respectively were mobile phone skilled in priori. The overall outcome showed a positive acceptance (90%) of CAPI by participants. Respondents and interviewers accepted CAPI over PAPI, while the technique had some challenges as identified by different stakeholders. Conclusion: Researchers pondering the use of CAPI for VASA surveys on child mortality must consider discussed factors in order to increase the participants’ acceptance level, maximizing the involvement of CAPI in child health surveys and to obtain a quality data, especially from developing countries.


Introduction
The global estimates on child mortality demonstrate that during the year 2016, almost 5.6 million under-five children died due to various reasons 1 . Despite a continued reduction in mortality rate and improvement in the trend of child survival, the newborn mortality remained a challenge among the developing countries 2 . After the Millennium Development Goal target, the Sustainable Development Goal era requires sustainable efforts in tackling the different challenges involved in improving child survival, especially in developing countries 3&4 .
The developed countries have well established and sustainable system of surveillance and data collection on maternal and child health indicators 5 . On the contrary, most of the developing countries, lacks fully functional national civil vital registration system (NCVRS) which results in a deficiency of reliable and authentic data on the estimates of child mortalities 6&18 . To identify the accurate and timely trend of child health and survival indicators, these developing countries need to undertake strong and sustainable efforts for collecting comprehensive and reliable data, especially through the surveillance and community surveys. Such data collection methods usually require door to door visits 6 involving lengthy and time-consuming questionnaires 7 . Many of such questionnaires involve technical complexities such as skiplogic patterns that need to be referred back on specific responses during data collection, which itself is a laborious task 8 . Dealing with huge complexities during the interviews results in data errors and inconsistencies that require interviewers to refer back to respondents for subsequent visits to get corrected. On multiple occasions, despite subsequent visits, the interviewing team does not get hold of respondent due to non-availability of respondents or due to multiple reasons. Such exercises consume precious time and energy of the data collection team.
In most of the developing countries, the research surveys are traditionally carried out through PAPI which takes most of the energy and concentration of the data collector during interview; however, in the recent era, there has been a shift of trend towards the use of CAPI 8,[9][10][11][12][13] . Due to several reasons, CAPI is recommended over the traditional PAPI technique 9 . The former has an edge over the later in having short interview timing; being less expensive; avoidance of missing data and duplication of identifiers; elimination of routing problem; checks for inadmissible responses etc 8 .
Evidence shows that the use of CAPI is increasingly being practiced in developing countries [10][11][12][13] , however, there is a mix level of acceptance of this technology among the field staff and respondents. The members of the research team (researcher, field staff, and data collectors) carries concerns which are technical in nature, however, they usually prefer to employ CAPI in their survey 9,[14][15][16] , while the survey respondents and community members have different sets of anxieties and apprehensions toward acceptance 12,13,15&17 . They usually do not have technical awareness and hence their apprehensions are of different nature 12,13,15&17 . These concerns impede the frequent and fluent use of this technology in usual surveys, especially among the underserved populations. Field insecurities (i.e. compromised law and order situations of the field), financial constraints, illegitimate use of devices and connectivity are the major issues that have been noticed as key concerns in low and middleincome countries, especially in their .
In-depth understanding of the key concerns is important in developing the best methodology for CAPI, allowing community surveys and surveillance system to utilize this technology ensuring the collection of superlative quality of data from the research without any hindrances and barriers 8&12 .
Pakistan is one of the high burdened child mortality countries that ranks among the top in relation to neonatal mortality 18 . Here, the community-based surveys identifying determinants of child health and mortalities and the usual child health surveillance programs traditionally consume paper-based system to record the data 19 . No local evidence exists showing the use of CAPI technology in child mortality surveys. A detailed assessment is, therefore, a requirement focusing the causes of acceptance (among the respondents and research team), their concerns, and SWOT analysis of the technique for utilizing it in child mortality surveys in Pakistan 8 .

VASA child mortality survey
Under the working panel of Health Advocacy Council for Women and Children (HAC-WC) along with Child Health Registry of Pakistan (CHRoP), Pakistan's first Integrated VASA child mortality survey is aiming to identify the prevalence of child mortality rate across the urban slums of Karachi, Pakistan. It also aims to identify and explore the in-depth understanding of different determinants linked with the child mortalities. The VASA study is a mix-method interventional study, involving both, the quantitative and qualitative components. The survey will interview mothers of all the deceased under-five children who died within one year before the data collection, using validated version of CHERG (Child Health Epidemiology Reference Group) integrated VASA child mortality questionnaire as computer-assisted personal interview application loaded on handheld devices (i.e. CAPI technology). This paper projects the results of a sub-study (CAPI survey) of VASA child mortality survey that focuses the level of acceptance of CAPI technology among respondents (mothers of deceased under-five children) along with the potential issues and concerns raised by the project team, respondents, and community members. Additionally, a detailed SWOT analysis of the technique has also been carried out by different stakeholders. The aim of this CAPI qualitative survey is to explore the challenges and determinants affecting the acceptance of CAPI in VASA surveys across the under-developed slums of Karachi city so that the best methodology can be developed ensuring the utilization of CAPI in child mortality surveys in Pakistan and to collect a quality data.

Methodology
The complete research activity of this survey was supervised by Principal investigator (MBS) i.e. the corresponding author of this paper, in collaboration with Health Advocacy Council for Women and Children (HAC-WC), and Child Health Registry of Pakistan (CHRoP). Software development and device availability was managed by HAC-WC. Trainings on data collectors' capacity to conduct interviews were undertaken by MBS in collaboration with HAC-WC. The involved field staff belonged to CHRoP and HAC-WC. The monitoring and evaluation was conducted by a separate team. Ethical approval for the survey was taken from Advanced Educational Institute and Research Centre (AEIRC) ethics committee. Written informed consents were taken from each participant, which were scanned and stored electronically. This qualitative study involved two phases, CAPI acceptability and SWOT analysis.

Phase 1 -CAPI acceptability:
A written consent was taken from all of the twenty (20) respondents. The first half of the CHERG integrated VASA child mortality questionnaire was administered to respondents using PAPI, and the second half was administered using CAPI. CAPI was conducted using the application loaded onto android phones. The duration of each PAPI/CAPI interview (involving both PAPI/CAPI), that occurred inside the respondent's household was variable (ranging from one hour to 5 hours), because the respondents (mothers of deceased children) had to look after other household activities as well during the interview. This was followed by FGDs conducted from all the respondents enquiring their experiences gathered from the PAPI/CAPI exposure; and their preferences with detailed reasoning towards PAPI versus CAPI. FGD sessions were conducted by a trained moderator and note-taker, using established interview guidelines. To explore a more in-depth understanding, IDIs were also conducted from respondents, interviewers and community members using interview guidelines. Separate interview guidelines were developed for interviewers and community members. Each FGD and IDI was carried out until the level of saturation was achieved and were conducted in a community room where all the participants gathered on a pre-agreed time. All interviews (FGDs and IDIs) were audio and video recorded for the ease of transcription and recording expressions of the respondents.

Phase 2 -SWOT analysis:
A total of 8 IDIs were undertaken from different stake-holders/key experts working for VASA survey, i.e. HAC-WC and CHRoP. The IDIs were conducted by the principal investigator (MBS) of this study. Each IDI was carried out until the level of saturation was achieved and were conducted in a community room where all the participants gathered on a pre-agreed time. Keeping in view of the expertise of the stakeholders, Interviewer guideline were developed and utilized for their IDIs. The questions embraced the issues on plotting and designing, methodology, training, problems confronted, data quality and suggestions. All IDIs were audio and video recorded for ease of transcription and recording expressions of the respondents. Based on the findings, the clear points were elaborated in the SWOT analysis (table-1).
With the help of interview notes, audio and video recordings; interviews were initially transcribed verbatim, later verified from notetakers and facilitators and lastly translated into English. Coding was done in NVivo 10 software. Major themes and sub-themes were generated. Thematic analysis was conducted.

Results
All of the 20 female respondents i.e. participants have their median age of 28.2 years (interquartile range: 27.1, 29.4) and median education level of 5 th grade (interquartile range: 5.0, 5.75). Virtually, 78.1% of respondents and 83.7% of interviewers respectively were mobile phone skilled in priori.

CAPI acceptability
Out of twenty (20), a total of eighteen (18) respondents showed acceptability for CAPI survey and only two (02) showed preference of PAPI (90%). All the interviewers (100%), and 80% of community members expressed acceptability. All the interviews were scheduled according to the appointment given by respondents. After the FGDs from interviewers and respondents; and IDs were conducted from respondents, interviewers, and community members to explore their concerns and benefits from CAPI implementation. Following themes were identified as a major factor regarding CAPI acceptability: 'New concept'; 'cost-cutting'; 'security'; 'data security', and 'data accuracy'. The interviewers felt that the technique is a major threat as perpetrators know that the data collectors will not leave the site before the completion of the interview. Insurance is again of no use in such situations, because it does not help in recovering data after the device snatching. There should be some security personnel with the field team.
'Insurance does not help in such circumstances, as the recorded data will get lost and you cannot complete the data collection with satisfaction. How many days will you take to complete interviews, they know that this team has mobile phones, and they will surely come again for snatching until you have security measures/personals with you'.….

d) Data security:
The Respondents and community members felt that data is stored in the device very securely.
'The data is always secure in the device'. [Respondent] It is the most secure place for the data, however, the device should be kept secure'.
[Community member] The interviewers felt that until unless the device is not snatched, the data security is maintained. The data is securely transferred from one location to another without any issue.
'The data is transferred securely from one place to another, however, the element of snatching does exist. In this case, the data security is always at risk'.

SWOT analysis
The IDIs provided four of the following themes which are mentioned in table 1.

a)
Staff and training issues: Some of the interviewers rather than following the provided guidelines and training, used their own techniques and skills while interviewing. For example, rather than using already provided specific phrases and words to be used in certain situations during interviews, they used their own set of words. This resulted in a communication gap with the respondents in a certain situation.

b)
Quality of data issues: Sometimes, when the answer of the questions is not available among the list of responses on CAPI device, the interviewer only selects the available options rather than getting touch with the field supervisor, this result in compromised data quality and loss of actual facts. Sometimes, data collectors spend more time on gazing and staring on the android device rather than talking with interviewees. These points were raised by some stakeholders.
'Sometimes, data collector stares on the device and sometimes they select the option which is available on the software rather than selecting the most appropriate option required'. [Field supervisor]

c)
Technical issues: Some of the field staff reported failures of involved gadgets at different point of times.
'The devices went hang up and their software went stalled during the data collection. This made the data collector stay on the household and wait for the device to return to normal functioning-luckily there was technical staff to solve these issues'. [IT expert] Internet connectivity was another a major problem in areas where the signal strength gets weakened. Sometimes the internet signals went lost and sometimes, the data package ended up without proper notification from the data provider.
'Despite the data collection occurring in the big city, the signals of the internet gave us trouble throughout the field activity. What will happen when we will conduct data collection rural areas of Sindh or Baluchistan'. [Interviewer]

d)
Methodology issues: The use of electronic devices and applications developed for data collection in the VASA child mortality survey was highly acknowledged by different stakeholders. They raised the point of data security, short interviews, easiness and low cost compared to PAPI. Lengthy interview times were one of the major concerns, interviewer, technical staff and field supervisors pointed out.
'The process is simple, less costly, quicker need less training for android competent people'.
[Field supervisor] 'There should be an appointment-based interview time for collecting data on PAPI/CAPI interviews, as we had to wait longer for completing interviews from some of the respondents'.
[Interviewer] Overall experience of CAPI: The stakeholders mentioned key issues during the field activity that need to be resolved for proper and fluent field activity. Issues of network connectivity on the android devices were reported by 59% of the field staff. Hanging and stalling of the device were reported by 43% of the field staff. Staff communication problem was reported by the field supervisors. Out of 20 PAPI/CAPI interviews, three (05) i.e. 15% of the interviews were delayed.

Discussion
This is the study done so far in Pakistan identifying the acceptability; strengths/weaknesses/opportunities/threa ts; and concerns of the different stakeholders of the VASA child mortality survey. Our study has been found to be consistent with several surveys 10-12,16&20 , however, none of these studies have mentioned any concern and anxieties towards the use of CAPI. Our study found a high level of acceptance of the use of CAPI among the project team members as well as moderate acceptance among the respondents and community. This outcome is consistent with some studies conducted in the past 10,11,16,20&21 , however, only a few studies showed strong resistance towards the use of technology 22&23 .
Our study results clearly focuses on the issues during the field activity of a child mortality CAPI survey ( Table 1). One of our identified themes was 'new concept' which was clearly discussed in one study 12 . Respondents have always been pessimistic towards the advanced use of technology, and are very contented with these developments, especially in the field of medicine and health sciences 16,23&24 . Our study also noted this behavior of the respondent, however, one Kenyan study reports that the respondents were ridiculed after they saw an electronic device in the field for data collection 22 . This actually depends upon the awareness, knowledge and past experience of the community members regarding the technology in priori. Since the Kenyan study was undertaken in 2003, when the population might not be that much aware of the technologies that they are currently, we can imagine such contrasting results.
Data confidentiality and security was well discussed in some of the studies 9,23&25 . Their results are consistent with our study 9,23&25 , however, Angolan study showed a higher prevalence of awareness towards technology than men 20,23&26 . However, this scenario may change with different socio-cultural and socio-economic settings.
Training and monitoring on how to use the devices, overcoming technical issues while on the field etc. are quite important and very necessary for any survey to consider 27 .
Our study holds a great strength that we gave a thorough training to the field staff, although most of them had a good knowledge in priori on how to use the technology (Table 1).
Another strength of the present study was that a well-developed system was used to get hold of error check reports on a daily basis (Table 1). This helped us to undertake measures to minimize errors while on fields as well as on a day to day basis by arranging continuous quality control feedback sessions. We also had the arrangement to obtain quality control reports on a daily basis. This helped us to explore the quality of data on a frequent basis. Our methodology is consistent with another study from Indonesia 28 .
Maintaining eye to eye contact with the respondents during the interview is quite crucial in surveys especially when we are dealing with the mothers of deceased children. These mothers could go in a state of emotional trauma during the interview. The data collectors in the study were trained on how to improve this element during the interview. We also took specific measures to prevent such events, which are discussed in another paper in detail 18 29 .

Limitations
One of the limitations of the present study is that it doesn't involve any measures to prevent the unfavorable event, especially the snatching of the device during the compromised law and order situations or even during the normal circumstances. Such events are destined to occur in any secure or unsecure place, especially in developing countries. The team could be accompanied by the security officials, however, such interventions may have different consequences as well. We definitely feel that this issue need to be addressed with special interest so that the issue of data security can be addressed, ensuring more involvement of such technology in the usual surveys to get a quality data. Additionally, some interviews took longer duration (approximately 5 hours), despite the interviews were fixed on the time provided by the respondents. This was because the mothers (respondents) were busy in dealing with their children and in household activities. A separate interview place and an already agreed scheduled time (just like we did for FGDs and IDIs) where the respondent can come and interviewed could be a better idea to avoid delays.

Conclusion
This study is first to describe the level of acceptance and challenges involved in the implementation of CAPI technology specifically for child health surveys in the urban slums of Karachi, Pakistan. The discussed factors must be considered by the researchers pondering over the use of CAPI for VASA surveys on child mortality in Pakistan.
Implications for future research More research is needed to identify the challenges (strengths, weakness, opportunity, and threats) in relation to the field, technology, field security, maintaining data accuracy, new innovations etc. across other developing countries. Additionally, there should be standardized CAPI implementation guidelines, especially in developing countries for undertaking such technology in the usual child mortality surveys.

Conflicts of Interest
None.