Usability assessment of a mobile app for patients with peripherally inserted central catheters

Abstract Objective: to evaluate usability of the Meu PICC (My PICC) app for follow-up of outpatients using peripherally inserted central catheters through the validated System Usability Scale instrument. Method: a cross-sectional study that applied the System Usability Scale to 30 patients using peripherally inserted central catheters, ten nurses and eight Information and Communication Technology professionals to assess usability of the app. Results: a statistical difference was observed between age and usability (p=0.006), as well as a negative correlation between app use time and usability (p=0.002). As per the System Usability Scale adjectival classification, 40.0% and 33.3% of the patients considered the app as the best possible to be imagined and as excellent, respectively. In relation to the nurses, 70.0% considered the app as the best possible to be imagined and 20.0% as excellent; of the Information and Communication Technology professionals, 50.0% considered the app as the best possible to be imagined and the other 50.0%, as excellent. Conclusion: the usability assessment showed that patients, nurses and ICT professionals considered the app useful for monitoring patients using PICCs and evaluated it as appropriate, evaluating it as the best possible to be imagined or as excellent. These results corroborate use of the Meu PICC app in the monitoring of outpatient use of PICCs.


Introduction
Use of Peripherally Inserted Central Catheters (PICCs) has been expanded to different care areas due to the lower risk of complications during insertion, when compared to centrally inserted central catheters, to good durability for medium to long-term Infusion Therapy (IT) and to the possibility of outpatient treatment, with an improvement in the patient's quality of life and optimization of hospital resources and beds (1) .
For outpatient IT, the eligible patients are those clinically stable, with good adherence to the treatment, and conditions in phases with low risk of complications.
The main indications are treatments with antimicrobials, chemotherapy drugs, hydration, parenteral nutrition and analgesics (2)(3)(4) . In order to ensure IT continuity, it is important to incessantly seek to mitigate the complications. In the outpatient environment, these measures should involve education of patients and caregivers regarding purpose of the treatment, type and duration of the therapy prescribed, risks and benefits involved, possible adverse effects, potential complications of the intravenous device, care plan, and information on how to access the health service (2)(3)(4) .
Education in health is related to "health literacy", a term used in the literature to designate the extent to which individuals have the ability to find, understand and use diverse information and services to communicate healthrelated decisions and actions. "Health literacy" is considered a complex phenomenon that involves individuals, families, communities and systems. Its concept covers the materials developed for patient education, the environments involved, and the challenges specifically associated with health conditions and treatment and prevention measures. In addition to that, it involves a high number of skills such as reading, understanding and analyzing information, following instructions, performing calculations, decoding symbols and interpreting graphs and diagrams (5) .
Patients with a low level of "health literacy" are more likely to have unfavorable health outcomes, as it is associated with lower adoption of preventive behaviors, lower adherence to treatments and more frequent hospitalizations (6) . In addition to "health literacy", the Infusion Nurses Society (INS) recommends that factors such as age, degree of cognitive development, access to resources and technologies, the patient's preferences and possible cultural influences are also considered during development of each patient's health education strategy (4) .
To enable education in health and outpatient follow-up, the use of Information and Communication Technology (ICT) by health professionals has expanded worldwide. As using these technologies promotes the dialogic and participatory education in health advocated by Paulo Freire, a process of awareness raising of patients beyond the hospital environment is initiated, through the co-construction of knowledge from their own contexts and demands at home (7) . Thus, the number of studies that address different types of interactions with patients with the objective of guiding, monitoring and encouraging adherence to the treatment of chronic, contagious and psychiatric diseases has increased (8)(9)(10) .
There is still a lack of studies examining the results related to central vascular access devices outpatient care and development of complications (11) . To date, a meta-analysis of 36 Chinese studies with 2,623 controls and 2,662 patients who used the WeChat app for monitoring outpatients with PICC showed that the group that underwent follow-up through the app presented a lower risk of PICC-related complications with an odds ratio (OR) of 0.23 (p<0.00001), better self-care ability with a mean difference of 36.4 (p<0.00001), greater dependence on PICC maintenance with an OR of 4.27 (p<0.00001) and greater patient satisfaction with an OR of 6.20 (p<0.00001) (12) .
At the hospital institution of this study, at the time of discharge and according to the institutional protocol, the patients receive guidelines for safe maintenance of the PICCs in the outpatient environment, in order to preserve integrity of the devices and identify complications. As they are central vascular access devices, with rapid capacity for systemic deterioration in case of complications and demand for specific care, it was deemed necessary to develop a smartphone app to allow patients to access these guidelines and ease communication with the nurse of the institution at any time and place.
Prior to this study, the PICC patient discharge protocol in force at the research institution was reviewed in accordance with the standardized practices for the use of IT (4) , the infection control measures related to invasive venous devices set forth by the International Nosocomial Infection Control Consortium (INICC) (13) and the Brazilian National Health Surveillance Agency (14) , and the guidelines by the Hospital Infection Control Committee (Comissão de Controle de Infecção Hospitalar, CCIH) and the Infusion Therapy Group of the institution itself, subsequently validated by experts and patients using PICCs. After obtaining this updated content, the Meu PICC app for smartphones was developed for the monitoring of outpatients in use of PICCs.
It is important that the development of apps goes through a usability assessment, as it is considered a product quality attribute that involves five components, namely: ease of learning, efficiency, ease of memorization, minimization of errors, and satisfaction (15)

Study type and period
A cross-sectional study was conducted between September 2020 and January 2021.

Study locus
The study was carried out in inpatient units and in the day hospital of a tertiary-level health care teaching hospital

Population
The study had three different groups of participants to enable usability assessment of the app, from different points of view: patients with PICCs for IT; nurses and ICT professionals from the same institution.

Inclusion criteria
The patients should use PICCs, have the Public Health System as a provider, be over 18 years of age, and state mastery in the use of a smartphone with a camera and

Exclusion criteria
The patients excluded were those with cognitive and spatial-temporal alterations or psychiatric diseases without a caregiver or not in due clinical conditions for smartphone manipulation, according to data available in the patient's Electronic Health Record (EHR).

Sample
The sample was intentional and non-probabilistic and recruitment occurred according to the availability of the participants present in the institution during the data collection period and the COVID-19 pandemic.
One of the most renown experts in software usability considers a sample of five users as enough to identify usability problems (15) , in contrast to NBR ISO/IEC 14598-6 (16) , which suggests a minimum of eight participants for consistent results. Thus, for this study, we included 30 patients using PICCs at the time of data collection, ten nurses with experience in the care of outpatients with PICCs and eight ICT professionals.

Study variables
To assess usability of the Meu PICC app, the System Usability Scale instrument validated for Portuguese (17) was used, consisting of 10 items and five-point Likerttype answers that ranged from "I totally disagree" to "I totally agree". The structure of the questions alternates between positive aspects (odd questions) and negative aspects (even questions), in order to promote reflection of the answers (17)(18) . Although it is a unidimensional instrument, factor analysis of the SUS tool shows two factors; usability (questions 1, 2, 3, 5, 6, 7, 8 and 9) and learning (questions 4 and 10). The questions addressed aspects related to use frequency, ease, need for support for handling, functionalities and trust in the app. For the current study, at the end of the assessment two nonmandatory essay questions were added, addressing possible positive and weak points of the app.
Specific instruments were prepared for each group.
With regard to the patients, the following information was collected: gender, age, schooling, main diagnosis, transportation means from the residence to the hospital, databases. In addition to that, it has an encryption system for in-transit and at-rest data, as well as an NBR ISO/ IEC 27001:2013 certificate for data safety (19) and an NBR ISO/IEC 27017 certificate for data safety in cloud services (20) , as stated on its platform (21) . The Meu PICC app is registered at the National Institute of Industrial Property

Data collection procedure
To assess usability, all the participants were approached at the study institution, submitted to the same app tests, as explained below, and answered the SUS instrument. Initially, the study and the app objectives were presented to all three groups of participants.
Subsequently, the participants from all groups were asked to use the app through a test user Login (Access)

The Meu PICC app for smartphones
The Meu PICC app was developed with the Scrum methodology, Ionic Angular programming language and Django server, directed to Android and iOS platforms.
It has six interactive screens containing the guidelines on how to take care of the PICC, frequently asked questions and the possibility of sending messages and a photograph of the device to the nurse. Figure   How would you do that?".
After this activity, the SUS scale was applied and there was no interference by the researcher in handling of the app. Figure 2 presents the step-by-step process of this activity.

Data analysis and treatment
The data were introduced and stored in forms developed in the REDCap (Research Electronic Data Capture) (22)  Pearson's correlation coefficient was used to verify the existence of a relationship between the SUS score and the "age", "PICC use time" and "app handling time" variables. The same correlation coefficient was also used to analyze if there was any relationship between age and app handling time. Kendall's coefficient was employed to verify if there was any correlation between usability and the "schooling" and "app handling time" variables.
The Wilcoxon-Mann-Whitney test was used to verify the existence of a relationship between the usability score and the "gender" and "previous experience with PICCs" variables; as well as the relationship between "gender" and "app handling time", "PICC use time" and "safety to be discharged with the PICC using the app" variables.
Presence or not of an association between the usability score and the subgroups of participants was determined by means of the Kruskal-Wallis test. The existence of a relationship between feeling safe to be discharged with the PICC using the app (dichotomous variable) and age was analyzed by means the Student's t-test, while for the "gender" and "previous experience with PICCs" variables, the Chi-square association test was used and, for "schooling", Fisher's exact test. The statistical significance level adopted was 5.0%.
To obtain the final SUS score, the following calculations were performed: for the odd answers on the Likert scale (that is, 1, 3 and 5), 1 was subtracted from the score indicated by the user, and 5 was subtracted for the even answers (2 and 4). To calculate the total score, the values obtained from the calculations for even and odd answers were added up and multiplied by 2.5. The overall usability score as per the SUS instrument can vary between 0 and 100 points (17) .
For this study, 70.0% of favorable items was considered as an acceptable parameter, with the sum of the two maximum categories for the odd questions ("I totally agree" and "I agree") and the sum of the two minimum categories for the even questions ("I totally disagree" and "I disagree"). After calculation of the final score, the app was classified according to the SUS adjectival classification scale, as follows: <20.5 -the worst possible to be imagined; 21-38.5 -deficient; 39-52,5 -average; 53.73.5 -good; 74-85.5 -excellent; 86-100 -the best possible to be imagined (23) ("imagined" understood as expectation).

Ethical aspects
The study met the recommendations set forth in

Characterization of the PICC insertion procedure
The PICCs were preferably inserted in the basilic vein (n=24; 80.0%), followed by the brachial vein in fewer cases (n=5; 16.7%) and, for one procedure, the

Characterization of the nurses and ICT professionals
All the nurses were female, with a mean age of

Usability assessment
The mean SUS score obtained by the patients was  In the comparison of the SUS with the patients' demographic variables, a statistical difference was only observed for age (p=0.006), with a reduction in usability as age increased, but not for gender (p=0.074) or schooling (p=0.892). There was also a negative correlation between app use time and the score in the usability assessment (p=0.002), that is, the greater the demand for app use time, the lower the usability score.
Considering the total answers obtained from the SUS items, it was observed that, of the total responses provided by the patients (n=300), 7.3% (n=22) evaluated some characteristic of the app in a negative way. As for the total answers provided by the nurses (n=80), this rate corresponded to 3.7% (n=3), whereas this was not observed in relation to the total answers provided by the ICT professionals (n=80).
It was observed that the percentage of positive answers was below 70.0% on two occasions: in question seven by the patients, which deals with use of the app by most people (66.6% agreement), and in question one by the ICT professionals, which deals with wanting to use the app frequently (62.5% agreement).
As for the SUS adjectival classification, 40.0% of the patients considered the app as the best possible to be imagined and 33.3%, as excellent; 70% of the nurses considered it as the best possible to be imagined and 20.0%, as excellent; and 50.0% of the ICT professionals considered it as the best possible to be and the other 50.0%, as excellent (Table 2). Table 2  In the essay questions, some participants reported positive points in a generic way or related to practicality and usefulness aspects of the app. Suggestions for improvements related to the diverse information and guidelines or to their location or to the app layout were also presented.

Discussion
In the current study, it was observed that more than half of the sample had already used PICCs, which is mainly due to the high number of patients with chronic diseases, who may present frequent acute cardiac decompensation, requiring readmissions and new therapy strategies. Nonadherence to the treatment and presence of infections and kidney failure are some of the main reasons for decompensation (24) . Given the above, it was verified that the type of IT most frequently found in the sample was for the administration of antimicrobials and vasoactive drugs. Simultaneous administration of these medications is not uncommon; therefore, INS recommends IT planning, allowing choice of the appropriate device (4) , justifying the use of double-lumen PICCs for one third of the sample.
It is also important to highlight that, as this is a reference institution in cardiopneumology care, it provides assistance to patients from different regions of the country. female (25) . Also in relation to the same research, there was a slight divergence regarding the mean age, which was higher in the current study when compared to the national mean value, which can be attributed to the profile of nurses with longer experience in the institution.
Regarding the ICT professionals, there was predominance of males and greater variation in the training level, which can be due to their lower mean age when compared to the nurses.
In relation the health education process, it is estimated that between 40.0% and 80.0% of the information provided by the health professionals is immediately forgotten by the patients and, in addition to "health literacy", different factors can be related, such as: use of technical terminology by the professionals, the patient's memory deficit caused by age, anxiety and distress caused by the context, perceived importance of the information, and the way in which the information is provided (4)(5) .
In addition to that, it is important to emphasize dexterity in the execution of subtle movements such as dragging or clicking on small objects or buttons (26) .
Over the years, these and other skills undergo changes and may exert an impact on the acceptance and handling of new technologies, which could be observed in this study, where lower scores obtained in the SUS were noticed among older age groups. However, a metaanalysis carried out with 144 studies that evaluated how chronological age is related to acceptance of the use of technologies, observed that the negative effects of age were evident only for technologies considered less useful and with less perceived ease of use (26) .
Only patients who had access to the Internet and When the ICT professionals assigned a lower score to the item of the instrument that points out that the user wants to use the app frequently, it is possible that they interpreted the statement considering use of the app to report complications related to the PICCs. As the Meu PICC application is a device to support outpatient PICC use, the fact that it will not be accessed frequently does not disqualify it, if the device remains intact throughout the treatment and the patient is free from complications. In addition, it can mean that the patient education process for discharge was successful.
Some patients required more time to know the app and learn its functions, a fact that was reflected in a lower score in the usability assessment. Despite the positive comments about practicality of the app, there is a need for greater attention in the learning process of those patients who are less familiar with the technology. However, in general, through the mean score obtained in the SUS, it was possible to verify that the app was mostly evaluated as the best possible to be imagined or as excellent in the adjectival classification.
In the literature, studies were found that reported the

Conclusion
The usability assessment of the Meu PICC app through the System Usability Scale presented a mean score of 82.7 among the patients, 89.2 among the nurses and 85.6 among the Information and Communication Technology professionals, respectively. The participants rated the app as the best possible to be imagined or as excellent, based on the adjectival classification of the scale employed in the usability assessment.
The results of this assessment generated subsidies for improving the app before proceeding with the comparative intervention study with the app and the routine practice of guiding patients with PICCs in outpatient care.