Process Evaluation of a Wireless Wearable Continuous Vital Signs Monitoring Intervention in 2 General Hospital Wards: Mixed Methods Study

Background Continuous monitoring of vital signs (CMVS) using wearable wireless sensors is increasingly available to patients in general wards and can improve outcomes and reduce nurse workload. To assess the potential impact of such systems, successful implementation is important. We developed a CMVS intervention and implementation strategy and evaluated its success in 2 general wards. Objective We aimed to assess and compare intervention fidelity in 2 wards (internal medicine and general surgery) of a large teaching hospital. Methods A mixed methods sequential explanatory design was used. After thorough training and preparation, CMVS was implemented—in parallel with the standard intermittent manual measurements—and executed for 6 months in each ward. Heart rate and respiratory rate were measured using a chest-worn wearable sensor, and vital sign trends were visualized on a digital platform. Trends were routinely assessed and reported each nursing shift without automated alarms. The primary outcome was intervention fidelity, defined as the proportion of written reports and related nurse activities in case of deviating trends comparing early (months 1-2), mid- (months 3-4), and late (months 5-6) implementation periods. Explanatory interviews with nurses were conducted. Results The implementation strategy was executed as planned. A total of 358 patients were included, resulting in 45,113 monitored hours during 6142 nurse shifts. In total, 10.3% (37/358) of the sensors were replaced prematurely because of technical failure. Mean intervention fidelity was 70.7% (SD 20.4%) and higher in the surgical ward (73.6%, SD 18.1% vs 64.1%, SD 23.7%; P<.001). Fidelity decreased over the implementation period in the internal medicine ward (76%, 57%, and 48% at early, mid-, and late implementation, respectively; P<.001) but not significantly in the surgical ward (76% at early implementation vs 74% at midimplementation [P=.56] vs 70.7% at late implementation [P=.07]). No nursing activities were needed based on vital sign trends for 68.7% (246/358) of the patients. In 174 reports of 31.3% (112/358) of the patients, observed deviating trends led to 101 additional bedside assessments of patients and 73 consultations by physicians. The main themes that emerged during interviews (n=21) included the relative priority of CMVS in nurse work, the importance of nursing assessment, the relatively limited perceived benefits for patient care, and experienced mediocre usability of the technology. Conclusions We successfully implemented a system for CMVS at scale in 2 hospital wards, but our results show that intervention fidelity decreased over time, more in the internal medicine ward than in the surgical ward. This decrease appeared to depend on multiple ward-specific factors. Nurses’ perceptions regarding the value and benefits of the intervention varied. Implications for optimal implementation of CMVS include engaging nurses early, seamless integration into electronic health records, and sophisticated decision support tools for vital sign trend interpretation.


Background
Most adverse events occurring in hospital wards are preceded by a considerable period of changes in vital signs, which are important indicators of clinical deterioration [1]. Monitoring vital signs allows for early detection and timely interventions that may improve outcomes [1][2][3][4]. In high-care units with patients who are critically ill, continuous monitoring of vital signs (CMVS) is the norm, whereas in general wards, vital signs are usually monitored intermittently, and interpretation is guided by Early Warning Scores (EWSs) [5][6][7]. Although the EWS system may facilitate early detection, there are still limitations owing to its intermittent nature and variable compliance [8][9][10]. Consequently, patients' condition may deteriorate unnoticed, which can lead to avoidable adverse events, adverse outcomes, and higher costs [11][12][13].
Given recent technological developments, CMVS using wearable wireless sensors has become available to patients in general wards. Previous studies have shown that these systems can accurately measure vital signs and detect deterioration [14][15][16]. However, evidence on the effects of these CMVS systems on patient outcomes is scarce [17,18]. This may be related to the fact that the implementation of CMVS at scale remains challenging and requires considerable upfront financial investment by hospital administrations [19,20].
Although many health care professionals acknowledge the potential benefits of CMVS for patient care, several studies have highlighted considerable challenges, such as difficult implementation into existing nursing workflows, poor integration with hospital electronic health record (EHR) systems, and primitive alarm management strategies [21][22][23]. In addition, monitoring vital sign trends may be challenging for most ward nurses because of a lack of experience with interpreting graphic representations of CMVS trends [15,24,25].

Objectives
Given these challenges, implementing CMVS in hospital wards is considered a "complex intervention" with many interacting components and the need for behavior change of health care professionals and affecting multiple patient outcomes [26,27]. Successfully scaled implementation in wards is necessary before any possible beneficial effects of CMVS on clinical outcomes can be expected [17,21,28]. Unfortunately, there is only scant knowledge on the facilitators and barriers to CMVS implementation [24,25,29]. We have previously conducted 2 feasibility studies [15,23] and 2 qualitative studies [24,30] that aided in developing and refining our CMVS intervention and an implementation strategy. For this study, an implementation-effectiveness hybrid design was used for the parallel evaluation of the implementation and effectiveness of the intervention [31]. This report focuses on the process evaluation of the implementation, with the primary aim of assessing and comparing intervention fidelity in 2 wards (internal medicine and general surgery). The secondary aims were to assess and compare implementation fidelity, technical fidelity, perceived appropriateness, acceptability, usability, adoption, and feasibility according to nurses. The effectiveness of the intervention will be analyzed and described in a separate paper.

Design
A mixed methods sequential explanatory design [32] was used for an 8-month period in a surgical ward and an internal medicine ward (September 2021-July 2022) of a 1245-bed tertiary teaching hospital in the Netherlands. The study was reported according to the Standards for Quality Improvement Reporting Excellence 2.0 checklist [33].

Context
The surgical ward consisted of 49 beds, in which patients of gastrointestinal and vascular surgery were admitted. A total of 57.4 full-time equivalent of nurses were employed at the ward. The nurse-to-patient ratio was 1:5 for the day shift, 1:6 for the evening shift, and 1:12 for the night shift. A nurse specialist or junior resident assessed the patients daily during morning rounds.
The internal medicine ward consisted of 48 beds and was divided into 2 teams of nurses based on subspecialties: general internal medicine and gastroenterology. The nurse-to-patient ratio was 1:4 for the day shift, 1:12 for the evening shift, and 1:12 for the night shift. A junior resident assessed the patients during morning rounds.
Patients admitted to the surgical and internal medicine wards were eligible to receive the CMVS intervention (Multimedia Appendix 1). Inclusion criteria were age of ≥18 years, no cognitive impairments, expected hospitalization time of ≥2 days, and ability to speak and read the Dutch language. The exclusion criterion was inability to wear the CMVS sensor because of an allergy. Nurses who were employed at the ward during the study period participated and were eligible for participation in the process evaluation if they had worked with the CMVS system for at least one month during the study period. Nurses temporarily employed from the flex pool were excluded from the study.

Intervention
In addition to standard care, patients included in the study were monitored using the Conformité Européene-marked Healthdot sensor (Philips Healthcare) and IntelliVue Guardian Solution (IGS) software platform (Philips Healthcare). Standard care consisted of intermittent monitoring (at least once daily) through manual measurements performed by the nurse and assessed using the Modified EWS (MEWS) according to the local hospital protocol [34].
The wireless wearable sensor was a water-resistant disposable patch worn on the patient's chest (Multimedia Appendix 2); it continuously records the heart rate (HR) in beats per minute and respiratory rate (ReR) in respirations per minute both using accelerometry. Previous studies have shown that this sensor is accurate [35,36]. The 2 vital sign measurements are transmitted wirelessly every 5 minutes through a long-range, low-power Internet of Things connection (LoRa; Semtech) to the IGS software. After connecting the sensor to the patient and to the software by scanning the QR code using a separate mobile phone, the software platform with trends was displayed on the computer on wheels and in a mobile app (Multimedia Appendix 3). The battery life of the patch was 14 days, and during the performance of an electrocardiogram, computer tomography, or magnetic resonance imaging, the sensor was temporarily removed.
Within the IGS software, individual vital sign trends were visualized, and complementary to the hospital MEWS protocol, a partial MEWS (D-EWS) was presented every hour to promote adequate detection. The D-EWS was based on the HR and ReR measurements and was in line with the MEWS thresholds and scores (Multimedia Appendix 4) on the preinstalled thresholds for HR and ReR. Patient numbers and names were automatically synchronized with the EHR using a Health Level 7 linkage, so manual entry was not required. As the device measures only 2 vital signs, routine manual measurements of other relevant vital signs (eg, temperature and blood oxygen saturation) by nurses were maintained throughout the study. Every 4 hours (ie, twice per shift), nurses routinely assessed vital sign trends and reported the D-EWS and any deviations and subsequent nursing activities in the EHR at the end of every shift. When the D-EWS was ≥3, additional checks and interventions could be performed as deemed appropriate by the nurse. No alarm strategy was applied in this study based on the substantial alarm fatigue experienced by nurses in our previous feasibility studies [15,23].

Implementation Strategy
Before the start of the study, the 2 wards were technically prepared for CMVS, and an e-learning module was developed (Textbox 1). This comprehensive 30-minute e-learning module (Articulate 360; Articulate Global) was developed by the project manager (JL) together with an educationalist. The e-learning module ended with a knowledge assessment that had to be successfully completed. The contents were pilot-tested by 4 nurses of the project team (Multimedia Appendix 5). This project team was formed per ward comprising the project manager, 4 nurses, the ward manager, and 1 consulting physician. First, information and goals about the project were presented in a regular team meeting 2 months before the start of the project. Subsequently, the e-learning module for nurses was made available on the web (Multimedia Appendix 5). It consisted of information about the purpose of the project, the rationale for CMVS, the protocol of the D-EWS, the work processes and policy, the practical use of the IGS system, and how to assess the vital sign trends. Afterward, there was a week of daily meetings with the project manager to provide ample opportunities to ask further questions. In addition, all relevant physicians were informed of the project and workflow in a team meeting.
During the first 4 weeks of the study, bedside training and coaching were provided by the project manager (Monday-Friday) 3 times daily. In addition, weekly status updates and feedback on the implementation were provided to the entire team via email. During the study period, the project manager coordinated the inclusion process (Monday-Friday). A small number of dedicated project team nurses acted as key users to provide support for all nurses.
To accurately monitor the implementation, the use of performance feedback was deemed essential. Each month of the study-as a structured evaluation moment-a dashboard with interim results of the inclusion rate and intervention fidelity was discussed in a project team meeting. In addition, a patient case study with deviating vital sign trends was presented, and CMVS experiences were discussed. Subsequently, actions were defined according to the Plan Do Check Act cycle [37], resulting in an iterative process of improvement of the implementation strategy. The results of the meeting, including the dashboard and related actions, were communicated to all team members via email. In addition, every 100th patient with CMVS was celebrated as an inclusion milestone in the team meeting.

Study Procedures
Admitted patients who met the inclusion criteria were approached and received information about the study. Patients of surgery were asked for informed consent during the preadmission call, and patients of internal medicine were asked when admitted to the ward. When the patients agreed to participate, the nurse started CMVS directly or immediately after the surgical procedure until discharge.

Sample Size
The study sample size was based on the primary aim of the project: evaluation of the implementation.
There is insufficient guidance in the literature regarding sample size calculations for this type of implementation evaluation studies. On the basis of historical data and the recruitment rate of our previous feasibility studies [15,23], we estimated that we would be able to include 350 patients across both wards over a period of 6 months.

Ethical Considerations and Informed Consent
The Medical Ethics Committee of the Isala Hospital reviewed the protocol (protocol 210414) and declared that the Medical Research Involving Human Subjects Act (also known by its Dutch abbreviation WMO) did not apply for this study. This study was conducted in accordance with the Declaration of Helsinki. Written informed consent was obtained from all patients participating in the study. All patient data were registered in case report forms and stored securely.

Quantitative Data
On the basis of the outcome definitions by Proctor et al [38], a broad range of implementation outcomes was assessed-overall and per ward-to comprehend the full extent of the implementation. An overview of the measured constructs and timing is presented in Table 1. The main outcome was the "intervention fidelity," defined as the proportion of written nurse EHR reports on the CMVS trend assessment per patient per nursing shift. A 100% score would be 3 reports per 24 hours per patient. We considered 70% of written reports per patient to be sufficient for implementation success based on our previous feasibility study [15]. In addition, any follow-up nursing activities in case of deviating trends were described.
The secondary outcomes were implementation fidelity; technical fidelity; and a survey of nurses on appropriateness, acceptability, usability, adoption, and feasibility. For implementation fidelity, the proportion of nurses who had completed the e-learning module, the proportion of monthly evaluations with the project team, and the implementation measures were documented and described. In addition, exposure (defined as the proportion of hospitalized patients receiving the intervention at the ward during implementation), recruitment (defined as the proportion of actual patients willing to participate), and retention rate (defined as retention of patients using CMVS during admission) were recorded. Moreover, regarding technical fidelity, the following data of the CMVS system were collected: number of measurements, proportion of data artifacts, D-EWSs, and premature replacement of the sensor because of technical failure. An artifact was defined as an invalid measurement as identified by the algorithm of the system and presented as -?-.
The surveys for nurses were sent via email and consisted of several questionnaires ( Table 1). The 4-item Acceptability of Intervention Measure evaluated acceptability, the 4-item Intervention Appropriateness Measure evaluated appropriateness, and the 4-item Feasibility of Intervention Measure evaluated feasibility, all on a 5-point Likert scale (score 1-5). A median score of ≥3.5 was considered sufficient [39].
The 10-item System Usability Scale measured usability on a 5-point Likert scale, resulting in a score of 0 to 100. A usability score of ≤50 was considered unacceptable, 51 to 70 was considered marginal, and >70 was considered acceptable [40].
The 15-item Evidence-Based Practice Attitude Scale on a 5-point Likert scale measured adoption (score of 0-4) with the following subscales: requirements, appeal, openness, and divergence. Scores were reported as overall scores and per subscale. A higher score indicated better adoption. A median score of ≥2.5 was defined as sufficient adoption.
Finally, we collected data on patient characteristics (gender, age, BMI, American Society of Anesthesiologists classification, urgency of admission, Short Nutritional Assessment Questionnaire [41], smoking status, alcohol use, and comorbidities [Charlson Comorbidity Index score ranging from 0-12]) [42,43] and outcomes (length of stay, mortality, unplanned intensive care unit admissions, and rapid response teams) from administrative data from the EHR. Nurse demographics (gender, age, job position, working experience in years, and working hours per week) were collected from the hospital's personnel records.

Qualitative Data
In addition to the quantitative data, semistructured interviews were conducted with nurses ( Table 1). The qualitative element of this study aimed to clarify the quantitative data. A pilot-tested topic list guided the interviews (Multimedia Appendix 6), which were conducted by 2 nursing students who were trained and supervised by the project manager (JL). The interviews were audio recorded and transcribed verbatim. No field notes were taken. Per ward, at least 10 semistructured interviews were conducted in a secluded room on the ward in the last month of the study.

Quantitative Data
Data were analyzed using descriptive and inductive statistics with SPSS Statistics (version 26; IBM Corp) for Windows. Each continuous parameter was checked for normality using the Kolmogorov-Smirnov test and visually using a Q-Q plot and histogram. Normality-based reporting was performed using means with SDs or medians with IQRs. For categorical data, frequencies and percentages were reported.
To explore the differences between the wards, the unpaired t test, Mann-Whitney U test, and chi-square test of the Fisher exact test were performed based on normality and test assumptions. In addition, multiple linear regression was performed for explorative analysis of intervention fidelity of the nurses based on the reports. The independent variables were the Charlson Comorbidity Index [42], length of stay, number of D-EWSs of ≥3, amount of artifact data (in percentage), and the month of implementation. Implementation month was a dummy variable divided into early (months 1-2), mid-(months 3-4), and late (months 5-6) implementation. For all tests, P<.05 was considered statistically significant.

Qualitative Data
The interviews were analyzed using deductive thematic analysis with the qualitative data analysis software NVivo (version 12; QSR International). The raw data were analyzed using a 6-stage thematic analysis as outlined by Braun and Clarke [44]. The stages included (1) immersion, (2) generating initial codes, (3) searching for and identifying themes, (4) reviewing themes, (5) defining and naming themes, and (6) writing the report.
In total, 2 researchers (JL and HR) conducted stages 1 to 3 independently. During the first and second stages, JL and HR became familiar with the data by listening to the audio recordings, checking the transcriptions against the audio recording, reading, listening to sections again, and rereading the final transcripts. During the third stage, both researchers read the transcripts and codes for categorizing similar statements into first themes. In stages 4 to 6, all authors were responsible for reviewing, defining, and naming themes through discussion. During the sixth stage, the themes were brought to the nurses for member checking via email, which did not result in any changes to the themes.

Mixed Methods: Integration and Interpretation
Integration of the quantitative and qualitative elements of the study occurred through linking the methods of data collection and analysis. Linking of methods occurred through building: the data of the inclusion and intervention fidelity per month served as the start for the interview, and possible explanations based on the nurses' experiences were discussed. Interpretation and reporting occurred through the contiguous approach: presentation of qualitative and quantitative findings in consequent but different sections [45].

Intervention Fidelity
Eventually, 6142 shifts were analyzed. The overall mean intervention fidelity for both wards was 70.7% (SD 20.4%); it was considered sufficient in the surgical ward but not in the internal medicine ward (73.6%, SD 18.1% vs 64.1%, SD 23.7%; P<.001). Multiple regression analysis showed that intervention fidelity remained stable over time in the surgical ward but decreased over time in the internal medicine ward (76.3% at early implementation vs 56.5% at midimplementation vs 48.2% at late implementation; P<.001; Figure 1 and Table 4). Changes in intervention fidelity could not be explained by other variables (Multimedia Appendix 8).
With respect to the documented nursing activities (n=174; range 1-9 per patient), for most patients (246/358, 68.7%), no nursing activities were needed based on the vital sign trend assessments. A total of 101 interventions were carried out by nurses individually; it mostly consisted of an extra bedside assessment of the patient followed by wait and see (n=73). In addition, 73 activities were performed after consultation with a physician (59/73, 81% of these were at the surgical ward; Table 5).

Implementation Fidelity
Regarding implementation fidelity, most nurses attended the e-learning module, all elements of the strategy were delivered, and monthly evaluations were performed. There were 27 implementation measures conducted but no major changes in the intervention itself (Table 4 and Multimedia Appendix 9). Furthermore, recruitment and retention rates were 98.4% and 99.4%, respectively, and did not significantly differ between the wards (Table 3). Exposure to the intervention was 33.6% (248/738) of patients at the surgical ward versus 21.8% (110/504) of patients at the internal medicine ward (P<.001). In addition, the proportion of patients who participated over time in the internal medicine ward was 46.4% (51/110) at early implementation, 34.5% (38/110) at midimplementation, and 19.1% (21/110) at late implementation.

Technical Fidelity
Regarding technical fidelity, a total of 45,113 hours of monitoring was available (

Nurses' Surveys
A total of 194 surveys were returned (Table 6). At the start of the study, surgical nurses found the intervention sufficiently appropriate in contrast to internal medicine nurses (median score 4.0 vs 3.1; P=.03). In addition, the overall attitude toward the adoption of new interventions was high (score of 3.5) in both wards (P=.82). Nurses in both wards found the intervention sufficiently acceptable during the study but not at the end (score of 3.5 vs 3.0; P=.02). Acceptability was significantly lower in the internal medicine ward at the end of the study (P=.02). Usability was rated as marginal in both wards at both measurement times. Feasibility was rated as sufficient but decreased at the end of the study (score of 4.0 vs 3.4; P=.002).

Characteristics
A total of 21 semistructured interviews were conducted with a mean duration of 10.9 (SD 3.3) minutes. Of the 21 interviewees, 11 (52%) worked at the surgical ward, 8 (38%) were senior nurses, and 3 (14%) were male. In total, 5 themes were identified.

Theme 1: Prioritizing CMVS
Nurses indicated that the prioritization of CMVS depended on the caseload during the shift. A commonly mentioned factor was perceived workload, frequently mentioned as tasks that they must perform during their shift. A nurse said the following:

Yes, I think it is when the workload is high, and then it easily forgotten because of it is not your priority to check and report the trend. If it's just a quiet shift, then it's easier to perform. [Internal medicine nurse 2]
In addition, some nurses indicated that this varied by type of shift. Day shifts had a higher workload than evening and night shifts. Although night shifts were predominantly experienced as quieter, the actual intervention fidelity was not better during evenings or nights. A nurse explained the following:

Theme 3: Experiencing CMVS as an Added Value for Patient Care
Nurses differed in their opinions regarding the benefits for patient care of the intervention. Nurses who were positive about the added value of CMVS mentioned that it provided more insights into the patient's clinical status, especially during night shifts and in patients who are critically ill. However, they also indicated that these types of patients do not often present at the general ward. In addition, several nurses mentioned that they had limited experience with the intervention and even no experience with deviating vital sign trends and taking action on them. Therefore, nurses questioned whether proactive trend assessment was feasible as standard care as, in many cases, it did not alter their nursing care at that time. A nurse said the following about this:

Theme 4: Experienced Usability of the CMVS System
The nurses frequently mentioned the experienced usability of the hardware and software as an explanation for the decreasing intervention fidelity. Although some nurses found that the necessary time investment was limited and CMVS was feasible during their shifts, several barriers to regular daily use were mentioned. The most often-mentioned barrier was the pairing of the sensor with the software platform as this had to be done through a separate web-based application on a prepared mobile phone rather than via the regularly used phone with a call system. A nurse said the following: Sometimes the separate mobile phone with the specific codes malfunctions and it simply takes too much time, which eventually results in that you leave it at that.
[Internal medicine nurse 5] Another barrier mentioned was the convenience of gaining visibility of the trends. The software for assessing trends was not integrated well enough into the EHR. Although the bed overview with patient names and numbers was paired, they preferred the trends to be also presented in the EHR or to be able to view them through a central monitoring display on the ward. Finally, removing the sensor when performing diagnostics for the prevention of interference was considered a barrier; in particular, they felt this was important as diagnostic tests such as electrocardiograms or computer tomography scans are often ordered for patients who are ill. A nurse said the following: It is annoying when a sick patient has to go for a scan and then just at that important moment, the sensor must be removed. [Surgical nurse 4]

Theme 5: Future Perspectives of CMVS on the General Ward
Several nurses shared their thoughts on what improvements are needed for future routine use. In addition to full integration of the software into the EHR, as mentioned in the previous section, nurses considered it important that the sensor be able to measure more vital signs than only HR and ReR. The main reason for this was that manual measurements of the other routine parameters (such as blood pressure and blood oxygen saturation) are still considered necessary, and therefore, CMVS with just HR and ReR does not result in measurable time-saving benefits. This would only be possible when all vital sign measurements and trends are directly visualized in the EHR. Although this would save time, it would not eliminate the need and value of bedside nursing assessments during rounds, as discussed in the previous section. A nurse said the following: Furthermore, some nurses thought that there might be benefits in continuing the intervention after discharge from the hospital. A reason given for this was that remote clinical assessment is more difficult in a home situation. Moreover, they found that it could potentially encourage early discharge by incorporating CMVS into an early recovery protocol, such as Enhanced Recovery After Surgery.
Finally, nurses indicated that the use of assistive technology is desirable for the future of nursing care, considering the enrichment of nursing care and in view of future challenges in terms of capacity shortages. A nurse said the following:

Principal Findings
In this study, we evaluated the process of implementation of CMVS on 2 general wards. Using a comprehensive implementation strategy, our overall results suggest that CMVS was sufficiently implemented in both wards, although intervention fidelity was highly variable and decreased over time. This decrease was explained to a large extent by the declining intervention fidelity in the internal medicine ward (it remained stable in the surgical ward). Another contributing factor was that nurses in both wards perceived little added value to the intervention. Taken together, the results show the complexity and interconnectedness of implementation and intervention fidelity with the technology and the perceptions of nurses.
Although the recruitment and retention rates of the intervention were high, indicating high patient acceptance, both wards showed a decline in several dimensions of implementation: intervention fidelity (although not statistically significant for surgery), perceived acceptance, usability, and feasibility. Interestingly, this decline was lower in the surgical ward than in the internal medicine ward. There are several possible explanations for this discrepancy between the surgical and internal medicine wards. Although 110 patients were included in the internal medicine ward, compared with the surgical ward, exposure to the intervention was still limited (110/504, 21.8%) and decreased over time-especially during the last 2 months of implementation. Second, nurses in the internal medicine ward considered the intervention less relevant to their practice. A first possible explanation as far as the internal medicine department is concerned, is the hospitalization procedure for emergency patients. After presentation in the emergency room and subsequent admission to the acute ward for a maximum of 48 hours, the patient is then transferred to the internal medicine ward. At that time, the diagnosis is established, and treatment starts, and so these patients have already passed the precarious, critical stage of their condition, and deviations in vital signs may be considered of lower clinical relevance [46]. This was different in the surgical ward, where CMVS was started directly after surgery, the period in which the patient is at the highest risk of complications and deterioration [47]. This may also be an explanation for the low appropriateness ratings from nurses in the internal medicine ward. Nevertheless, it is noteworthy that the proportion of patients with abnormal D-EWSs was highest in the internal medicine ward, but this was not deemed clinically relevant.
Although a broad range of interventions was performed by nurses based on the trend assessments, several reasons might explain the perceived low added value of the intervention for nursing care. First, the rationale for using CMVS is likely to be less convincing when also maintaining the conventional manual nurse measurements to calculate the MEWS. This could be explained by the fact that nurses highly value being at the bedside and observing the patients themselves while performing their manual measurements. Nurses explained that they use this moment to perform a more comprehensive patient evaluation, including assessing domains of clinical deterioration other than vital signs as well as other nursing domains through patient interaction. Second, the high degree (246/358, 68.7% of all patients) to which no subsequent activities were initiated based on the trend analysis may indicate that intervention fidelity was limited for this reason. In general, nurses stated that they had little or no experience interpreting deviating vital sign trends.
In specific cases, trends may have prompted more timely additional measurements or diagnostics, such as blood tests or imaging, or the initiation of a physician's consultation, but overall, it remains difficult to determine to what extent vital sign trend monitoring actually contributed to decision-making.
In addition, the current state of the technology may have affected intervention fidelity. Despite the generation of a very large amount of data, technical difficulties remain. Approximately 10.3% (37/358) of the sensors had to be replaced prematurely owing to different types of failure, such as malfunctioning of the sensor during pairing, unexplainable sensor failure, or high artifact ratios in some patients. This was also reflected in the high artifact rate for HR measurements, which may have had a negative influence on intervention fidelity and acceptability. A possible explanation is that adequate HR measurements using an accelerometer may be more complicated, but this has not yet been adequately studied [35]. The higher HR artifact rate in the internal medicine ward is also unclear. We checked patients with high artifact rates for incorrect sensor placement, but this was rarely the cause. Current limitations of the technology likely contributed to the low usability scores during the implementation period in both wards. In the interviews, nurses commented that these issues made it cumbersome to use the system while reducing trust in the technology. Furthermore, the current threshold-based D-EWSs to guide trend assessment do not sufficiently consider the context of the patient (eg, "in bed" or "actively mobilizing"), resulting in contamination of vital sign trends (eg, simultaneously HR and ReR) that are actually normal as the patient is actively mobilizing. Consequently, it will be harder to recognize true deterioration early. In contrast, when nurses manually record an abnormal set of bedside vital signs, CMVS trends may show an important correlation with the current (abnormal) bedside observation and can support the nurse's decision to seek consultation with the on-call physician.
The correlation between vital signs and direct bedside observations is important for clinical decision-making, which is missing when relying entirely on remote trend assessments.

Comparison With Other Work
Comparison of our results with those of previous studies is challenging because of differences in patient populations, monitoring devices, and outcomes addressed. Intervention fidelity in this study was somewhat lower than in our previous feasibility study over a period of 3 months with a similar CMVS intervention [15]-71% versus 81%, respectively. However, if we compare the first months, this difference is smaller (75.8% vs 80.5%).
The need to still perform manual vital sign measurements and the lack of experience in assessing deteriorating trend patterns-as previously mentioned by nurses-are likely to have affected nurses' perspectives and may have influenced intervention fidelity. This observation is also in line with the results of our previous feasibility study [15]. Moreover, although abnormal HR and ReR are important signs of patient deterioration, evidence is still lacking that CMVS monitoring of only 2 vital signs is sufficient to capture most cases of deterioration. In contrast to our results, Verrillo et al [48] showed that, when CMVS using a bulkier multiparameter device was used as the single method for vital sign monitoring, nurses' acceptance and compliance over a period of only 6 weeks increased (initially 38% to a sustained average of 62% compliance). This may indicate that automating the manual measurements is better for the acceptability of nurses. Nonetheless, larger devices measuring all vital signs may result in poorer patient acceptance. Early termination of the intervention was rare in our study, which is in contrast to the 21% of patients in a previous study with a wrist-worn multiparameter device [14]. However, in our study, approximately 10.3% (37/358) of the sensors were prematurely replaced owing to technical errors such as connectivity issues. Furthermore, the need to gain experience with the use of the wearable device in clinical practice was also mentioned by nurses in the study by Izmailova et al [49]. Moreover, in line with previous studies, nurses also sometimes questioned the accuracy of the device and doubted the benefits of being able to observe their patients' vital signs remotely [50,51]. In contrast, many nurses also expressed a positive attitude toward CMVS interventions, mentioning that it could increase patient safety by providing more insight [52]. Finally, experienced usability of a similar wearable patch device in ward nurses was higher in the study by Boatin et al [53], although this may be because of the relatively small, short-term study of 32 pregnant women.
Other studies have also reported on technical fidelity. Our observed artifact rates were slightly higher compared with a validation study in patients of surgery at the postanesthesia care unit with the same sensor [35]. A potential explanation is that motion artifacts are more prevalent in patients in wards than in patients during the early stages of recovery after anesthesia and surgery in the postanesthesia care unit.

Limitations
To our knowledge, this is the first study that extensively focused on evaluating the process of CMVS implementation at scale in daily clinical practice in general hospital wards. The data can provide valuable information to other hospitals considering CMVS implementation and highlight some important issues to consider when developing an implementation strategy. However, several limitations should be considered when interpreting the results. First, in both wards, exposure to the intervention was still limited, which forced nurses to work with 2 systems of vital sign monitoring (intermittent and continuous) and may have hampered implementation. Second, it is important to note that the development of the implementation strategy and intervention took place in the surgical ward, which might have resulted in an intervention more suited to a surgical ward than to an internal medicine ward. In addition, goodwill toward the project manager, a former nurse in the surgical ward, might partly explain the higher intervention fidelity in the surgical ward. Third, even after analyzing every individual nurse trend assessment report, it is still not possible to determine with certainty to what extent these vital sign trends actually influenced subsequent diagnostic or therapeutic decision or both. This is mainly because several other factors contribute to additional activities and medical decision-making. Moreover, it is not clear how large the variation is between nurses interpreting similar trends. This would require a separate study. Finally, we did not include the magnitude of nurses' exposure to the intervention as a factor in the regression analysis, which could cause bias. However, we extensively focused on education and bedside training in the implementation strategy.

Implications
Our study highlights the complexity of implementing a CMVS system with wearable wireless sensors in hospital nursing wards. Therefore, policy makers should involve nurses early in establishing the intervention and implementation strategy and selecting the appropriate patient populations to enhance the fit with the needs of current nursing practice. To leverage the full potential of CMVS in general wards, several barriers to implementation in the routine workflow need to be addressed, for which we suggest the recommendations outlined in Textbox 2.

Textbox 2.
Recommendations to address the barriers to implementation in the routine workflow.
• Secure full and seamless integration of the continuous monitoring of vital signs (CMVS) into the hospital electronic health record, avoiding any separate software platforms or dashboards. This will improve fidelity and usability for caregivers.
• Use advanced and validated multiparameter CMVS sensors, which are sufficiently accurate and comprehensive to allow for the discontinuation of standard manual vital sign measurements by nurses, thus reducing nursing workload.
• Combine CMVS with reliable personalized clinical decision support tools to facilitate correct and timely interpretation of these measurements. Algorithms still need to be developed that can incorporate patient-specific baseline data, facilitate routine automated input of contextual factors such as patient movement, and perform automated trend analysis and event detection to timely detect and alert on clinical deterioration [22]. When such systems are available, this will obviate the need for vital sign trends to be proactively monitored and interpreted by nurses, which currently increases nursing workload and is difficult because of their lack of experience in this respect.
• Finally, carefully select (high-risk) patient populations that are likely to benefit most from CMVS. This could potentially include all acute care admissions (especially those without a clear diagnosis at admission) and all patients undergoing intermediate-or high-risk surgery in the postoperative phase (both in the ward and at home directly after discharge). Thus, the intervention could be integrated into an early discharge protocol with extended telemonitoring at the patient's home [54].

Conclusions
We successfully implemented a system for continuous wearable remote vital sign monitoring at scale in 2 hospital wards, but our results show that intervention fidelity decreased over time, to a larger extent in the internal medicine ward than in the surgical ward. This decrease appears to be dependent on multiple ward-specific factors. Nurses' perceptions regarding the value and benefits of the intervention were variable. Our study provides valuable insights into the optimal implementation of CMVS in general wards. Specifically, we conclude that implementation of a CMVS while at the same time maintaining routine manual vital sign measurements is not advisable as it increases nurse workload. Proactive vital sign trend assessment by nurses is feasible but challenging to embed sustainably at scale in current workflows even when using an extensive implementation strategy. Wearable wireless monitoring technology should be further developed and optimized, including seamless integration into the EHR and development of more sophisticated decision support tools for interpretation and alarms that are suitable for general wards, before it can consistently improve nursing workflows, increase patient safety, and enhance quality of care.