Towards an evaluation framework for inclusive technological innovation in social and health care services

With the COVID-19 pandemic came an influx of digital technologies introduced into the care home sector. Many care homes were not ready for digital changes, and subsequently, the inclusivity of these innovations was variable. We develop a holistic evaluation framework to assess the inclusivity of service innovations, including the well-being of care home residents and employees. We adopt a mixed methods approach using the ‘non-adoption, abandonment, and challenges to scale-up, spread, and sustainability (cid:0) complexity assessment toolkit ’ (NASSS-CAT) to evaluate the introduction of tablet computers into care homes across Scotland and assess the inclusivity of the innovation. The paper offers a theoretical synthesis by indicating how the seven dimensions of the NASSS-CAT can evaluate the six stages of the ladder of inclusive innovation. Results specify how the care home sector could be more inclusive of its residents and staff by co-creating innovation, developing staff training, and supporting the well-being/activities coordinators.


Introduction
Older people within care homes and people with disabilities are often subject to social, spatial, digital, and care inequalities, all of which were exacerbated during the COVID-19 pandemic (Buffel et al., 2021;Jang & Kim, 2020).Accordingly, residents and employees within care home services suffered many challenges and negative consequences.Not only was there high mortality and morbidity among care home residents, a population at acute risk from COVID-19, but a further consequence was long-term social isolation due to prolonged social distancing measures (Williams et al., 2021).This was worsened by the lack of digital connectivity within some care homes, including limited digital infrastructures and discomfort with new media.Subsequently, residents in long-term care facilities were more likely to experience social and digital exclusion, which had severe detrimental effects on their physical and emotional health and heightened their experiences of loneliness and isolation (Seifert et al., 2021).
Due to the situation's urgency, service innovations were created and deployed rapidly, which was exceptional for the health and social care industry, with many of these interventions becoming deeply ingrained in care systems (Litchfield et al., 2021).Such innovations include video conferencing designed for social interaction (Badawy et al., 2022), consultation between residents and health care professionals (Warmoth et al., 2022;Wherton & Greenhalgh, 2020), and befriending schemes (Fearn et al., 2021).These studies reveal fewer positive outcomes than pre-pandemic innovations and report more barriers to inclusivity and innovation adoption.Barriers include the additional work routines, tasks, and responsibilities of staff members already on limited time and reluctance among a few residents to engage with technology (Badawy et al., 2022;Schuster & Cotten, 2022).Poorer outcomes during the pandemic suggest that innovations were implemented to improve the residents' inclusivity but without considering the readiness of the organisation and/or the adopters.Accordingly, there needs to be a process of evaluating the level of inclusivity and consequent impact on inequalities within the care home sector.
Traditional methods of evaluating service innovations in the business domain measure effectiveness and profit by utilising quantitative data and extensive surveys (Peng & Lai, 2014).However, there is a low response rate to online questionnaires in the care home sector due to the demands placed on staff time and the lack of digital infrastructure (Ellwood et al., 2018).Furthermore, due to the inequalities, evaluations in this sector need to move beyond measuring profit towards measuring lived experiences and the impact of the innovation on staff and residents' welfare and well-being.We subsequently evaluate service innovations in care homes for older people with dementia or frailty through an inclusive innovation lens, which is henceforth defined as "the development and implementation of new ideas which aspire to create opportunities that enhance social and economic well-being for disenfranchised members of society" (George et al., 2012, p. 663).To do so, we adopt the ladder of inclusive innovation, a series of successive stages that an innovation should adhere to to achieve inclusivity of excluded groups (Heeks et al., 2013(Heeks et al., , 2014)).
To undertake the evaluation of inclusivity, we adopt a mixedmethods approach using the non-adoption, abandonment, and challenges to scale-up, spread, and sustainability − complexity assessment framework (NASSS-CAT; Greenhalgh et al., 2020).The framework assesses technological service innovations within health and social care.It is flexible and encourages the collection of qualitative, quantitative, and secondary data, depending on data availability.To explore the different aspects of the NASSS-CAT framework, a mixed methods approach enables breadth in understanding the context of the technology adoption and depth in understanding the experiences in individual care homes.The approach to mixed methods here is one of complementarity, completeness, and diversity, where the different types of data provide a complete picture of the adoption of technology in care homes and support engagement with the diverse stakeholders involved (Caruth, 2013).We, therefore, use the NASSS-CAT to evaluate the inclusivity of tablet computers (for residents and staff) being placed into Scottish care homes during the COVID-19 pandemic in order to answer the following research question: To what effect can a synthesis of the NASSS-CAT framework and ladder of inclusive innovation be used to evaluate the inclusivity of service innovations in the care home sector?
For business scholars, we contribute here by synthesising two frameworks.We map the evaluation domains of the NASSS-CAT to the six rungs of the ladder of inclusive innovation to evaluate the inclusivity of service innovations in health and social care (Greenhalgh et al., 2020;Heeks et al., 2013Heeks et al., , 2014)).Our framework is valuable in contexts where there are inequalities and quantitative data is difficult to acquire or does not reveal the nuances of the intervention, for instance, in care services, health care, social care, services for older consumers, services for vulnerable consumers, specialised and personalised services, and small organisations.For managers, we contribute to the limited understanding of inclusive innovation in care homes by developing recommendations for future practice, staff management, and resident well-being.For policymakers, we develop recommendations for introducing and supporting inclusive innovations in care homes, a sector which suffers from inequalities.

Evaluations of inclusive innovations in services
Inclusive innovation, which originated with Utz and Dahlman (2007), stems from the observation that the success of a mainstream innovation (as a product or service) is frequently measured by its impact on economic growth, which creates inequalities within society (Klochikhin, 2012).Alternatively, inclusive innovation involves the active inclusion of people who are disenfranchised from mainstream development (Foster & Heeks, 2013;George et al., 2012).This is with the intention to address the needs, desires, and issues of marginalised groups and create a positive impact on their livelihoods and well-being (Heeks et al., 2013;Heeks et al., 2014).Most inclusive innovation scholarship focuses on innovation as a tool for affordability and inclusion within lower-middle-income countries (LMICs; Mortazavi et al., 2021).However, inclusive innovation and subsequent theories and frameworks can potentially reduce inequalities of excluded groups worldwide, including within high-income countries (HICs; e.g.van Hagen et al., 2021).The ladder of inclusive innovation, for instance, applies to any context where innovations are introduced to meet the needs of an excluded group, irrespective of location (Heeks et al., 2013;Heeks et al., 2014;Smith et al., 2023;Ustyuzhantseva, 2017).
As an innovation climbs the ladder, the marginalised group becomes further integrated into the process, and the innovation gains a broader impact (Heeks et al., 2014;Woodson et al., 2019).For instance, level 1 is reached if the innovation intends to be inclusive and to address the values of the marginalised group.Many innovations fail at this level because although there might be an intention, the innovation is not adopted or does not establish an impact (e.g.Harsh et al., 2018;Woodson et al., 2019).Level 2 is achieved if the excluded group uses and consumes the product or service-based innovation.This likelihood is enhanced through incentives, donations, or innovation trials (Harsh et al., 2018;Smith et al., 2023).Level 3 occurs if the innovation positively impacts the livelihoods, health, and well-being of the excluded group, considering both the short-and long-term effects (Heeks et al., 2014;Woodson et al., 2019).
An innovation transitions into inclusivity when it starts to climb levels 4, 5, and 6 on the ladder (Heeks et al., 2014).Level 4 requires the excluded group, usually exempt from the innovation system, to be included in the development process, ideally through co-creation and driving innovation within the community (Woodson et al., 2019;Ustyuzhantseva, 2017).Level 5 entails the broader structure of the organisation and associated institutions to be inclusive, including changes to the organisation's policies or corporate social responsibility and the marginalised group gaining access to quality education and health care (Harsh et al., 2018;Woodson et al., 2019).Finally, level 6 involves post-structural inclusivity, where the innovation should be created within a discourse that is also inclusive, including changes in how society uses language, attains knowledge, and understands morality (Heeks et al., 2014;Woodson et al., 2019) (Fig. 1).
Despite the importance of the latter levels on the ladder, the inclusive innovation literature lacks the evaluation of the process behind implementing innovations inclusively and how to improve this procedure (Mortazavi et al., 2021).The ladder, often referred to as a heuristic rather than a framework, would, therefore, benefit from an established evaluation framework to assess an innovation along each stage (Smith et al., 2023).Previous evaluation processes in the service innovation literature tend to use models developed originally for the manufacturing industry to measure the success of the service innovation through quantitative analysis, relying on either survey responses or subjective inputs from industry experts (Peng & Lai, 2014;Witell et al., 2016).The outputs of these evaluations focus on quantitative outcomes of the innovation (e.g.measures of service quality, trustworthiness, security, and convenience; Peng & Lai, 2014), which all improve profit rather than encourage inclusivity along the innovation process.
Recently, there has been a movement towards defining the value of innovation from the consumer's viewpoint (Lusch & Nambisan, 2015).This is especially beneficial in social and health care, where innovations might cost organisations economic value but create value for the consumers by enhancing well-being and reducing inequality (Witell et al., 2016).However, evaluation frameworks specifically designed to assess health and social care service innovations are sparse within the business literature (Miles, 2016).Those that do exist explore a range of values such as electronic loyalty, information privacy, and satisfaction (Lotfi & Soleimani, 2020), customer-perceived quality of care (Wu & Hsieh, 2011), and improved communication, information access, and organisational performance (Campanella et al., 2021).These studies predominantly employ a mixture of qualitative and quantitative data, demonstrating the importance of collecting both (Obayashi et al., 2020).For example, Khaksar et al. (2016) use interviews and surveys to evaluate whether social robots reduce the social vulnerability of older residents in care homes, which provides an understanding of the complexities of the intervention and qualitative insights into lived experiences.
C. Wilson-Nash et al.

Inclusive innovation in care services
With an emphasis primarily on LMICs, no known studies within inclusive innovation scholarship focus on interventions in care homes (Mortazavi et al., 2021).However, extant research on technology-led advancements in care homes often takes an inclusive approach by measuring the impact of the innovation on the livelihoods of residents (Heeks et al., 2013).These studies measure singular outputs, such as social connectedness (Moyle et al., 2020;Schuster & Cotten, 2022), loneliness (Zamir et al., 2020), and well-being (Mueller et al., 2021).The innovations tend to involve communicative technologies, adopted for synchronous audio-visual communication with relatives, friends, and health care providers (Evans et al., 2017;Schuster & Cotten, 2022) and asynchronous messaging (Schneider-Kamp & Fersch, 2021).Tablets are the preferred option, but alternative technologies include Skype on Wheels (Zamir et al., 2021), laptops (Tsai et al., 2020a), accessible tablets (Badawy et al., 2022), and smartphones (Tsai et al., 2020b).Most studies are embedded within gerontology and health sciences, with scarce examples employing innovation theory within marketing and management.There is consequently scope to develop a holistic framework to evaluate the adoption of inclusive innovation within care homes.
Residents of care services benefit most from innovation when it involves a social element of added communication with friends and family (Siniscarco et al., 2017), befriending schemes (Fearn et al., 2021) or support from staff members (Moyle et al., 2020;Schuster & Cotten, 2022;Siniscarco et al., 2017).Research in similar settings to residential care (e.g.psychiatric outpatient settings) also illustrates the importance of building familiarity and trust between residents, technologies, and staff members' support (Schneider-Kamp & Fersch, 2021).Although the involvement from employees is of great interest, the knowledge and skills required to use technology are less discussed, resulting in 'hidden' work to incorporate technology into care (e.g.ensuring devices are used, stored, and maintained correctly), which paradoxically creates inequality for care home employees (Monin et al., 2020;Zamir et al., 2020;Zamir et al., 2021).There is consequently a lack of research that engages with the complexity of care homes, further supporting the need for a holistic evaluation framework to measure the inclusivity of service innovations in the care sector, including the uptake of the intervention by residents, staff members, management, policyholders, funders, suppliers, and other stakeholders.

Evaluation framework: NASSS-CAT and the ladder of inclusive innovation
The NASSS is a theory-informed methodological framework developed to examine and assess the individual, organisational, and policyrelated factors that affect how health and social care innovations are adopted (Wherton & Greenhalgh, 2020).It is developed by Greenhalgh et al. (2017) who aim to understand why technological innovations fail through non-adoption and abandonment (van Limburg et al., 2011), including interventions such as telehealth (Bentley et al., 2014).The more complex an innovation is, the more likely it is to fail (Cresswell & Sheikh, 2013).There are seven domains in the framework that are used to assess the challenges associated with scale-up, spread, and sustainability of an innovation or technology (see Fig. 2).
First, the condition refers to the condition or illness that the technology is designed to address, including details about the condition, co- existing illnesses, and the likelihood of the condition to change over time.Second, the technology domain evaluates the technology or innovation, including the intricacies of the technology, the supply chain, data collection, privacy concerns, and usability.Third, the value proposition is the value of the technology for the interested parties, such as commercial stakeholders (e.g. return on investment), patients (e.g.quality of life), and health care professionals (e.g.quality of care).Fourth, the adopters include anyone who might use the technology, such as patients, residents, professionals, administrative staff, and support staff.Fifth, the organisation(s) domain provides further understanding of the organisation/s involved in the project, such as their capacity to innovate, changes to routines and processes, and work that needs to be undertaken to introduce the innovation.Sixth, the wider system includes how external influences might impact the innovation's success (or not), such as policy, legislation, professional and patient organisations, and the commercial context.Finally, embedding and adaption over time evaluates the potential of the innovation to evolve over time and the resilience of the organisation(s) to handle changes (Greenhalgh et al., 2017).
A practical and adaptable resource, the NASSS-CAT framework aids organisations in using the NASSS domains to assess their service innovations.NASSS-CAT combines desk-based methods with primary qualitative and/or quantitative research to identify outcomes and individual or organisational factors influencing successful technology adoption across the seven aforementioned domains (Greenhalgh et al., 2020).The NASSS-CAT is an advancement on the original NASSS as it is a combination of Greenhalgh et al.'s (2017) evaluation framework, which was designed for academic analysis, with the complexity assessment tool (CAT), a practical tool for managing project complexity (Maylor & Turner, 2017).The NASSS-CAT is, consequently, an interdisciplinary evaluation with foundations in both health sciences and business.It is used in a range of evaluations of technology projects, including the evaluation of the 'Near Me' video consulting service in Scotland (Wherton & Greenhalgh, 2020) and the use of video consultation technology between care homes and health and social care professionals during COVID-19 in Hertfordshire (Warmoth et al., 2022).Despite evaluating the impact of innovation on the welfare of staff, residents, and carers, it remains unutilised in assessing the inclusivity of innovation within the business literature.
We, therefore, adopt the NASSS-CAT as an evaluation framework to assess the inclusivity of a service innovation within care homes.To synthesise the NASSS-CAT with the ladder of inclusive innovation, we explore how the seven evaluation domains map onto the six rungs of the heuristic ladder (Greenhalgh et al., 2020;Heeks et al., 2013Heeks et al., , 2014)).

Research context and design
Our research is based on an evaluation of the 'Connecting Residents in Scotland's Care Homes' (CRSCH) inclusive service innovation.The Scottish Government introduced the CRSCH innovation in November 2020 with the aim of providing tablet computers to every care home in Scotland.The goal was to integrate the technology into everyday routines by allowing residents to use video conferencing platforms to stay in touch with friends and family during periods of social isolation and national lockdowns.The tablet computers were given to the employees at the care homes, who looked after a pool of available devices, which C. Wilson-Nash et al. were then offered to the residents to facilitate their adoptive uses.By the end of the initiative, 75% of the 1325 registered care homes received at least one tablet computer, and 33% received mobile MiFi routers, which allowed wireless internet access if an internet connection was unavailable or of poor quality.
The interaction between stakeholders, infrastructures, policy, and the care environment made the CRSCH initiative a complex intervention.We, therefore, adopted the NASSS-CAT evaluation framework at the beginning of the project, which enabled a flexible approach to data collection between December 2021 and July 2022 (Greenhalgh et al., 2017;Greenhalgh et al., 2020).We used secondary data analysis, interviews, and workshops to assess the service innovation from the viewpoints of all stakeholders, including residents, care workers, managers, and policymakers.The suitability of the NASSS-CAT to evaluate the inclusivity of the innovation latterly emerged during collection and analysis procedures, as the emerging NASSS-CAT findings mapped onto the ladder of inclusive innovation (Heeks et al., 2013(Heeks et al., , 2014)).

Secondary data analysis and sampling
This evaluation phase involved a secondary quantitative analysis of pre-existing data collected from participating care homes.Data included a database of care homes that applied for the tablets, a database of organisations that attended CRSCH training, and a database published by the Scottish Care Inspectorate in February 2022, providing characteristics of all residential care facilities in Scotland.Descriptive statistics were used to explore the characteristics of care homes that engaged with the initiative, including information such as total beds, number of residents in each room, number of staff, registered places, risk assessment score, and 'quality' as graded by the Scottish Care Inspectorate.
Out of the 1325 care homes in Scotland, 59.02% (N = 782) were for older people, and from these care homes, 79.54% (N = 622) applied for and received tablet computers during the pandemic, making this the largest adoption group of the intervention.Of the care homes employing the innovation, 66.55% were private, 21.28% were voluntary or not-forprofit, and the average number of beds was 39.55.Regarding technological readiness, 54.02% of these care homes reported full Wi-Fi coverage, 36.90%partial Wi-Fi coverage, and 8.16% no Wi-Fi coverage.MiFi devices were provided for care homes that reported no or partial Wi-Fi.Moreover, 44.02% (n = 383) already owned a tablet and 24.14% (n = 210) owned multiple devices, suggesting a readiness for most, but not all, care homes.
From this data analysis, we developed a purposive sampling framework (Silverman, 2019).We identified care homes based on geographical location, resident population, variations in geography (urban, rural, remote, island), size of care home, and number of iPads received.We contacted care homes via phone calls and explained the purposes and process of the study, which resulted in qualitative interviews with staff members and deep-dive workshops.Most participating homes provided care for older people with frailty and/or dementia.

Qualitative interviews
We performed 22 semi-structured interviews with 26 staff members from care homes in Scotland who assisted residents using tablet computers (e.g.managers, staff providing direct care, and activities/wellbeing coordinators; see Table 1).Semi-structured interviews enabled the researchers to gather evidence to inform the predefined NASSS-CAT domains whilst allowing participants to reflect and elaborate on their unique experiences of the initiative (Brinkmann & Kvale, 2018).The NASSS-CAT framework was previously and successfully employed using this method (Kadesjö Banck & Bernhardsson, 2020).
We conducted semi-structured interviews via telephone or video conferencing software with individuals or small groups of up to three staff members.Qualitative research via telephone or online platforms has become more conventional since the start of the pandemic and was found to offer an excellent alternative to in-person methods when these were not possible (Saarijärvi & Bratt, 2021).Interviews typically lasted between 30 and 45 min and covered topics relating to all seven NASSS-CAT domains (Greenhalgh et al., 2017;Greenhalgh et al., 2020).Questions focused on the context of each care home, specifics of the CRSCH implementation, and perceptions of the innovation's benefits, facilitators, and challenges for residents, staff, and management.Finally, participants were requested to provide suggestions for potential areas of improvement.

Stakeholder workshop
We conducted an online stakeholder workshop with five members of the CRSCH initiative (see Table 2).This workshop explored stakeholder perspectives about the service innovation and sought to identify factors influencing the development and delivery of the initiative within the wider policy context of care homes in Scotland during the pandemic.
The workshop was held online with the aid of video conferencing software.A Miro online whiteboard was chosen to encourage engagement due to its documented success in supporting research co-design methods and enhancing participant engagement and interaction (Boone et al., 2023).The Miro board displayed four infographic posters that supported exploring the wider system factors of the NASSS-CAT framework (political/policy context, regulatory or legal issues, sociocultural context, and inter-organisational working).During the online workshop, the Miro board was populated, which later generated in-depth discussions.

Care home deep dive workshops
The final phase of the evaluation involved deep-dive workshops in four care homes across Scotland; the design of the workshops was based on a successful approach to engaging staff and residents in care homes developed in earlier research by a member of the evaluation team (Jepson et al., 2023).To conduct the workshops, two research team members visited a communal area in each care home for six hours.Eighteen residents and five staff members were involved in the workshops, partaking in informal interviews and activities (see Table 3).Two posters were used to encourage participants to provide their thoughts on how they used the tablets, how they felt about the technology, and how much and what kind of support was required.Sticky notes with participant comments were placed on the posters and served as a record of the comments received from each participant.After combining the data from residents and staff from the four participating care homes, we analysed them using the NASSS-CAT framework.

Data analysis
We evaluated the qualitative data using the NASSS framework and its accompanying analytical tool, the NASSS-CAT (Greenhalgh et al., 2020).The qualitative data was processed in NVivo 12 using a deductive thematic method (Clarke & Braun, 2021), which involved coding from a curious, open, and questioning perspective while employing the NASSS-CAT domains as a foundation.Rather than just determining whether the dimensions were present, the objective was to evaluate how the innovation was adopted under the NASSS-CAT headings (e.g. the situation, condition, or illness), which became the main themes of the analysis, with more detailed codes (e.g.characteristics of residents) feeding into these themes (see Table 4).Using deductive coding, we then established how the NASSS-CAT domains could evaluate the inclusivity of the initiative by mapping them against the ladder of inclusive innovation (Table 4; Heeks et al., 2013;Heeks et al., 2014).

Inclusion of intention (Level 1): Addressing the condition, illness, or situation
The first level of inclusive innovation is the inclusivity of intention, where the innovation must address the values, needs, wants, and issues of the marginalised group (Heeks et al., 2013), in this case, care home residents.We found that the NASSS-CAT dimension of exploring the nature of the condition or illness that the innovation is attempting to address (Greenhalgh et al., 2017;Greenhalgh et al., 2020) provides detail on the inclusivity of the intention of the innovation by further understanding the care home context and characteristics of the residents (see Table 4).However, we extend this evaluation construct to also include the situation of the marginalised group.This is because, for the CRSCH project, the innovation is not attempting to address a condition or illness but the social isolation of care home residents, as created by pandemic-related restrictions.In this example, a care home staff member is articulating the extent of the restrictions: The Scottish Government have lifted all restrictions, but they haven't lifted them in care homes, so we're still under strict guidance with regards to visitors and family visiting, so we can use the iPads to  compensate for the fact that we're still…restrictions that you might not have in the wider community (CH10, female, aged 56).
A consequence of these restrictions is that many care home residents are confined to their bedrooms, and family and friends are unable to visit.As a result, residents said that "lockdown was like a prison" and "all the busyness, visitors, and activities disappeared" (R305, male, aged 35-45).Prior to the implementation of the tablet computers, residents are described as "slipping away" due to loss of contact with their loved ones, hence the need for an innovative solution.The innovation considers the intention of the residents as "it's a massive part of care that the families are involved as much as possible with the residents who live here" (CH9, female, aged 33).The iPads, therefore, provide the mechanism to connect with family and friends; "they helped when [residents felt] alone and bored" (R305, male, aged 35-45).
The implementation of iPads in care homes in Scotland during the pandemic, therefore, achieves the first stage of inclusive innovation by addressing the needs of the care home residents to stay in contact with their family and friends (Heeks et al., 2013).This is evaluated through the first dimension of the NASSS-CAT, which explores the nature of the adopters' illnesses, conditions, or situations that the technology attempts to address (Greenhalgh et al., 2017;Greenhalgh et al., 2020).A detailed evaluation is required to fully assess whether an innovation has reached the first stage of the ladder of inclusivity and thoroughly addresses the needs of the excluded group (Heeks et al., 2013).

Table 4
Mapping the NASSS framework domains onto the ladder of inclusive innovation.

The situation, condition, or illness
The nature of the situation, condition, or illness that the technology aims to address, including any comorbidities and sociocultural factors impacting adoption.

Embedding and adaptation over time
Scope for adapting and coevolving the technology and service innovation over time and the resilience of the organisation(s) to handle critical events and react to unforeseen circumstances.

Changes over time Responding to changes in pandemic restrictions
Embedding use of tablets going forward Reasons that tablets may not be used going forward Integration of tablets in everyday life of care home

Inclusion of consumption (Level 2): The technology and the adopters
The second level of inclusive innovation is the inclusivity of consumption, which involves the innovation being adopted by marginalised groups (Heeks et al., 2013).In this example, the technology should be easily accessible to the care home residents and other adopters.We, therefore, find that a combination of the technology and the adopters dimensions on the NASSS-CAT provides invaluable insight into the inclusivity of consumption.

The technology
The technology domain examines the technology utilised in developing the service innovation, including its key characteristics, the knowledge needed to operate it, and the knowledge produced by using it (Greenhalgh et al., 2017).As tablet computers are an incremental innovation, the care home employees find them familiar, simple to use, and accessible for a variety of needs.The data analysis shows further information regarding the use of the tablet computers as well as the staff members' training requirements.
The tablet computers, initially intended to connect residents with family during the pandemic, are now used for diverse activities in addition to making video calls, including listening to music, taking pictures, watching videos, reminiscence, playing games, singing, and following online fitness classes.The following example demonstrates how the usability of the device allows the care home to adopt the technology for a range of uses: Looking up different images when service users are speaking about things, and staff just doing that ad hoc, research about a particular aspect of a condition, or that games aspect is probably more what we're doing now.The big thing for us is also being able to take pictures of little events that we've been having.It's been the Platinum Jubilee for the Queen and our service did a little jubilee afternoon for the service users.So, being able to take pictures and use it as that kind of camera function as well (CH19, female, aged 40).
Additionally, as staff members at care homes gain a better understanding of the technology and its potential, they begin using the tablets for their own needs, such as note-taking, keeping resident files, training via videoconferencing, online meetings, and care planning using specific apps.
However, not all staff members find the iPads intuitive, and there are concerns about the usability of the technology.Such as CH1 (male, aged 39), who explains that "using iPads was a very daunting thing" at first and there needs to be training "from the staff's perspective, how they use it with residents and how we can make it more accessible to them."Due to communication issues and the heavy workload during the pandemic, the initial staff training provided by CRSCH was not well attended: It's just unfortunate that the time of the training, we were obviously in the middle of a pandemic, so it was very difficult to join on the training courses, you know.Because obviously we had to be in the forefront with the residents was the priority (CH5, male, aged 52).
Consequently, a large portion of the care home employees demand "better training" with an emphasis on "things we could be doing with it [the iPads]."The data, therefore, demonstrates that the inclusion of consumption (Heeks et al., 2013) could be improved by providing more detailed training (led by other care home staff members) on how to use the devices and what activities they can be used for.

The adopters
The adopters domain investigates the consumers of the technology, including adjustments to staff responsibilities, identities, and practises, as well as residents' expectations and assumptions (Greenhalgh et al., 2017).When CRSCH provided the tablets, they assigned a digital champion to each care home to supervise the implementation and utilisation of the iPads.However, few participants discuss the digital champion and are unaware of who holds this role in their care home.Instead, well-being and activity coordinators often oversee the usage of the tablet computers in addition to other duties, including ensuring tablets are charged, updated, and connected to Wi-Fi.Other care homes have more informal approaches as to who assists residents in using technology: No, not one in particular.We found that different residents have different relationships with staff members.So, it would…it was always better to choose the person that has a better relationship with that resident and their relatives that they're communicating with.And then that person would be, kind of, their link, rather than just one person to oversee it all (CH14, female, aged 63).
Although residents are enthusiastic when using the tablets, there is also some anxiety because of the unfamiliarity of the technology.For instance, some demonstrate fear of doing the wrong thing: "don't know what harm you do if you press a button" (R201, female, aged 65+).Moreover, cognitive disability inhibits participation since it causes decreased attention spans, issues recognising individuals on the screen, and trouble remembering how to use the devices, while sensory impairments pose hurdles to engagement.Staff, however, use tablets with residents and promote meaningful activities, which encourages engagement: That was the challenge at first, it was the actual resident, they weren't actually too sure of what was happening, why they were on this device but then, when they get used to it, it became a bit of understanding for them.And once the carer could explain things to them, what was happening in the world, then they began to understand why we had to go through this way of contact (CH13, female, aged 65).
Residents rely heavily on staff support to use the tablet computers, and few can utilise the technology without their input, which represents significant extra work for care home employees engaging with the project (Monin et al., 2020).Regarding the inclusivity of consumption, residents and staff use iPads in many creative ways.However, some residents have difficulties navigating the technology alone, which creates a further workload for the care home employees.The innovation, therefore, surpasses the second stage of the ladder, but with recommendations for improvement to staff training, roles, and workload, and to the accessibility of the technology itself.

Inclusion of impact (Level 3): The value proposition
The third level of inclusive innovation involves the innovation positively impacting the welfare, well-being, or livelihoods of the marginalised group (Heeks et al., 2013).Inclusive impact can be measured in different ways, depending on the intention of the innovation.Our data demonstrates that the value proposition dimension of the NASSS-CAT provides detailed information on the impact of the innovation on care home residents.For example, it not only evaluates the value of the service innovation to the developer (e.g. the business case) but also to the user in terms of its desirability, efficacy, safety, and costeffectiveness (Greenhalgh et al., 2017).We find that the tablet computers are utilised to support the social connectedness of residents, and this provides value for residents, their families, and care home staff: The iPads made a big difference during the pandemic.I was so happy when I was able to get in touch with my family, especially my son.During lockdown it made my life easier, a lot easier!Talked to family, played games, listened to music, and even did quizzes.Brilliant!Best thing ever (R401, female, aged 45-54).
This example from a resident explains the value of the innovation as video calls help maintain social connectedness with friends and relatives, primarily during lockdown and self-isolation periods, when inperson visits are not possible.
C. Wilson-Nash et al.Throughout the data, care home staff describe how family and friends benefit from the ability to see residents and keep in touch, especially when visits are not feasible due to lockdown.Relatives are reported to find the ongoing communication comforting because it allows them to observe the residents' health and well-being: In the pandemic when there wasn't so much visiting, you could take pictures and show it to family members or email it to them to let them see what activity they were taking part in.So, it helped put their minds at rest, that they were actually still, you know, able to do things, and they looked settled and happy when they couldn't see them so much (CH16, female, aged 40).
Moreover, there is a positive impact on staff members, as they are presented with additional ways to engage with residents.Their work becomes more satisfying as they see improvements in the residents' interactions with family and engagement with activities.For instance, the following example indicates how difficult it is working in the care home sector, but seeing a resident connect with a family member can be so rewarding: It's meant a lot to us as well, so it's lifted our spirits when you see a resident talking to their family member, you know.It's not been an easy two years for any of us working in care homes and to see a son or a daughter speaking to their mum and they're smiling or they're crying with joythat makes us feel good as well and actually helps us (CH3, female, aged 50).
In addition, the CRSCH value proposition of enabling residents to contact family members during the pandemic was achieved and has transitioned into creating new benefits for both residents and employees in care homes.The various activities they now engage with provide satisfaction, a better mood, more physical and mental exercise, and frequent socialisation within the care homes.The following example conveys how an iPad can be used in a one-to-one with a resident to improve mood and well-being: So, with a one to one, if there's someone that's maybe having a hard day, we can take the iPad to them and sit with them and play music, show them pictures, reminisce with them, which is great, because you actually can do it and let them slide the screen and touch the buttons, and it's amazing, amazing how they love that (CH10, male, aged 54).This evaluation establishes that despite the consumption issues, the innovation positively influences the well-being of the care home residents, as a marginalised group, as well as family, friends, and care home staff.The inclusivity of impact, as a step on the ladder of inclusive innovation (Heeks et al., 2013), is assessed through the value proposition in the NASSS-CAT (Greenhalgh et al., 2017).The value proposition is achieved as residents are successfully connected with friends and family during the pandemic through innovation, which evolves to enhance well-being further through additional and creative activities, consequently enhancing the inclusivity of impact (Greenhalgh et al., 2017;Heeks et al., 2013).

Inclusion of process (Level 4): The organisation(s)
The fourth stage on the inclusive innovation ladder is the inclusion of process involving the marginalised group within the innovation development process (Heeks et al., 2013).To assess this for the CRSCH project, we use the organisation dimension on the NASSS-CAT.This dimension interprets the capacity of the organisation(s) to innovate in general, readiness for the technology, nature of adoption, extent of change needed for organisational routines, and the work needed to plan, implement, and monitor change (Greenhalgh et al., 2017;Greenhalgh et al., 2020).
Our data establishes that due to the reactive but necessary implementation of the iPads, care home staff and residents are not involved in the process of innovation, which leads to issues in implementation.For instance, some care homes possess less technological readiness than others, lacking WiFi access and limited experience of using technology previously.The following example indicates how the lack of WiFi can cause problems whilst staff and residents are attempting to use the tablet computers: Our Wi-Fi is not as good, so sometimes we take it into residents' rooms and the internet just cuts off.In certain places there's a wee bit of a kind of blind spot for the internet that's the only thing that really is a kind of hindrance sometimes for using iPads (CH1, male, aged 39).Furthermore, the uptake and utilisation of innovation are very much dependent on the workplace culture.According to interview data, enthusiastic managers are crucial in overseeing and supporting innovation and motivating employees.To facilitate the usage of tablet computers, teamwork between managers and staff members (such as activity and well-being coordinators) is essential: And management is very supportive…she's always been very supportive with the well-being team to get more.She's delighted with what we do, she's really pleased with our work…as long as the residents enjoy it and they're happy, that's what we come in every day to do…and it's not everywhere else, you've got some carers that are interested, and some aren't.But yeah, on the whole, there's a team of us of five, so we kind of bounce off one another (CH10, female, aged 56).
This often leads to cultural changes as staff members become more comfortable using and integrating the technology into routine activities.For instance, some care home staff are initially "sceptical and wary" about using iPads.However, as the organisation's culture changes, they begin to "build up their confidence" (CH2, male, aged 22) and use the technology for many different purposes, such as quizzes and music for residents.
In other cases, technologies are likely to be abandoned in care homes where specific staff members (e.g.activities and well-being coordinators) are responsible for their usage.The following example indicates how DDCH2 is in a vulnerable position because their iPads are only used by well-being coordinators and not embedded within the daily routines of other staff members: Basically, it is the well-being coordinators who use the iPads, and they use them the way that they think is best.There isn't really any one particular person within our company that checks whether they are being used to their best effect.They have busy schedules every day and they use them where they can (CH1, male, aged 39).
In these cases, where activities and well-being coordinators are solely responsible for the iPads if they leave employment, the technology is often abandoned or left in a drawer.This becomes an issue in a few care homes, mainly because of the high staff turnover.
Using NASSS-CAT enables data to be collected on the management, changing workplace cultures, and the readiness of the organisation(s) for technology-enabled innovation (Greenhalgh et al., 2017;Greenhalgh et al., 2020).This dimension reveals the readiness of the care homes for innovation and whether residents and staff are involved in the innovation development process (Heeks et al., 2013).Due to the reactive nature of this innovation, which is instigated rapidly during a pandemic, residents and staff are less involved in the development process but are more involved in the use and evolution of the innovation to be utilised for different purposes.

Inclusion of structure (Level 5): The wider system
The aim of the fifth stage of inclusive innovation is for the innovation to be embedded within an inclusive structure, which means that the institutions, organisations, and stakeholders involved in the innovation system are all-inclusive (Heeks et al., 2013).The wider system domain of the NASSS-CAT provides a good platform for this evaluation as it involves political, economic, and regulatory/legal issues, professional bodies, sociocultural factors, and inter-organisational networking that influence the service innovation in health and social care (Greenhalgh et al., 2017).
The CRSCH goals are consistent with the Scottish Government's broader commitment to promoting digital inclusion and social connection, increasing access to digital care and digital opportunities, and lessening the pandemic's effects.According to data from the stakeholder workshop, these goals are accomplished through enhancing "familiarity with digital devices being utilised within care homes," "connecting residents with family," "using iPads for activities," and "continuing the digital journeys of care homes'" (SH1) by exposing personnel to new digital options.The availability of tablet computers for the care homes arrived at a critical juncture, which was warmly welcomed by the care home staff, who also praised the Scottish Government for their innovative solution: During lockdown, a lot of time, it felt as if we were just on our own.So, when we were getting iPads and when we were getting the data, it genuinely felt that people were, government, whoever, they knew we were here.They knew our residents were here and they were wanting to help.We're a stand-alone single charity so for us the fact that we could have this technology at no cost to the charity was a Godsend to be quite honest.So, it made us feel as if, right, we're not being forgotten about (CH3, female, aged 50).
There are, however, barriers to effective communication between care homes and the larger policy and practice setting.Stakeholders indicate, for instance, that the CRSCH plans create learning and support forums to establish and maintain ties with care home employees.Establishing such connections, nonetheless, presents difficulties because staff members are not always available for training, and developing communication takes time.In summation, the innovation achieves inclusion of structure (step 5), as the wider system is inclusive of the marginalised group, especially during times of crisis.However, some improvements could be made to stakeholders formulating relationships with care home staff and residents, as there currently appears to be a lack of connection.

Post-structural inclusion (Level 6): Embedding and adaptation over time
The final level on the ladder of inclusive innovation is post-structural inclusion, where the innovation should be embedded within inclusive knowledge and discourse, comprising inclusive language (Heeks et al., 2013).The NASSS-CAT domain of embedding and adaptation over time is, therefore, equipped to explore society's overall inclusivity, how much scope there is for adapting and coevolving technology and service innovation, and the resilience of the organisation(s) to handle critical events and react to unforeseen circumstances (Greenhalgh et al., 2017).
With CRSCH, tablet computers successfully support social connectedness during the pandemic, however, as the situation changes over time, the use of the technology also changes.There is now less focus on video calls and more on entertaining and communal activities, and consequently, iPads are embedded within the care homes.The subsequent illustration specifies how the use of the iPads evolves and that their introduction alters and embeds technology-based communication between residents and family members forevermore: Maybe slightly less Skype calls because more of those people are able to come in, but there are a lot of relatives who are not able to come in because of their own health or live in England and live further away.And it's just become the norm to do Skype calls for them.So, I think the iPads really are still used just about as much (CH14, female, aged 63).
Consequently, most care homes predict that they will change the discourse and continue to use the tablets in the future as they are still learning about their use and potential.Only a few care homes indicate stopping using the tablet computers, either permanently because it "seems to have run its course" (CH7, female, aged 57) or temporarily because of continuous personnel or management changes.
The introduction of iPads into care homes changes the discourse of care home services being digitally devoid and residents being digitally and socially excluded.Therefore, in most care homes, post-structural inclusion is achieved (Heeks et al., 2013) as the tablets are now embedded in the broader activities of the home, and this encourages further technology usage, which improves the well-being of residents and consequent discourse of digital and social inclusivity.For example, DDCH3 and DDCH4 demonstrate the tendency to adopt new technology with an emphasis on improving accessibility.They use a larger tablet, the size of a foosball table, to combine the functionality of a tablet with the advantages of a larger screen.Smaller tablet computers are preferred for individual use, while table-sized tablets are used in common areas for group activities such as karaoke and fitness classes.DDCH4 also uses VR technology to allow residents with limited mobility to go for virtual walks while remaining in their rooms.The tablets are now part of the broader repertoire of technological resources available to residents.However, the inclusive discourse surrounding staff in care homes (e.g.activities and well-being coordinators) is lacking, and there needs to be more support on using, embedding, and maintaining these technologies.

Evaluation framework for inclusive innovation
To address the gap in the inclusive innovation literature on the evaluation of the process (Mortazavi et al., 2021), we map the seven NASSS-CAT domains (Greenhalgh et al., 2020) onto the six stages of the ladder of inclusive innovation (Heeks et al., 2013(Heeks et al., , 2014) ) demonstrating how this theoretical synthesis can be used to measure the inclusivity of technological innovation in social and health care services (see Fig. 3).Moreover, previous evaluations of social and health care service innovations in the business literature focus on metrics such as satisfaction (Lotfi & Soleimani, 2020), customer perceived quality of care (Wu & Hsieh, 2011), and improved communication (Campanella et al., 2021) to assess the success of the intervention.By adopting this approach, success is perceived less in economic terms and more in relation to the well-being of the marginalised group during scale-up, spread, and sustainability of the innovation.The following discussion explains how the ladder of inclusive innovation can transform from a heuristic to an evaluation framework (Smith et al., 2023).
First, to proceed to Level 1, the innovation should address the intentions (e.g.values, needs, and wants) of the marginalised group (Heeks et al., 2013(Heeks et al., , 2014)).To evaluate whether this has been achieved within health and social care services, the first NASSS-CAT domain can be used to assess the situation, condition, or illness that the intervention is targeting (Greenhalgh et al., 2020).This goes beyond exploring the values of marginalised groups, such as vulnerable older consumers and consumers with disabilities in the marketplace (Baker et al., 2005;Elms & Tinson, 2012;Wilson-Nash, 2022), towards understanding the conditions, illnesses, and situations that they might be experiencing and whether an innovation intends to address these circumstances.For example, in our data, the innovation intends to improve the social isolation of care home residents by introducing digital communication with family and friends, therefore exhibiting an inclusivity of intention.
To assess whether the marginalised group consumes the innovation (Level 2; the inclusion of consumption; Heeks et al., 2013Heeks et al., , 2014)), the technology and adopters NASSS-CAT domains can be implemented to develop in-depth insight into how the technology is used, examples of good practice, employees as adopters, influence of family and friends, and consumption by the excluded group (see Fig. 3).The extensive qualitative data gathered by the NASSS-CAT can, therefore, offer invaluable examples of innovative uses of technology, continuous consumer demands, and changes in consumption over time.This approach extends previous service innovation literature, which either evaluates the social and health care innovation from the perspective of the consumer (Wu & Hsieh, 2011) or the perspective of the staff member (Obayashi et al., 2020).The NASSS-CAT, alternatively, encourages flexible data collection from all adopters to develop essential observations on the consumption of innovation and create recommendations for the intervention moving forward.
To assess the inclusion of impact (Level 3), which is whether an innovation positively influences the well-being of the marginalised group (Level 3; Heeks et al., 2013Heeks et al., , 2014)), the value proposition from the NASSS-CAT can be adopted to measure the value of the technology to everyone involved (see Fig. 3).Our findings reveal that when the originally expected impact is met, it sometimes transitions to other outcomes in practice.This advances the current understanding of a value proposition in the service innovation literature (Skålén et al., 2015), where it is often assumed that meeting the value proposition is a measure of success for the innovation and its consumers (Cullen, 2008).In this case, the value proposition was achieved during the pandemic, but since restrictions have changed, the technology is no longer being used for communication, and instead, users are finding new value in the technology (e.g. for entertainment).The shift is only recognised due to the qualitative evaluation of the NASSS-CAT framework and might have been missed (or even seen as a failure) by many quantitative evaluation frameworks.
To evaluate the inclusion of process (Level 4; Heeks et al., 2013Heeks et al., , 2014) ) and how involved the marginalised group are in the innovation development, the organisation domain from the NASSS-CAT can be used to understand the capacity and readiness of the organisation to innovate, as well as the practices required to change (see Fig. 3).This helps us understand how involved the excluded groups (e.g.care home residents) are in the process of innovation.Level 5, the inclusion of structure (Heeks et al., 2013(Heeks et al., , 2014)), can be evaluated by the wider system domain of the NASSS-CAT, which assesses the government organisations, professional bodies, non-profit organisations, and sociocultural factors that influence the service innovation (Greenhalgh et al., 2017) and whether they are inclusive, open to changing policies, improving corporate social responsibility, and providing education and training to marginalised groups.Finally, NASSS-CAT's adaptability in data collection and analysis enables the innovation's future viability to be evaluated (e.g.embedding and adaption over time), which provides an assessment of post-structural inclusion (Level 6; Heeks et al., 2013Heeks et al., , 2014)).The approach invites academics in social and health care services to consider the future of innovation and how to alter discourses of inequality rather than only evaluating the success of what has already occurred.
We contribute to the inclusive innovation literature by providing an evaluation framework that can be used to assess the inclusivity of technological innovation in various social and health care services (e.g.hospitals, care homes, and home care).Evaluations of inclusive innovation are sparse and often quantitative (e.g.Andries et al., 2019), which is why this research has developed a flexible evaluation framework that can incorporate both qualitative and quantitative data collection using the NASSS-CAT methodological tools, be utilised before, during, or after the innovation, and includes the marginalised group in the development process (Greenhalgh et al., 2020).This further contributes to the inclusive innovation literature, as we demonstrate that inclusivity is essential for marginalised groups outside LMICs (Mortazavi et al., 2021).Previous frameworks, such as the ladder of inclusive innovation, when synthesised with an evaluation framework, can be effective in assessing the level of inclusivity of novel technology in social and health care services within HICs (Heeks et al., 2013;Heeks et al., 2014).

Managerial and policy implications
Our findings have significant ramifications for managers and policymakers who direct and launch inclusive innovations in social and health care.For successful inclusivity of consumption (Level 2; Heeks et al., 2013Heeks et al., , 2014) ) by the adopters, it is crucial to positively affect the existing culture in social and health care services, especially as management and staff attitudes towards technology are crucial for full acceptance of the devices.For instance, one approach to achieving this is identifying, fostering, and expanding the activity/well-being coordinator position in care homes.The technology and how to use it with residents is often the responsibility of these members of staff, who also coordinate social, physical, and other engagement activities.Not all care homes contain or foster these positions.For those who do not, encouragement should be provided to formalise them and include technology usage in the activities/well-being coordinator role.Protected time and the appropriate tools for the task would be needed for this to happen (for example, they would need to be listed in job descriptions and role profiles).
Furthermore, social and health care services should provide support in the form of training, made available through continuing professional development, particularly as technologies change.For example, in care homes, all personnel should ideally participate, even though activities and well-being coordinators bear the bulk of using technology with residents.Therefore, training should concentrate on two things: 1) technology fundamentals to guarantee that staff members with different degrees of digital literacy are included.It might discuss how well devices work, accessibility problems (such as those involving dementia or sensory impairments), and solutions to those problems.2) Instruction in innovative and user-centred technology use.Staff members who oversee creating novel initiatives to improve the lives and wellness of their residents should receive this training.
To achieve inclusion of process (Level 4: Heeks et al., 2013Heeks et al., , 2014)), the marginalised groups should be involved in developing the innovation, from cradle to grave.In our example, this means having representatives from care homes (staff and residents) in meetings to decide on the innovation, how the innovation is to be implemented, and how to overcome barriers and challenges.It also means that residents should be involved in developing any technologies to avoid issues with accessibility.With CRSCH, residents and employees were less involved in the development phase of this innovation because it was implemented rapidly during a pandemic and subsequently faced several barriers (e.g.Badawy et al., 2022;Fearn et al., 2021;Monin et al., 2020;Schuster & Cotten, 2022), due to the demands of COVID-19.However, they were involved in its use and evolution, using the technology for many purposes and demonstrating its potential.Future innovations must cocreate this effort with members of the marginalised group(s) for it to ultimately become an inclusive innovation, which would alleviate concerns with preparedness and uptake.
Finally, to enable post-structural inclusion (Level 6; Heeks et al., 2013Heeks et al., , 2014) ) with embedding and adaptation over time, initiators of innovations in social and health care services should provide a more personalised, flexible, and person-centred approach to the implementation of technology, whilst acknowledging the trade-offs involved in the implementation process (e.g.hidden work for employees).For example, every care home has important and distinctive needs and varying degrees of technology adoption; some care homes are more independent in their digital journeys, while others need more assistance.Consequently, introducing additional technology is not always the answer for some care homes.However, for care homes where technology could be helpful, it should be introduced in a supportive and inclusive manner by assisting individual care facilities in locating, acquiring, and utilising devices that are beneficial to their needs and the needs of the residents (e.g.voice-activated curtains).This more personand organisation-centred approach can tailor technology selection and allow care and health facilities further discretion over decisions.

Limitations and directions for future research
Even though our analysis includes a clear sample of care homes from the CRSCH project, our findings are not entirely indicative of the whole Scottish care home industry, as recruiting care homes for research is extremely difficult (Ellwood et al., 2018).To mitigate this, we used a purposive sample, inviting variances in geographic location, deprivation, and types of care homes.However, our assessment is concentrated on care homes that provide a service to older people who have dementia or frailty.As a result, Scotland's network of care homes that provide care for different populations, such as younger individuals, is underrepresented.Future research could evaluate inclusive innovations in care homes for younger people, as these facilities are likely to have residents who use digital technologies in different ways due to a range of educational needs or requirements for access to health and social services.
Our research demonstrates the value of synthesising the NASSS-CAT with the ladder of inclusive innovation to evaluate the inclusivity of service innovation in health and social care.Our evaluation framework contributes to the extant literature on service innovation and inclusive innovation and demonstrates how evaluations can utilise mixed methods to be extensive, flexible, and overarching.There is, consequently, scope for our framework to be adopted to evaluate the inclusivity of innovation in further health and social care services, such as artificial intelligence in hospitals, wearables monitoring patients, and smart speakers for in-home care (Cresswell et al., 2020;Hamblin, 2022).

Fig. 3 .
Fig. 3. Evaluation framework for inclusive innovation in social and health care services.

Table 1
Care home staff interview participants.

Table 2
Stakeholder workshop participants.

Table 3
Deep dive care homes and participants.
AdoptersThe users of the technology, including changes in staff roles, practices, and identities, expectations of the residents, and the role of family and friends.The value of the innovation to the developer, organisation(s), and users in terms of its desirability, efficacy, safety, and cost-effectiveness.The capacity of the organisation(s) to innovate in general, readiness for the technology, extent of change needed for organisational routines, nature of adoption and/or funding decision, and the work needed to plan, implement, and monitor change.