How to improve success of technology projects in health and social care September 2018, Volume 28, Issue 3

Introduction

Technology is widely viewed as a major part of the solution to the growing challenge of providing health and social care to an ageing population. But despite significant investment and high expectations, five problems persist (abbreviated as NASSS): digital technologies are either not adopted or soon abandoned by professionals and/or their patients and clients, or the technology-supported service succeeds as a small-scale demonstration project but fails to scale up locally, spread to other comparable settings or be sustained over time.

My team has been studying failed (and, more rarely, successful) public sector technology projects for many years. We developed a framework for teasing out the multiple interacting influences that can potentially derail such projects. The framework is based on a narrative systematic review of the literature¹ and a series of six new case studies, followed using mainly qualitative methods for 3 years.² Our case studies included both ‘heavy’ technology such as major new organisational hardware and ‘light’ technology such as smartphone apps, and covered a very wide range of service models and settings. Three of the case studies were in healthcare (video consultations via Skype and FaceTime, remote monitoring of heart failure patients, and risk analytics for estimating the chance of hospital admission in patients with multimorbidity) and three in social care (pendant alarm services, Global Positioning System [GPS] tracking for people with cognitive impairment who ‘wander’, and care-organising apps for families to coordinate the social care support of a dependent relative).

In these case studies and our previous empirical work, we have found that planners and policy makers are often overly focused on technologies, and distracted by simplistic models and metaphors of technology adoption by individuals (e.g. ‘tipping point’). They have paid scant attention to the dynamic sociotechnical system into which new technologies and care practices must become embedded.³ This system has seven interacting domains, which form the basis of the NASSS framework and are described briefly in this paper. Each domain (and the subdomains within it) can be simple (few components, predictable – as in making a sandwich), complicated (multiple components but still largely predictable – as in building a rocket) or complex (dynamic, composed of multiple interacting elements and unpredictable – as in raising a child).

Seven domains where complexity lies

Complexity – that is, the extent to which a phenomenon is dynamic, unpredictable and interdependent on other phenomena – is a feature of many headline problems facing healthcare planners and policy makers.^2,4 Multimorbidity and interacting sociocultural influences mean that the patient in the guideline does not correspond to the patient in the bed. Super-science cures are ubiquitous in the media but the old lady in the tower block cannot get to see her family physician. New staff roles and service models sometimes seem to worsen the very problems they were introduced to solve. The policy sacred cow of integrated care repeatedly proves impossible to deliver in practice. And so on. In among all these complex stories is usually a technology (or an idea for a technology), designed to improve data capture, hasten communication, coordinate routines or align disparate stakeholders. It rarely does – at least not without more blood, sweat and tears than originally envisaged.

The NASSS framework is designed to identify and explore where complexity lies in a technology project. It considers seven domains (see Figure 1):

Domain 1: The illness or condition. A simple illness (e.g. sprained ankle) is well characterised and has a clear diagnostic and treatment pathway, so management is predictable and consistent. A complicated illness (e.g. cancer) requires logistical coordination but is still predictable to manage. A complex illness (e.g. psychosis, complicated by drug dependency and hepatitis, in an asylum seeker who speaks limited English) is unpredictable and not amenable to management by algorithm.

Domain 2: The technology. A simple technology (e.g. the telephone) is dependable, freestanding, cheap and substitutable (meaning that if a manufacturer withdrew from the market, you could easily get another one that would do the same job). It is also well designed for the task (including attention to ‘human factors’⁵), and it generates data that is easy to interpret and clearly reflects changes in the patient’s condition. A complex technology is one that is intended to be widely interoperable across multiple organisations and sectors, or which is less dependable (e.g. keeps crashing) or does not yet exist. And it may generate data that clinicians cannot interpret or do not trust, and/or whose provenance or intellectual property is contested.

Domain 3: The value proposition. Technologies in development have supply-side value (potential for return on investment) and demand-side value (benefits to patients and the healthcare provider).⁶ A simple value proposition has a robust and well-justified business case and demonstrable benefits in terms of health technology assessment (HTA) studies (efficacy, safety and cost-effectiveness). In a complex value proposition, the business case for developing the technology is implausible or rests on unverifiable assumptions, and/or the results of HTA are unavailable or contested.

Domain 4: The intended adopters. The most common reason for program failure in healthcare is that clinicians do not use the technology.⁷ In a simple situation, intended users (staff and patients/clients and their carers) are able and willing to use the technology and easily trained to do so; using the technology does not upset or threaten them. In a complex situation, intended users lack the capability or willingness to learn to use the technology – perhaps because the technology represents a threat to their professional identity or scope of practice (staff), or symbolises a stigmatising illness (patients/clients).

Domain 5: The organisation(s). In a simple situation, all participating organisations have high capacity to innovate, high tension for change, a good innovation–system fit, formalised links with partner organisations (e.g. an existing subcontract), and a budget that can be allocated to set up and monitor the new technology-supported service. Introducing the technology aligns well with existing work routines, which are readily adjusted to accommodate them. Things get complex when one or more participating organisations lacks the capacity to innovate, does not wish to change, lacks a partnership agreement or has no available budget – or when the technology requires significant disruptive changes to hard-wired organisational routines.^8,9

Domain 6: The wider system. If this domain is simple, there will be a clear policy push for the technology to be introduced, with relevant levers and incentives, and the intended adopter(s) will know and accept that the technology has legal and/or regulatory approval, that it is endorsed by their professional body and that using it is the accepted thing to do.⁶ Adopting organisations will share knowledge with other adopting organisations. If the domain is complex, there may be a top-down directive to change but no funding, regulatory or professional bodies may not yet have taken a position on an aspect of the problem, or inter-organisational networking and support may be weak.

Domain 7: Evolution over time. In a simple situation, the technology and the care pathway are (to some extent) adaptable and future-proof, and the organisation has a high degree of resilience to external shocks and setbacks. In a complex situation, the technology and service model are implemented rigidly and mechanically by an organisation that lacks resilience. In the latter case, even if a program is successfully implemented in the short term, its long-term survival is unlikely.

Complexity in each of the domains of the NASSS framework can be both logistical (relating to such things as scope, scale, deadlines and resource constraints) or sociopolitical (relating to attitudes, feelings, conflicts of interest or historical path dependencies). Complexity may also be an emergent feature of a changing system (e.g. a small-to-medium sized technology provider that gave an agile response to requests for modifications to the technology may be taken over by a large commercial company whose business model is to stick rigidly to contracted transactions).^4,10

Figure 1.     The NASSS framework: multiple interacting domains affecting the adoption, nonadoption, abandonment, and barriers to scale-up, spread and sustainability of health technologies (click to enlarge)

How the NASSS framework can improve success

Many, if not most, problems in contemporary healthcare are inherently complex; for example, we can’t tell patients not to have multimorbidity or low health literacy, we can’t tell healthcare providers or professional bodies that information governance is a minor issue, and we can’t dictate the pace or price for new software development. How then can the NASSS framework help us improve the success of technology projects?

First, those developing new technologies or seeking to implement them in a healthcare setting should resist the temptation to address an oversimplified, abstracted and deconstructed version of the problem. Technology developers and those who plan to use technologies need to acknowledge and explore complexity in all seven domains of the NASSS framework. We have developed a NASSS complexity assessment tool (NASSS-CAT) based on the domains above (www.phc.ox.ac.uk/files/research/nasss-cat-tool) to help generate a narrative of where the key complexities lie.

Second, it is crucial to try to reduce complexity wherever possible. It is important, for example, to consider carefully the trade-off between nice-to-have features (such as interoperability across multiple domains) and the potential knock-on effects of such interoperability in other parts of the system.

Third, there are principles that can help a project team respond adaptively to complexity (e.g. as the sociotechnical system evolves over time). Try these 10 simple rules (adapted from Maylor and Turner¹⁰) for managing technology projects in complex systems:

1. Strengthen program leadership (which may be distributed across the project and across contributing disciplines)
2. Codevelop an overall vision for the project, and maintain dialogue around that evolving vision
3. Nurture key relationships between individuals and organisations
4. Develop individuals, and encourage them to solve local problems creatively
5. Make resources available for creative individuals and teams to use for generating solutions to local challenges
6. Capture data on progress and feed it into ongoing deliberations
7. Acknowledge and address the concerns of front-line staff
8. Work with intended users to codesign technologies and the work routines they are intended to support, building in adaptability
9. Control scope creep
10. Address regulatory and policy barriers.

In sum, our findings suggest that the overarching reason why technology projects in health and social care fail is multiple kinds of complexity occurring across multiple domains. Oversimplifying the challenge, or ignoring the uncertainties and interdependencies, won’t help. We need to shift gear and learn to run with complexity.

Our team is working to refine and test the NASSS-CAT tool on a new set of technology implementation case studies. We hope it will help planners and policy makers select and manage projects, including systematically weeding out low-value technologies and high-risk ventures at an early stage and improving the success rate of projects worth supporting.

Peer review and provenance

Externally peer reviewed, commissioned.

Copyright:

© 2018 Greenhalgh. This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Licence, which allows others to redistribute, adapt and share this work non-commercially provided they attribute the work and any adapted version of it is distributed under the same Creative Commons licence terms.