Innovation tactics for implementing an ML application in healthcare: A long and winding road

Artificial intelligence techniques, including machine learning (ML), have shown remarkable test results over the past decade but struggled with the transfer to practical application. The present study applies action research to investigate this last stage of a project to implement an ML algorithm for predicting no-shows at a Danish hospital. We approach the implementation of the no-show algorithm as an innovation process and identify 14 tactics that were employed to provide the innovation necessary at the implementation stage. The tactics span three analytic levels – organization, project, and practice – and alternate between efforts to train the algorithm and to establish trust in its predictions. These efforts are interdependent, highly sociotechnical, and hence blur the boundary between technical development and organizational implementation. They also show the intricacies involved in innovating during ML implementation. Despite sustained support at the organization level, the implementation of the no-show algorithm at the practice level remained unsettled.


Introduction
Within healthcare, the use of machine learning (ML) and other artificial intelligence (AI) techniques has attracted widespread attention over the past decade as a way of deriving new benefits from the big pools of data that are continuously produced and accumulated (Panesar, 2021;Qayyum et al., 2021).ML algorithms have shown remarkable results and accuracy during the development and testing phases, for instance in automated image analysis (Poostchi et al., 2018) and the early detection of disease outbreaks (Chen et al., 2017).However, the results have often been confined to the experimental stage and have not been possible to realize in clinical practice.This study contributes to recent research on the innovativeness required of projects that aim to apply ML in practice.
ML research focuses mainly on the technical aspects of developing reliable and accurate algorithms.This research has substantially furthered our understanding of how data can be curated, and algorithmic models built and validated.The challenges of applying ML algorithms in practice have attracted less research attention until the emergence of the discourse on the last mile (Coiera, 2019).This discourse pictures the development of AI and ML technologies as consisting of three successive and equal-size efforts: data capture and cleaning (the first mile), model building and testing (the middle mile), and real-world implementation (the last mile).Two chasms must be bridged to achieve successful real-world implementation (Cabitza et al., 2020).The first, referred to as the hiatus of human trust, concerns the challenges related to gaining acceptance of ML in practice, including the resolution of any cognitive dissonance between the system and its users.The second chasm, referred to as the hiatus of machine experience, concerns the challenges involved in ensuring reliable data in amounts sufficient to allow the proper training and continued adjustment of the ML algorithm.That is, the last mile comes with its own challenges; its success does not simply ensue from successfully completing the first and middle miles.
While the discourse on the last mile provides a conceptual framing, it says little about how people, in practice, work to implement and scale ML solutions.Yet, this work is pivotal in realizing innovations, as recognized in studies of digital entrepreneurs (Arvidsson and Mønsted, 2018) and trust in data science (Passi and Jackson, 2018).With this paper, we focus on the tactics applied by digital entrepreneurs working with the implementation of ML in healthcare.The studied tactics were applied to respond innovatively to local circumstances.Empirically, we conducted an action-research study of a project for using ML to predict and reduce patient no-shows at a Danish hospital.The project remained unsuccessful at delivering the intended reduction in no-shows but contributed, over its multiyear lifespan, to demonstrating and strengthening the hospital's commitment to ML.We analyze the events that led to this outcome.Throughout the analysis, our main focus is the project, while we treat the hospital's commitment to ML as an influential contextual issue.Thereby, we seek to answer the following research question: What tactics do digital entrepreneurs apply in striving to generate the innovation necessary at the implementation stage of ML projects?
By bringing out these tactics, we show that ML projects require innovative entrepreneurship throughout the development and implementation process, not just during algorithm development.Although the tactics are derived from a project that did not result in a lasting implementation, the project provides rich insights into the challenges experienced during the last mile of ML implementation.Because of these challenges, the project was in constant pursuit of avenues for improving its situation.The tactics are the outcome of this pursuit.They may stand out more clearly in a project that faced and overcame many challenges, thereby making the studied project well suited for an analysis of innovation tactics.Our findings contribute to research on implementation processes by documenting their dependence on innovation and identifying the tactics used in generating it.Specifically, this study contributes to research on ML implementation by shifting the focus from issues such as black-boxed performance and comprehensive digitization (e.g., Faraj et al. 2018) to innovation tactics and their application.

Theoretical background
The background for our work is the research on generating innovation potential and on implementing ML in healthcare.

Generating innovation potential
While the potential value of ML is being recognized and incorporated in the digital strategies of hospitals and other healthcare organizations, current initiatives are often driven by individuals or small groups of people.The reliance on small groups may reflect that ML is a novel technology that has not yet been fully incorporated in information technology (IT) departments and, thus, is dependent on the initiative of individual digital entrepreneurs.When these entrepreneurs are appointed the role as innovation drivers, the approach is often referred to as corporate entrepreneurship (e.g., Phan et al. 2009).However, ML initiatives may also emerge locally and be driven by staff members who self-select to develop, implement, and champion the initiative (e.g., Strohm et al. 2020).In both cases, single staff members play an important role in exploring and realizing the potential of ML, including the efforts required to navigate the many implementation barriers.Bérubé et al. (2021) identify and rank 16 barriers to the implementation of ML applications.Ranked from most to least important, these barriers are: (1) lack of understanding of the business potential, ( 2) lack of quality data, (3) lack of top-management support, (4) lack of strategic vision, (5) insufficient availability of talent, (6) uncertain return on investment, (7) lack of skills for industrialization, (8) lack of understanding of the technical aspects, (9) low volume of available data, (10) data governance issues, (11) resistance to change, (12) security and confidentiality risks, (13) technology infrastructure issues, (14) change management issues, (15) immaturity of the legal environment, and (16) ethical issues.Some of these barriers are specific to ML applications (e. g., 1, 2, 7, 8, 9, 10).Most of them are common to the organizational implementation of information systems (see, e.g., Hertzum 2021;Petter et al. 2013).For example, high-ranking barriers such as the lack of top-management support, strategic vision, and sufficiently talented people are not specific to the implementation of ML technology.To circumvent the barriers, organizations need to generate and harness innovation potential.
It is recognized as challenging for digital entrepreneurs to contribute to innovation and a large body of research has explored the antecedent conditions (e.g., Ireland et al. 2009;Sambamurthy et al. 2003).This research mostly examines the organizational structures put in place to foster entrepreneurship and the environmental pressures for doing so, but increasingly also investigates why individual members become entrepreneurs (Corbett et al., 2013;Ireland et al., 2009).Commonly mentioned reasons include organizational resource commitments, formal sanctions, and a high tolerance for failure (e.g., Burgelman 1984;Kuratko and Morris 2018).The actual processes and mechanisms through which digital entrepreneurship unfolds are far less understood.Based on the assumption that entrepreneurship must be sanctioned, most research on these processes and mechanisms takes the organization as its unit of analysis and examines how managers select projects, allocate resources, and orchestrate learning (e.g., Zahra et al. 1999).However, some studies have challenged this assumption by examining how individuals sometimes act outside formal structures to promote new ideas.For example, Jarvenpaa and Ives (1996) showed how employees used the internet as a platform for prototyping and showcasing new services without telling management.Free to explore the new technology, the entrepreneurs were able to string many, small actions together until they had progressed sufficiently to win management support.
In continuation of this, Arvidsson and Mønsted (2018) found that entrepreneurs working with digital technologies often decouple their innovation process from the organizing logic that they seek to transform.The decoupling is temporary but serves to reduce the sociotechnical complexity associated with applying technology, which tends to create not only change but also inertia (Orlikowski, 2000).Through decoupling, digital entrepreneurs create better conditions for scaling the innovation, thereby generating innovation potential.Specifically, Arvidsson and Mønsted (2018) found that innovation potential can be generated by employing four tactics: First, concealment involves hiding innovation activities from other stakeholders to reduce complexity and allow the innovation to gain momentum.Second, sequencing is the order in which an innovation is moved from one organizational setting to another with the intention of rallying the most support possible.Third, anchoring links an innovation to already existing technologies, practices, and strategies to stabilize its use and make it 'sticky'.Finally, propagating seeks to create synergies between an innovation and other developments in the organization to extend the reach or use of the innovation to new areas.
The innovation tactics are the overall analytic framing of this study.However, the tactics of concealing, sequencing, anchoring, and propagating were not developed through the study of an AI or ML innovation.The digital entrepreneur may need other tactics to implement such innovations.Therefore, we take inspiration from the four tactics identified by Arvidsson and Mønsted (2018) but remain open to the use of other tactics for innovating at the implementation stage.

ML in healthcare
The promise of ML in healthcare, and more generally, is often expressed as a capacity for providing predictive algorithms that learn on the basis of data examples (Domingos, 2015).The challenges typically discussed for such algorithms concern the consequences of their technical qualities and bypass the need for innovating at the implementation stage.In an influential study, Faraj et al. (2018) summarize four challenges that recur in discussions of the challenges associated with ML algorithms: First, their performance is black boxed in the sense that they are not based on pre-specified algorithms but on weights that are dynamically adjusted by the algorithm itself.Though it is an active research field to auto-generate explanations for the outputs produced by ML algorithms, users often struggle with these explanations (Ghassemi et al., 2021).The frequent absence of understandable explanations means that the predictions lack transparency, which is a major concern in justifying diagnoses, treatment recommendations, and other healthcare decisions (He et al., 2019).Explanations and transparency are key to establishing confidence in algorithms and, thereby, bridging the hiatus of human trust (Glikson and Woolley, 2020;Lee and Cha, 2023).
Second, the algorithms rely on comprehensive digitization for robust performance.Electronic health records (EHRs) have been associated with unprecedented opportunities for improving healthcare through the reuse of EHR data in, for example, algorithms.It extends these opportunities that ML algorithms tend to maintain high prediction accuracy even on noisy data.However, several studies find that the quality of the available data is inadequate (Edmondson and Reimer, 2020;Weiskopf and Weng, 2013).The provision of sufficiently complete, correct, and current datasets often involves additional work for healthcare staff, who may be too far removed from the use of the algorithm to see the relevance of the extra work (Lee et al., 2020).
Third, the presence of promising ML algorithms and large quantities of EHR data creates a quantitative turn.This turn is most drastic when algorithms replace human decision making and less drastic when they seek to augment it by providing input to the human decision-making process.At present, the most powerful role for algorithms in healthcare is to augment human decision making (He et al., 2019).A supplementary reason for using algorithms for decision support, rather than automated decision making, is that their scope tends to be restricted to subtasks, thereby requiring a clinician to perform the remaining subtasks and 'connect the dots', that is, complete the task (Shaw et al., 2019).
Fourth, algorithms have hidden politics.One source of these politics is theimplicit or explicitvalue choices of the algorithm designers (Bailey and Barley, 2020).Another source is the data used in training the algorithms.The selection, classification, and pre-processing of these data influence the predictions subsequently made by the algorithm.Features that are over-, under-, or misrepresented in the data may introduce bias in predictions (Gianfrancesco et al., 2018).To counteract bias, algorithm predictions must be monitored to ensure that they are meaningful in the concrete clinical situation (Parikh et al., 2019).Reduced algorithm transparency complicates such monitoring.
These four challenges show that the quality of an ML application is determined by multiple interdependent factors that do not become salient, at least not fully, until after the application has entered operational use.As a result, the implementation of an ML application must deal with challenges that became known during technical development as well as with challenges that emerge during use.During use, clinicians have, for example, lacked evidence about the accuracy of ML predictions because the technology evolves so quickly that validation studies lack behind (Verma et al., 2023).They have questioned ML predictions because they are based on codified knowledge whereas clinicians' insights are based on knowhow, which is practice-based and tacit (Lebovitz et al., 2021).And they have felt demoralized because they experience a reduced sense of agency when faced with ML predictions of treatment outcomes (Thieme et al., 2023).Such experiences widen the hiatuses of machine experience and human trust by casting doubt on whether ML applications have the necessary input and produce trustworthy predictions.We contend that innovation tactics are needed to counter this doubt because remedying action must be devised during the implementation.Without innovation tactics, the implementation will likely lose momentum and eventually grind to a halt.

Method
We investigated the innovation tactics employed in ML implementation through an action-research study at a Danish Hospital.With an action-research study, we could get close to the field over an extended period of time.Furthermore, action research enabled us to investigate the practical conditions for ML implementation at a point in time where Danish hospitals were still in the process of building the in-house competences to conduct such implementations.

Research context
Patients who do not show up for scheduled healthcare appointments create inefficiencies for healthcare providers, unnecessarily block treatment slots that could have been utilized by other patients, and require rescheduling of the appointment and any follow-up appointments.Reviews of no-show rates find that they average about 23% (Dantas et al., 2018) with a range from 2.6% (Mieloszyk et al., 2019) to at least 33% (Deyo and Inui, 1980).The most commonly reported significant predictors of no-show are high lead time and prior no-show history (Dantas et al., 2018).Other common predictors include patient demographics and having no health insurance (Dantas et al., 2018;Deyo and Inui, 1980;Shuja et al., 2019).
The present study concerned the endoscopy and cardiology departments of Bispebjerg Hospital, a teaching hospital providing care for approximately 480,000 inhabitants in the capital of Denmark.Patient no-shows were a sizable problem in these departments.At the endoscopy department, 8% of the patients did not show up for their appointments, thereby leaving staff with empty hospital beds and a population at risk of having undiagnosed rectal cancers and other lifethreatening conditions.To reduce no-shows, the department experimented with interventions such as text reminders and calling all patients five days before their appointment to ensure that they remembered it and knew how to prepare for it.The medical secretaries and nurses reported an increase in patients showing up for their appointments with the time-demanding intervention of phone calls but were at the same time reporting insufficient resources to sustain this pre-appointment intervention for all patients.In this department, an accurate ML algorithm for predicting no-shows would make it feasible to sustain the phone-call intervention for the patients who were predicted to be potential no-shows.
At the cardiology department, the no-show rate was 4.2%.This department had for a long time been collecting information about why patients failed to show up for their appointments.On the basis of this information, the department had worked on improving its outreach activities, which included calling patients in advance, ensuring door-todoor transportation for patients, and sending out electronic reminders prior to appointments.The cardiology department also worked systematically to attain high data quality in its EHR recordings and should therefore be able to supply quality input for an ML algorithm.In this department, an accurate ML algorithm for predicting no-shows would free resources for other activities by enabling the staff to target phone calls and other interventions at the potential no-show patients.

Data collection
Data were collected through action research (Erro-Garcés and Alfaro-Tanco, 2020; Whyte, 1991).The first author served the double role of researcher investigating the implementation of ML at the studied hospital and digital entrepreneur tuning and implementing the ML algorithm for predicting no-shows.The former role was performed in collaboration with the other authors, who did not take part in the activities at the hospital.The latter role was performed in collaboration with members of the hospital staff.Apart from the action researcher, the main hospital contributors to the project were a senior endoscopy physician and a senior cardiology physician.Several other members of the hospital staff contributed to the project on an ad hoc basis.Bispebjerg Hospital approved and, subsequently, co-funded the project.Oral consent was obtained from all persons observed and interviewed.
The project lasted from February 2018 to June 2021.Throughout this period, the action researcher had an office at the hospital and worked there two or three days a week until the two national Covid-19 lockdowns (March-June 2020 and December 2020 -May 2021).During and after the lockdowns, most work on the project was performed remotely.In total, the action researcher spent about 300 days at the hospital engaged in activities to understand data and hospital procedures.To achieve this end, the activities were informed by methods for participatory design (Bødker et al., 2004;Hertzum and Simonsen, 2011).In addition, the action researcher performed concrete project activities such as (1) tuning the ML algorithm to improve its predictions, (2) assessing the prediction accuracy of the algorithm on test data, (3) observing the medical secretaries' work with registering no-shows, (4) meeting with endoscopy staff to share experiences from interventions to reduce no-shows, (5) assessing the data quality of the input to the algorithm, (6) seeking to motivate improved data quality, (7) meeting with representatives from departments interested in adopting the algorithm, (8) preparing materials for introducing staff to the algorithm, (9) coordinating project activities, and (10) having informal conversations with assorted project stakeholders such as the hospital data management team.
The multiyear involvement in the field made the action researcher a key participant in the no-show project and created collegial relations with the hospital staff.This way, the hospital staff came to experience the action researcher as working in the interest of the hospital on a project that required contributions from many stakeholders to succeed.In providing their contributions to the project, the hospital staff also provided information and insights that were instrumental to the research investigation of innovation tactics.The active involvement in the project was not in opposition to the research investigation but a means of obtaining quality data that would not be available to an outsider.However, the active involvement necessitated considerations about how to fulfill the research requirement of remaining detached.First, we reminded ourselves that detachment, unlike social activity, is a mental state (Anteby, 2013).That is, the action researcher could be socially close to other project contributors and mentally detached from them.Second, we provided conditions for reestablishing detachment retrospectively because it proved difficult to uphold during the interactions in the field.We temporally separated the data analysis from the data collection to give detached analysis the final say.The data analysis started about halfway through the data collection and continued for over a year after it had ended.Third, the data analysis was a collaborative activity by all four authors to provide counterweight to the action researcher's field involvement.

Data analysis
In analyzing the data, we took an interpretive approach (Walsham, 2006).To get started, the first authorthe action researcherdescribed and explained the project to the other authors, who were new to it.These descriptions and explanations continued over multiple sessions, which resembled interviews in that they alternated between questions about what had happened and reflective descriptions of events and activities.While the questions were asked from the point of view of an external analyst, the descriptions included the action researcher's reflections and fostered extensive conversations among the authors about the meaning of the activities and about their interrelations.This way, more and more details about the project were brought up and subjected to collaborative analysis.The analysis involved revisiting numerous activities and decisions that the action researcher had been involved in but not necessarily reflected upon.By discussing what, when, who, and why, we retrospectively engaged in reflection on action, as opposed to reflection in action (Schön, 1983).Some of these reflections were sensitive in that they concerned issues the action researcher felt, or came to feel, responsible for having handled suboptimally.Our main means of handling such sensitivities was to go through multiple rounds of reflection and, thereby, provide time for analytic detachment to grow gradually.Overall, we aimed for an analysis that was informed by our active involvement in the project.Simultaneously, we aimed to keep the analysis in balance by contrasting this involvement with an external analytic view on the project and a conceptual focus that emerged during the analysis.
The conceptual focus emerged through a four-step analysis process.
First, we built a timeline with the significant events in the project and extended it with other activities, decisions, and observations.For example, the high prediction accuracy in the feasibility study was a significant event.It was met with comments such as "It was really cool to see the results of your [i.e., the action researcher's] intense work with the no-show data" and generated positive interest in the project.Similarly, the efforts to improve the data quality of the inputs needed by the algorithm were an important activity.Second, we walked through the timeline to elaborate and discuss its contents.Several conceptual options were explored in these discussions and a focus on innovation tactics (Arvidsson and Mønsted, 2018) gradually emerged.With this focus, we refined the timeline by applying the concepts associated with innovation tactics to its contents.For example, the feasibility study involved the tactic of temporarily decoupling the project from the clinic.At the same time, the project invested considerable work in aligning (another innovation tactic) its activities with those in the clinic.Evidence of such alignment work became a focus for our analysis; one example occurred in a mail to the action researcher: "I don't know if you have already been informed, but for the next two months a secretary will be calling all patients who have an endoscopy appointment with the aim of reducing the number of no-shows.She calls them a week before their appointment.So, you may see a change in your no-show numbers over the next few months." Third, the emerging tactics and conceptual focus brought about a distinction among three interrelated analytic levels: the organization level, the practice level, and the project level.The organization level concerned the ML-related activities and strategies of the hospital at large.For example, we discovered that the project aligned itself with hospital strategies by developing a standalone no-show algorithm that did not "require additional licenses from the EHR vendor".Another recurrent issue at the organization level was the prospect of "scaling the no-show algorithm".We analyzed how this prospect was utilized in the innovation tactics to win management support.In contrast, the practice level concerned the clinical staff's interests and daily activities.Their main concern was resources: "We [i.e., the endoscopy department] are under a lot of pressure regarding secretary resources."By pursuing this concern in our analysis, we identified innovation tactics that aligned with the practice level by acknowledging the resource scarcity but also discovered that the resource scarcity cut some project activities short and, thereby, necessitated additional innovation tactics.Finally, the project level was the main level of the analysis and concerned the activities and considerations undertaken within the no-show project.The project-level activities to generate innovation were dynamically shaping and shaped by the organization and practice levels.
Fourth, we segmented the timeline into five phases, each covering a distinct period in the life of the project.The phases reflected our primary focus on the project level and served to structure the presentation of the analysis in the following section.

Results
The analysis resulted in the identification of 14 innovation tactics employed at the implementation stage of the no-show project.Table 1 gives an overview of the tactics, divided onto the five project phases.In the following, we analyze each phase and its tactics.

Phase 1: feasibility study
The first phase of the project lasted from February to August 2018, and consisted of a feasibility study to investigate the prospects of using EHR data for predicting patient no-shows in a Danish hospital setting.At the hospital, the awareness of such prospects had been stimulated by the recent implementation of a new EHR.On this basis, a senior physician in the endoscopy department saw an interest in housing an action-research project with a local university.The high no-show rate in the department provided a good fit with the project aim.In parallel with the initiation of the project in the endoscopy department, consultants from the EHR vendor promoted their add-on AI module.However, the hospital considered this module expensive in the light of the limited knowledge of its potential.The uncertainty about the AI module increased hospital interest in the no-show project because it involved developing a standalone ML system with simple mySQL calls to the EHR databasethereby bypassing the expensive AI module.
Over the course of the feasibility study, multiple datasets were extracted from the endoscopy department and discussed with local staff.The data quality was modest as indicated by remarks that large amounts of data were deemed "unfinished", "not in the focus of my work", or "not being asked for by my direct boss".Only 20% of appointments were complete; the remaining 80% lacked one or more pieces of information.Despite the modest data quality, the best of the tested ML algorithms (random forest) predicted 68% of the actual no-shows, and only 35% of the predicted no-shows were false positives.This result won support for the project.The hospital decided to extend and scale up the project by, among other things, co-funding the action researcher during the remainder of the project.The endoscopy department also received funding for its continued involvement, mainly to improve data quality and contribute to a larger-scale evaluation of the algorithm.
During this first phase, the project employed three innovation tactics: Decoupling: The paramount tactic was to decouple the project from the daily clinical work.This decoupling created a temporary arena for experimenting with ML algorithms for predicting no-shows.By decoupling itself from the daily clinical work, the project ensured that the experimentation could be done without the risk of harming patients.Thereby, the project wasfor the duration of the feasibility studyexempted from the rules that governed the daily clinical work and, instead, allowed the freedom to experiment with yet unproven solutions.Without this freedom, it would not have been possible for the project to identify the most effective ML algorithm for predicting no-shows and to provide data about its accuracy.
Providing proof of concept: While the decoupling served to provide the conditions for innovating, the second tactic was about gaining momentum.The feasibility study provided the proof of concept necessary for the project to attract attention and win support.Prior to the feasibility study, ML algorithms appeared promising but also hyped.After the feasibility study, the project could present evidence about the accuracy of a specific no-show algorithm trained on local EHR data and predicting no-shows for local hospital patients.With this algorithm, there was reason to believe that substantial human resources could be saved by directing phone calls and other interventions at the patients identified by the algorithm.
Aligning: The third tactic was to align the project with current interests at the organization and practice levels.At the organization level, the project played into hospital interests in becoming an organization that made innovative use of ML technology.It was a distinct asset that the no-show project developed a standalone solution, which was independent of the AI module offered by the EHR vendor.At the practice level, the project aligned with department interests in housing innovation projects and with a resource situation in which the daily work consumed most department resources.The feasibility study required few department resources.Essentially, the endoscopy department merely had to supply and discuss training and test data.

Phase 2: attempted RCT
The second phase of the project lasted from February to December 2019, and had the overall goal of testing the no-show algorithm in the endoscopy department to show that the algorithm helped reduce the number of no-shows.In contrast to the feasibility study, this goal entailed that the algorithm was in use in the department in the daily work of reminding patients about their appointments.Because of the high status of randomized controlled trails (RCTs) in the medical domain, the project group decided to organize the test of the no-show algorithm as an RCT.This decision generated considerable work.To meet the strict standards of RCTs, the project group needed to describe the test in meticulous detail, to prepare the department for the workpractice changes associated with the test, to determine the size of test necessary to obtain statistically valid results, and to secure the resources for a test of this size.In addition, the no-show algorithm had to be extended with an interface for extracting data from the EHR in real time and a user interface for the medical secretaries to interact with the list of patients predicted as potential no-shows.
In preparing the department for the RCT, the project group worked with the medical secretaries to instill practices for better data quality in the EHR recordings of patients' appointment status.It was believed that better data quality would yield more accurate algorithmic predictions.However, the medical secretaries opposed the extra workload and requested more precise guidelines for what to record.Specifically, they pointed out that the current guidelines, which were in effect across the entire healthcare region, contained ambiguous and overlapping categories.The project group and medical secretaries jointly approached the region to get more precise categories for recording patients' appointment status.When the region did not clarify matters, the project group and medical secretaries met for several workshops and devised a shorter, more well-defined set of categories.But the region did not adopt these categories.The list of categories in the EHR remained unchanged and the medical secretaries were, thus, left with a poor basis for providing high data quality about the patients' appointment status.As a result, the medical secretaries gradually stopped showing interest in the project, which started to lose momentum.While the RCT never happened, the no-show project remained a high-profile case at the hospital.For example, the project was, in early 2019, selected as an ML showcase at the hospital and kept this formal status throughout the remaining phases.
During this phase, the project employed four innovation tactics: Recoupling: To show that the algorithm was not just capable of predicting no-shows but also of helping reduce the number of no-shows, it had to be in real use.Thus, the main tactic during this phase was to recouple the project with the clinic after they had been decoupled during the feasibility study.The recoupling involved convincing the endoscopy staff, especially the medical secretaries, to change their ways of working.However, the benefits associated with the no-show algorithm would only become salient after the department had used it for some time; they were not available as evidence in convincing the staff to start using the no-show algorithm.Therefore, additional tactics were necessary to enroll the endoscopy staff in testing the algorithm.Framing: To enroll the physicians, the test was organized as an RCT.This framing immediately increased the recognition of the no-show project.As an RCT, it was not merely a technology project but a research project alongside the physicians' clinical research.The physicians recognized that the project adopted their evidence-based standards anddown the lineprovided possibilities for the department to publish the RCT results in a medical journal.For the physicians, the RCT framing justified that this phase of the project was a test.While the feasibility study provided the basis for proceeding to the RCT, the RCT was needed before the algorithm could be released as a documented improvement of department practices.
Reciprocating: To enroll the medical secretaries, the no-show project sided with them in trying to get more precise categories for recording the patients' appointment status.This tactic involved giving something to get something.By supporting the medical secretaries' effort to improve the guideline, the no-show project hoped that they would, in return, support the project by adopting the no-show algorithm in their daily work.When the effort to improve the guideline failed, the tactic also failed: The medical secretaries did not adopt the no-show algorithm.In addition, the existing guideline did not enable them to improve their recording practice.Thus, the data quality of the algorithm input remained modest and constrained the quality of the algorithm output.
Manifesting: Apart from enrolling the clinical staff, the project needed to prove itself at the organization level to maintain its funding and attract additional resources.This was achieved by making the project visible as a concrete and promising initiative to exploit ML technology.To make the project visible, the project group for example participated in the highly regarded science day at the hospital.For this annual event, staff from the clinical specialties submitted posters about their ongoing research for presentation to the hospital directors and a board of clinical researchers.Through this tactic, the no-show project established itself at the organization level where it came to be seen as a manifestation of the promises offered by ML technology.

Phase 3: diversification
The third project phase revolved around the increasing disconnect between the organization-level status of the no-show project as an ML showcase and the loss of momentum experienced by the project in the endoscopy department.This phase started in January 2020 and lasted a year.For much of this phase, the project was in a state of limbo.Activities in the endoscopy department had grinded to a halt and possibilities for continuing the project elsewhere had not yet appeared.To create such possibilities, the project group asked hospital management to assist in relocating the project to another department and produced a project description for use in meetings with potentially interested departments.Eventually, a senior cardiologist heard about the project and saw a possibility for linking it with a large project about reaching frail elderly patients diagnosed with atrial fibrillation.This project had been struggling with patient recruitment and wanted to use the no-show algorithm for identifying frail elderly patients for recruitment.The cardiology department was a good match for the no-show project because it had long been working systematically with data quality and with collecting no-show reasons.
In parallel with the search for a new department, the no-show algorithm was improved.The improvement became possible after an upgrade of the EHR database opened for more advanced AI algorithms, including deep neural networks, to access the database.First, the noshow algorithm was modified to make use of deep neural networks and its improved accuracy was demonstrated on test data.Then, the improved algorithm was connected to the EHR database and made available to the cardiology department.The modification and test of the no-show algorithm could be accomplished by the no-show project group.In contrast, the real-time access to the EHR database could only be accomplished by collaborating closely with the IT department.To make this collaboration happen, the no-show project needed the support of hospital management.The status of the project as an ML showcase was instrumental in obtaining this support.
During this phase, the project employed three innovation tactics: Hibernating: In the first part of the phase, the project lay low.While this hibernation tactic was a response to adverse developments, it served two purposes.First, it created time for attracting a new department that was a good match for the no-show project.Second, it served to avoid calling management attention to the project during a period without progress.The hibernation tactic was possible because the project had previously earned the status of being an early and promising research initiative about the exploitation of ML technology.This status meant that the project at this stage operated under fairly lax deadlines.
Diversifying: The relocation of the project to another department was not an easy decision to make.It involved adopting a diversification tactic.That is, it involved undoing the work that had been done to fit the project to the endoscopy department and engaging in work to show the applicability of the no-show algorithm in other departments.The diversification was not simply the substitution of one department for another; it meant exploring a broader set of uses for the algorithm than originally envisaged.Relocating to the cardiology department was evidence of such exploration.The cardiology department wanted an algorithm for patient recruitmenta purpose similar, but not identical, to preventing no-shows.
Honing the algorithm: To improve the prediction accuracy of the algorithm, the project upgraded it technically.This tactic exploited more advanced algorithmic approaches as well as new technological developments in the EHR.While the resulting improvement in prediction accuracy was itself valuable, the most important outcome of the honing tactic was the indirect effect of showing hospital management the potential of opening the EHR database to more advanced AI algorithms.This indirect effect informed organization-level discussions about investments in technological developments to enable the exploitation of AI and ML technology.Thereby, the project amplified the manifestation tactic from the previous phase and renewed its goodwill at the organization level.

Phase 4: pilot implementation
During its fourth phase, February to June 2021, the project made progress in the cardiology department but faced increasing hesitation from hospital management.This situation was a reversal compared to the previous phase, during which the project had management support but lacked a department to work with.Concretely, the project had difficulty securing the resources for supporting its activities in the cardiology department.Most critically, the IT department was allocated fewer hours to support the no-show project.With fewer hours for customizing the data exchange between the algorithm and the EHR, the project could only meet some of the requests made by the department, which was ready to move forward.On their part, hospital management pushed for the project to document its results as input to a decision about whether to abandon the no-show algorithm or start implementing it across the hospital.
The phase culminated in a pilot implementation that ran for three weeks.During this period, the no-show algorithm flagged the patients who were likely not to show up for their appointments.Three of the cardiology secretaries responsible for scheduling patient appointments had access to this information in their work.On this basis, they provided feedback at the end of the pilot implementation.First, they were unconvinced by the performance of the algorithm.Too often its predictions did not match whether patients actually showed up for their appointments.Second, they requested that the algorithm should supply additional information, such as the patient's phone number, a direct link to the patient's record in the EHR, and a percentage indication of the patient's no-show likelihood.This information would make it easier to assess and act on the predictions.The outcome of the pilot implementation was that the secretaries declined to use the current version of the algorithm.The pilot implementation also renewed discussions about data quality and the data exchange between the algorithm and the EHR.
The project employed three innovation tactics during this phase: Sequencing: To win renewed support at the organization level, the project focused on moving forward at the practice level.With this tactic the project tried to make the most of its collaboration with the cardiology department.First, this collaboration would be used to make progress in a department that was already motivated.Then, this progress would be used to renew the support from hospital management, which had become hesitant.It was a challenge for the project to sustain the collaboration with the cardiology department because the reduced management support meant that project resources were scarce.Thus, the sequencing tactic was a temporary measure intended to make enough progress to get more resources.
Presupposing: To make progress in the cardiology department, the project members acted as though resources were available.Specifically, they met with the cardiology and IT departments to discuss which additional EHR data the IT department needed to supply to the no-show algorithm to satisfy the cardiology secretaries' needs.These meetings boosted the local belief in the algorithm and clarified the needs for data exchange.They also served as a forum for gently trying to talk the IT department into using any slack resources to develop these dataexchange possibilities.It was an uncertain tactic to presuppose that another department would commit slack resources to the no-show project to replenish the resources formally allocated to it.However, the presupposing tactic produced enough progress to secure continued support from the cardiology department at hospital board meetings, thereby complementing the sequencing tactic.
Jumpstarting: The no-show project needed to make visible that it was making progress.Without visibility, the progress would have little effect.That is, the project might lose momentum in the cardiology department and fail to win renewed support at the organization level.To this end, a pilot implementation was conducted.Compared to the RCT attempted in the second phase, the pilot implementation was much less work because it did not aim for formal evidence that the algorithm was effective.Instead, it aimed to collect practical experience with the algorithm and formative feedback about its performance.While the secretaries' experiences from the pilot implementation were valuable input to the project, it was unexpected that the algorithm performed poorly.

Phase 5: continuation
After the pilot implementation, the project participants expected that the hospital would discontinue the no-show project.This did not happen.Instead, hospital management transferred the project from the participants in the project group to the ML staff in the IT department.Management acknowledged the project participants' contribution to demonstrating the potential of ML and their push for the building of a capacity for running ML projects at the hospital.The new capacity involved both staff competence and technological developments.With this capacity in place, the IT department could take on the no-show project.Apart from the argument that the project formally belonged with the new ML staff in the IT department, a restaffing also appeared necessary to move the project forward.In addition, management argued that the poor performance in the pilot implementation was probably the result of too few data for training the algorithm.It was envisaged that this problem would be solved by moving the project from the endoscopy/cardiology departments to the IT department, which would train the algorithm on data from all departments and then make it available across the hospital.
During this phase, hospital management employed one innovation tactic: Escalating: Despite the disappointing pilot implementation, hospital management still considered the no-show algorithm promising.To realize its potential, management opted for an escalation tactic.With this tactic, the scope of the project was increased from single departments to the entire hospital.Concomitantly, the project was transferred to the IT department and restaffed.Thereby, the escalation tactic went beyond the innovation possibilities available to a project group anchored in single departments.By separating the project from the innovators who had been driving it up to that point, the tactic also marked the end of this analysis.

Discussion
The no-show project lasted 3.4 years.During that period, the algorithm and organization adapted to each other, but these adaptations did not produce a no-show solution in operational use.In the following, we discuss the reasons for this outcome, the tactics employed in striving to succeed, the implications of the study, and its limitations.Fig. 1 provides an up-front summary of our argument that ML implementation is an innovation process that fosters the use of various tactics to move the implementation forward.

A long and winding road
To understand the challenges involved in implementing the no-show algorithm, the first thing to understand is that the project was anything but a successive progression through the first, middle, and last miles.Rather, the implementation process can be pictured as a long and winding road.The innovation tactics were devised in response to the challenges, which thereby explain the background or reasons for the tactics.By discussing these reasons, we aim to clarify the conditions under which innovation tactics become part and parcel of ML implementation.Our analysis points to four reasons.
First, the work on the data and algorithm (i.e., the first and middle miles) was not finalized when the work on real-world implementation (i.e., the last mile) began.For example, the implementation activities ran in parallel with further data collection and technical development, such as devising new categories for patients' appointment status, training the no-show algorithm on new data, and honing it by exploiting new technological possibilities.This parallelism is not specific to the no-show project.Waardenburg and Huysman (2022) argue that the boundary between development and real-world use is blurred in AI projects.As a consequence, unresolved data and algorithm problems spill over into the implementation activities and must be resolved through innovation tactics.An example of such a spillover is the lack of resources in the IT department to secure sufficient data exchange for the no-show algorithm in the pilot implementation.Furthermore, essential details about what precisely to model by an ML algorithm might not appear until the implementation phase, such as when the no-show project moved to the cardiology department and changed to model patient recruitment.Another reason for the blurred boundaries is that it is difficult to simulate the outputs of ML systems in prototypes and, therefore, often necessary to postpone evaluations until the systems are ready for real-world use (Yang et al., 2020).By postponing evaluation, problems become salient during implementation rather than during pre-implementation testing, and users experience uncertainty about the accuracy of ML predictions (Verma et al., 2023).
Second, the change associated with introducing the no-show algorithm was highly sociotechnical because the algorithm merely performed a sub task.The algorithm flagged patients who would likely not show up for their appointments; it did not automate the entire process of reminding these patients about their appointments.For the algorithm to work, medical secretaries must provide data of sufficient quality, assess the algorithm predictions, and decide whether and how to remind the flagged patients.The point that algorithms merely perform subtasks has been made before (Grønsund and Aanestad, 2020;Shaw et al., 2019).It entails that the hiatus of human trust and the hiatus of machine experience are mutually reinforcing: The presence of one creates fertile conditions for the other and, conversely, the absence of one causes the other to struggle too.Such mutual reinforcement increases the complexity of implementing ML algorithms because it creates gridlock situations where it is difficult to get beyond low initial levels of human trust and machine experience.Innovation tactics must be devised and employed to unlock such situations.The mutual reinforcement also enriches analyses of barriers to the implementation of ML algorithms by pointing to dynamics among the barriers.For example, the 16 barriers identified by Bérubé et al. (2021) include barriers relating to the hiatus of human trust (e.g., resistance to change) as well as barriers relating to the hiatus of machine experience (e.g., lack of quality data).However, there may be additional dynamics, such as among barriers at the organization, project, and practice levels.
Third, the repeated efforts to test the algorithm show the nonlinear nature of the implementation activities.While the test efforts are evidence of the importance of instilling trust in the no-show algorithm, the activities in between the tests aimed to ensure data quality and train the algorithm.The nonlinearity resulted, in part, from the sensitivity of the algorithm to local conditions.Specifically, experiences with the algorithm in the endoscopy department could not simply be transferred to the cardiology department.New tests and training data were needed.The attempted tests included an RCT (Phase 2) and a pilot implementation (Phase 4).In retrospect, it was probably a mistake to attempt an RCT because it increased complexity and because the pilot implementation was a simpler alternative.The RCT resonated with the physicians but involved sorting out the intricacies of meeting the strict standards of clinical research.The pilot implementation was more of a stunt.In implementation studies, stunts are "highly visible, one-off events with a short-term horizon" (Aanestad and Hanseth, 2002, p. 40).By taking a more pragmatic and less definitive approach to testing, the pilot implementation succeeded in collecting experiences from secretaries who used the no-show algorithm in their daily work.This way, the innovation tactics aimed to create positive reinforcement, in which increased trust in an ML application leads to more use and better data recording, which in turn provides better ML predictions and more trust in them.However, the application may also enter a vicious circle, in which trust and predictions drive each other downward.
Finally, the hype that surrounds AI and ML technologies makes it difficult to set realistic expectations for concrete projects.On the one hand, this hype (Chen and Asch, 2017) contributed to a belief at the organization level in the possibility of developing an accurate and effective no-show algorithm.This belief was important in securing continued management support for the project.On the other hand, the hype contributed to an expectation that a mundane use case such as patient no-shows was an easy win that would quickly be achieved.When not met, this expectation turned into frustration with the experienced problems, which were unanticipated.The frustration started at the practice level where the secretaries were among the first to encounter that the algorithm introduced extra work in terms of stricter data quality requirements.To maneuver in such conditions, ML projects must creatively reconcile contradictory issues.Their innovation tactics must align the projects with hyped expectations to secure support and, at the same time, the tactics must align expectations with project realities to avoid disappointment.

Innovation tactics
Collectively, the innovation tactics employed in the no-show project highlight the significant efforts that went into maneuvering the hospital context.These efforts were necessary because the hospital context featured a diversity of opinions about the potential, constraints, and practical applicability of data-driven no-show predictions.To continue and scale, the no-show project had to create and nurture supportive opinions as well as to counter reservations and resistance.Aligning and framing are examples of tactics that aim to activate support; hibernating and sequencing are examples of tactics that aim to passivate resistance.Specifically, the sequencing tactic illustrates the dynamics between activation and passivation: The means to passivate the increasing reservations at the organization level was to activate the existing support in the cardiology department and, thereby, create the progress necessary to counter the reservations.Bourgoin et al. (2020) call attention to such dynamics by showing that activation and passivation are different means toward the same overall end and that actors switch back and forth between them to establish their authority and further their goals.Activation and passivation are the two basic mechanisms through which the innovation tactics seek to maneuver the hospital context.
The 14 innovation tactics show that activating support and passivating resistance are situated activitiesthey are dynamic responses to particulars and evolutions in the implementation context.These particulars and evolutions include the dynamics between the organization and practice levels.The organization level provides the resources necessary to make progress at the practice level and this progress, in turn, helps secure the continued provision of resources.However, this reinforcement cycle has additional dynamics.The provision of resources is influenced by issues other than progress, and progress requires more than resources.The innovation tactics exploit these additional dynamics.For example, the manifesting tactic organizationally promoted the promise of the no-show project, rather than its progress, and the presupposing tactic aimed to keep the project running at the practice level during a period with scarce resources.It required considerable effort from the project group to maneuver the politics at the organization level.This observation is possibly due to characteristics that are specific to the hospital and thereby stress the situatedness of digital innovation.However, it may also be indicative of the strong interest as well as skepticism that ML technologies currently attract, making this type of project prone to attract attention at the level of organizational politics.
The implementation of the no-show algorithm was characterized by multiple dynamics.Fig. 2 shows the four dynamics revealed and addressed by the innovation tactics.In addition to the activation/ passivation and organization level/practice level dynamics, we have previously discussed the dynamics between development and implementation and those between the hiatus of human trust and that of machine experience (Section 5.1).The innovation tactics wrestle with these dynamics and, thereby, foreground the complexities of ML implementation.Rather than being a linear rollout, ML implementation involves cycles of situated and interdependent activity.These nonlinear dynamics are underrecognized in the discourse on the last mile (Coiera, 2019).
The innovation tactics employed in the no-show project were similar to those identified by Arvidsson and Mønsted (2018), see Table 2.For example, both projects applied decoupling to be able to experiment without the constraints of operational decisions and timelines.The commonalities between the two studies generally confirm that the concept of innovation tactics applies well, also in the last mile of ML innovation.However, compared to the original concept of innovation tactics as presented by Arvidsson and Mønsted (2018), our study contributes new insights about the role of innovation tactics during implementation.First, by identifying 14 innovation tactics, the analysis of the no-show project adds nuance to the original concept.The extended set of tactics is indicative of the wide array of situations that digital entrepreneurs must be able to maneuver to ensure the continuation and scaling of innovations.Additional tactics may be employed in other projects.
Second, the tactics were applied in a different sequence in the two studies.For example, concealing was restricted to the first phase of the project studied by Arvidsson and Mønsted (2018).In contrast, it was repeatedly necessary to conceal the no-show project by decoupling it (Phase 1), hibernating it (Phase 3), and presupposing the availability of resources (Phase 4).That is, digital innovation in healthcare is generated dynamically in response to the specific circumstances faced.These circumstances include that healthcare professionals expect that new tools and procedures are backed by validation studies (Verma et al., 2023) and that they do not replace practice-based knowhow with more uncertain inferences (Lebovitz et al., 2021).The circumstances also include local particulars, such as the possibilities for incorporating ML algorithms in EHR-based workflows and the variation in project appeal due to department differences in no-show rates.Therefore, the tactics presented in this study and that of Arvidsson and Mønsted (2018) should not be seen as a framework to be followed, but rather as heuristics that enable researchers and digital entrepreneurs to reflect on how to cultivate the necessary support during the last mile.
Overall, this study strongly suggests that innovation tactics fulfill an important role during the last mile of AI and ML innovation in healthcare.As we have previously argued, AI and ML innovation is often the initiative of individual entrepreneurs rather than organizational entities.It is critical that these entrepreneurs devise and employ a rich set of innovation tactics to develop the innovation, create interest among stakeholders, and dodge the resistance that will often occur.

Implications
The main implication of this study is that the implementation of ML solutions is an innovation process.To succeed, implementers must employ tactics directed at innovating.That is, they must be competent in preparing sociotechnical change and making it happen.For this reason, it is questionable whether IT departments can drive ML implementation processes.These departments have indispensable knowledge about technology but may lack competence in preparing and making sociotechnical change.The prospective users may also lack this competence, but they have indispensable knowledge about local work practices.Work on the competences necessary in configuring information systems and work practices for each other points to the diverse competences needed and the importance of finding people with the required mix of competences (Hertzum and Simonsen, 2019).The discourse on the last mile tends to mask the amount of innovation required to implement ML solutions.
Three supplementary implications serve to elaborate the main implication: • ML solutions cannot be finalized while they are decoupled from the practice level.Their training and predictive performance depend on operational data that result from the work practices affected by the algorithm predictions.That is, work practices, data quality, and algorithm predictions interact in emergent ways.To respond to these emergent interactions, development activitiesinnovationcontinue after ML solutions are recoupled with practice.• The stakeholders in ML implementations may not be in alignment.
Like in the no-show project, implementers may for example enjoy top-management support and, at the same time, face local hesitation.
In the absence of pre-implementation alignment, it becomes a primary objective for the implementation process to achieve alignment.However, misalignment multiplies the work required to obtain positive reinforcement between the two hiatuses.• Future work should investigate how best to obtain mutually supportive cycles of positive reinforcement between the hiatus of human trust and that of machine experience.The innovation tactics identified in this study are not a complete set, and they were ultimately insufficient.Maybe, an agile process with quick iterations is more effective at generating positive reinforcement than the long phases in the no-show project.

Limitations
Three limitations should be remembered in interpreting the results of

Fig. 1 .
Fig. 1.The reasons (left) that make innovation tactics central to the implementation of the no-show algorithm and the implications (right) that follow from the presence of the innovation tactics.

Fig. 2 .
Fig. 2. The four ML-implementation dynamics revealed and addressed by the innovation tactics.

Table 1
Timeline of the five phases in the no-show project.