How does open government data driven co-creation occur? Six factors and a ‘perfect storm’; insights from Chicago's food inspection forecasting model

It is becoming increasingly clear that the concepts of open government data (OGD) and co-creation are related; however, there is currently only limited empirical material available exploring the link between the two. This paper aims to help clarify the relationship between these two concepts by exploring a recently coined phenomenon: OGD-driven co-created public services. These services 1) utilize or are driven by OGD; 2) are cocreated by stakeholders from different groups; and 3) produce public value for society. Due to the relative newness of the phenomenon an inductive exploratory case study is undertaken on Chicago's use of OGD in the co-creation of their food safety inspection forecasting model. This model forecasts critical food safety violations at food serving establishments and sends inspectors to the highest risk establishments first. The results of this exploratory work led to the discovery of a ‘perfect storm’ of six factors that seem to play a key role in allowing OGD-driven public service co-creation to take place. These factors are motivated stakeholders, innovative leaders, proper communication, an existing OGD portal, external funding, and agile development.


Introduction
On May 09, 2013 President Obama declared that the new default for government data would be open due to the ability of open government data (OGD) to "promote continued job growth, government efficiency, and the social good" (Obama, 2013). Many governments around the world are also taking up this pledge and are striving to release and maintain OGD portals; in the European Union this pledge was recently taken up as part of the 2017 e-Government Ministerial Tallinn Declaration where it was declared that the EU would "increase the availability and quality of Open government data that is of value to economy and society" (European Union, 2017). Within the European Union alone it has been stated that OGD will lead to the creation of up to 100,000 jobs by 2020, lead to savings in the amount of 1.7 bn EUR, lead to efficiency gains, and create large economic benefits ("Benefits of Open Data -European Data Portal,", 2018). Due to the newness of OGD in public administration and society and the widely touted benefits (though not largely studied or validated), the field of e-Government is seeing increased interest in research dedicated to understanding OGD (Scholl, 2016). Scholars in the field have started to look at the drivers and barriers (Barry & Bannister, 2014;Conradie & Choenni, 2014;Janssen, Charalabidis, & Zuiderwijk, 2012), benefits (Carrara, San Chan, Fischer, & van Steenbergen, 2015), and the innovative potentials of OGD (Mergel, Kleibrink, & Sörvik, 2018).
In a similar vein, the concept of co-creation that originated in service management and marketing literature (Osborne, Radnor, & Strokosch, 2016) has been adopted and is being studied by scholars in the fields of public administration and e-Government (Cordella, 2017;Uppström & Lönn, 2017). Though there have been many new and emerging studies on co-creation, there is not a widely agreed upon definition as it pertains to public administration and the term is often used synonymously with co-production; for the purpose of this study, co-creation is understood as the "involvement of outside, non-typical, stakeholders in the initiation, design, implementation, and/or evaluation of a public service" (Toots, McBride, Kalvet, & Krimmer, 2017). In a fashion similar to OGD, governments around the world are beginning to test out or implement co-creation practices in their public service creation process; it is believed that co-creation may lead to innovative public services, solve societal challenges, increase government transparency, and help build a bridge between citizens and government to provide higher levels of public value (Osborne et al., 2016;Voorberg, Bekkers, & Tummers, 2015).
Though both OGD and co-creation appear to have similar motivations driving their use (transparency, effectiveness, inclusion, innovation), there has, to-date, been limited empirical work examining how these concepts are related. As co-creation requires the involvement of non-traditional stakeholders, any technologies that removes or decreases the barriers for stakeholders to participate in co-creation would theoretically allow for more co-creation and thus higher levels of public value. The topic of "public value" is, of course, very much debated in the current academic literature, with no widely agreed upon definition. However, what is generally understood is that public value is something that is created at a societal level, that is to say, it is something that emerges when people use or create something. What also seems to be the case is that public service delivery is changing, it is becoming more open and collaborative and less top-down driven; this creates a new landscape where new technologies, such as OGD, allow stakeholders to create new services that are valuable to them.
In a recent paper, Khayyat and Bannister (2017) present a model that links the concepts of co-creation and demonstrates through multiple case studies that OGD can help to drive and foster co-creation ecosystems. In the same year, another paper linked the topics of OGD and co-creation through a discussion of "OGD-driven co-created public services" (Toots, McBride, Kalvet, & Krimmer, 2017). There are three critical components to an OGD-driven co-created public service: 1) It must utilize or be driven by OGD; 2) It must be co-created by stakeholders from different groups (for example, members from government, private sector, and citizens working together); and 3) It must produce public and societal value.
Other authors have begun to work on understanding the concept of co-creation and how it fits into the context of today's digital society, yet these works are largely theoretical and not backed by empirical evidence. One such example is Lember, 2017 where the author notes that OGD may transform how co-creation happens. There has also been work examining the innovative potential of OGD in service design (Juell-Skielse, Hjalmarsson, Johannesson, & Rudmark, 2014) and how private sector organization may use OGD to create new economic value (Carrara et al., 2015;Jetzek, Avital, & Bjorn-Andersen, 2014); however, the literature on OGD-driven innovation does not explicitly address how OGD can a) play a catalytic role in the co-creation of new public services or b) discuss the relationship between co-creation and OGD.
There appears to be two primary reasons for a lack of work so far on OGD-driven co-created public services, the first is due to the relative newness of the concept, and, secondly, due to what appears to be a general lack of real-world examples of OGD-driven co-created public services; though a recent paper by (McBride, Toots, Kalvet, & Krimmer, 2018) discusses an example in Estonia. However, one real-world example that met all three criteria required for it to be considered an OGD-driven co-created public service was discovered providing a unique opportunity for conducting the first inductive study on OGDdriven co-created public services. This case is the City of Chicago's OGD-based model for forecasting critical food safety violations.
The service relies almost completely on OGD that was already available on the City's OGD portal, it was co-created by stakeholders from city governmental agencies, private sector companies, citizens, and NGOs, and has been in use by the City for years since its development where it continues to create public value. In essence, the service is a machine learning model that takes in different data, predicts which restaurants may have critical food safety violations (critical safety violations "create an immediate health hazard that carries a greater risk of causing food-borne illness" (City of Chicago, 2018a), and then sends food inspectors to the highest risk restaurants first. This case is also strengthened by the fact that previous case studies have studied and demonstrated the effectiveness of the OGD portal in Chicago (Kassen, 2013).
The research is inductive in nature and aims to generate initial propositions related to the concept of OGD-driven co-created public services. As the research in inductive, an exploratory case study is conducted that focuses on the City of Chicago's OGD-based food safety inspection forecasting model. Specifically, this paper aims to explore two research questions: 1) How can OGD contribute to the co-creation of new public services? 2) What factors play a role in the co-creation of OGD-driven public services?
This paper makes contributions to the current academic body of knowledge in e-Government discourse by conceptualizing OGD-driven co-created public services. Additionally, the research provides empirical work on the relationship between OGD and co-creation and studies the factors that play a role in the co-creation of OGD-driven public services; this exploration could be of interest to policy makers, academics, and those who work with OGD and/or co-creation. The paper is rather inductive in nature rather than deductive and it aims at providing initial theoretical propositions on the nature of OGD-driven co-creation; specifically, this paper hopes to identify a network of key enabling factors for OGD-driven co-creation.

Background
When looking at new forms of public service delivery, in particular OGD-driven co-creation thereof, it can be noted that the understanding of public value has shifted. However, though the words "public value" are thrown around in many papers, there is still, to a large extent, little agreement as to what they actually mean. One article that provides a thorough overview of these different approaches is (Bryson, Crosby, & Bloomberg, 2014) where the authors present the predominant views on public value. One of the most commonly cited authors on public value is Mark Moore, who believes that "the task of a public sector manager is to create public value" (Moore, 1994) and that public value could be created by aligning different factors in a "strategic triangle" managers could create public value (Bryson, Sancino, Benington, & Sørensen, 2017). Gerry Stoker further investigates the notion of public value through the idea of networks and highlights the importance of the fact that you only know if public value is delivered by talking and engaging with stakeholders, and that the created "public value" can change over time (Stoker, 2006). However, this shift in the understanding of public value is visible as many authors talk of "public value" and "public value management" it is important not to overlook the fact that public value is not "created by the public sector alone" (Benington, 2007). This leads to a situation where the following appears to be true about public value: 1) public value is not static, 2) public value is something that is created at the societal level, 3) public value is the outcome or result of some service or activity.
One of the most prolific authors dealing with public value, is Elinor Ostrom who noted that, through a process of co-production, public value is created (Ostrom, 1972), that is to say that when a public service is used, the value society receives depends on the interactions with the service. In recent works, Stephen Osborne has been building on this idea of public value and co-production through his New Public Governance (NPG) paradigm (Osborne, 2006) and Public Service Logic (PSL) (Osborne, 2017). The primary focus of NPG and PSL is that public services can be created by any actor, that there should be a focus on public value, and that this public value emerges via interactions between service user and service provider (Osborne et al., 2016).
Closely linked to the concepts of public value and co-production, is the concept of co-creation. Similar to public value, co-creation is still being theorized in the literature and there are not many widely agreed upon definitions. Table 1, shows an overview of how the literature has evolved its understanding of coproduction and co-creation through the decades. There are three important takeaways: firstly, some definitions view coproduction and co-creation as something that involves only citizens in a G2C (government-to-citizen) type relationship whereas others posit that any actor can be involved in co-creation. Secondly, many understandings of co-creation and coproduction view it in a somewhat cyclical fashion where co-creators can get involved at many different stages of public service delivery. Thirdly, and most recently, there has been increased interest in the role of ICT in the co-creation process.
For the purpose of this paper, as we are discussing the role of OGD in a co-creation process, the conceptualizations by Paletti (2016) and Lember, 2017 are of high interest; OGD-driven public service co-creation may be understood as co-creation that occurs as a result of the availability and exploitation of OGD. In Lember's, 2017 piece, he also notes that OGD is one technology that has the potential to transform the co-creation process by creating new ways and avenues for co-creation to occur that were not previously possible (Lember, 2018).
In recent years, ample research has been published on OGD and its different aspects. One of the main recurring topics in literature is the benefits that are associated with OGD (Foulonneau, Turki, Vidou, & Martin, 2014;K. Janssen, 2011;OECD, 2016). OGD has been claimed to have the potential to drive innovation (Chan, 2013;Juell-Skielse et al., 2014;Toots, McBride, Kalvet, & Krimmer, 2017), foster transparency and accountability (Gonzalez-Zapata & Heeks, 2015;, drive the creation or implementation of new public services (Chan, 2013;Foulonneau et al., 2014;Toots, McBride, Kalvet, Krimmer, Tambouris, et al., 2017), and empower citizens and communities (Gonzalez-Zapata & Heeks, 2015;. However, the manifestation of the potential benefits of OGD in practice requires the available data to be actually usedsimply providing OGD does not automatically result in significant value for society (M. Janssen et al., 2012). As OGD is seen to be able to drive innovation and new ways of interaction, thus changing the ways in which services are created, it has the potential to contribute to the creation of innovative services that provide public value. However, a clear research gap has been indicated in this domain as "little is known about the conversion of public data into services of public value" (M. Janssen et al., 2012). That being said, what is known, is that public value is a networked and systemic process; it is something that is created as a result of services being used and the interactions between service user and service providers. It is also known that public value is something that can change over time and is heavily sensitive to individual and societal beliefs. OGD is one technology that allows for society to express what it values in two clear ways. Firstly, the demand for OGD demonstrates a high value placed on the availability of OGD for reasons such as transparency or accountability. Secondly, the availability of OGD allows for any stakeholder to ask a question that is interesting to them, analyze or exploit OGD, and find the answer to the question on their own and then share any new services created in the process. This thus helps bridge the gap between service user and service provider and give clearer overviews on this notion of "public value".
This relationship between OGD-driven co-creation and public value has not been widely touched upon, but there seems to be increasing interest in research in exploring this link (Khayyat & Bannister, 2017;Millard, 2015;Toots, McBride, Kalvet, Krimmer, Tambouris, et al., 2017;Van der Graaf & Veeckman, 2014). Millard, 2015 finds that one of the new responsibilities of government agencies is to provide tools and incentives for citizens to engage in co-creation (Millard, 2015) and Van der Graaf et. Al note that the availability of OGD may allow for new forms of co-creation (Van der Graaf & Veeckman, 2014). However, these two papers fall short in describing the process of OGD-driven cocreation and also appear to be government-centric describing a cocreation process where the government remains in the driver's seat, whereas this runs counter to the understanding of co-creation put forth in this paper. Khayyat & Bannister, 2017 provide one of the most thorough overviews and discussions of OGD-driven co-creation through their integration of activity theory into a framework describing the OGD co-creation process.
Outside of the literature discussing co-creation and OGD, lies the idea of OGD-driven innovation. One paper that discusses turning OGD into public value is put forth by Carrara et al. (2015) where they introduce the concept of "Data Value Chain" that offers a basis for understanding different types of re-use of data and covers various stages, such as data creation, data validation, data aggregation, data analysis, data services and products and aggregated services. They also see a major role for external stakeholders -for individuals or companies that analyze the data and create services and products. However, this model does explicitly address the relationship between OGD and co-creation.
Another interesting paper, "Is the Public Motivated to Engage in Open Data Innovation?" was put forth by Juell-Skielse et al. (2014). Though this paper discusses the relationship between coproduction and OGD, it does not quite address the issue of OGD-driven co-created public services as it focuses on the government specifically as the driver and organizer of events where services are to be co-created. This runs in contrast to the understanding of co-creation used in this paper, as it is believed that any stakeholder may lead and drive public service cocreation, not only the government. When discussing the role of OGD in service co-creation a paper by Foulonneau et al. (2014) finds that there are three main roles which data plays in a new service: "the service is based on data, the service uses data as a resource, and the service is validated or enriched with data but the data is not directly used or is not directly visible in the service." (Foulonneau et al., 2014). They also find that OGD is currently underutilized, and applications that create public value only utilize a small number of datasets. Charalabidis, Alexopoulos, and Loukis (2016) find that OGD can allow services to be co-created by non-typical service producers which results in the building of new and innovative applications.
To briefly summarize, the main benefits of an OGD-driven cocreation approach are as follows: • There is a transformation in how public services are delivered. Any stakeholder now has the power/potential to create services that exploit OGD to create public value.
• OGD lowers the barriers to participation on co-creation by providing more opportunities for more stakeholders to use and exploit data to create services. Additionally, it allows for creativity to flow as the data can be used in any way to create any sort of service desired.

Authors
Types of "coproduction/co-creation" Whitaker (1980): ▪ Citizens requesting assistance from public agents ▪ Citizens providing assistance to public agents ▪ Citizens and agents interacting to adjust each other's service expectations and actions Pollitt, Bouckaert, Loeffler (2006): : ▪ Individual acts of coproduction ▪ Collective acts of coproduction ▪ A mix of individual and collective acts of coproduction Osborne & Strokosch, 2013: ▪ Consumer coproduction ▪ Participative coproduction ▪ Enhanced coproduction Voorberg et al., 2015: ▪ Citizen as a co-implementer ▪ citizen as a co-designer ▪ citizen as an initiator Osborne et al., 2016: (2016): ▪ Coproduction through ICT: ▪ Applications to report problems ▪ Applications to crowdsource data ▪ Applications to involve citizens in the public service delivery Lember, 2017: ▪ ICT Indirectly affects Coproduction ▪ ICT Transforms Coproduction ▪ ICT Substitutes for Coproduction • OGD-driven Co-creation leads to higher levels of public value compared to other types of co-created services. Whereas other services only produce public value once the service is delivered, OGDdriven co-creation provides insights into public value both when stakeholders demand and ask for the data, and then also when the service is created/used.

Factors influencing OGD-driven co-creation
So far, there is only limited research on the factors influencing OGDdriven co-creation specifically, with the exception of Toots, McBride, Kalvet, & Krimmer (2017). This paper found that in situations involving both OGD and co-creation, the barriers to both tend to reinforce each other: the supply of OGD depends on its perceived value, whereas the perception of value depends on awareness of the benefits of open data, which can best be demonstrated through actual OGD-driven services. At the same time, as long as the barriers to OGD availability are not solved and there is a lack of usable OGD, the co-creation of new OGD-driven services remains limited. As there is limited research on OGD-driven cocreation specifically, the influencing factors for OGD and co-creation are presented individually.
Interestingly, while many barriers emerge from the public sector context, the drivers tend to come from forces external to the government. Some of the major drivers relate to "citizen pressure, market initiatives, emerging technologies and the ideas of thought leaders" (Huijboom & van den Broek, 2011), but also to the promises of benefits such as economic value, better governance, increased transparency, participation, improved government data infrastructure and improvements in public services (Gonzalez-Zapata & Heeks, 2015). However, some barriers also exist at the user side, for example users' limited understanding of data (Johnson, 2014;Zuiderwijk et al., 2012).

Factors influencing co-creation of public services
Similar barriers are present with regard to co-creation between governmental and non-governmental stakeholders. Previous research has noted a number of barriers in the organizational context of the public sector, including the incompatibility of organizational structures and processes with the process of co-creation, lack of open attitude to citizen participation, risk aversion (Bovaird & Loeffler, 2012;Voorberg et al., 2015).
Other co-creation drivers and barriers are related to the citizen side. These include the internal motivation of participants (Juell-Skielse et al., 2014), personal characteristics and values, awareness of participation opportunities, participation skills, perceived capacity to participate in co-creation initiatives, perceived responsibility for improving public services, social capital, trust in the co-creation initiative (Voorberg et al., 2015) relative importance of the service that is being co-produced, money, time and other resources (Jakobsen, 2013;Verschuere, Brandsen, & Pestoff, 2012), and mutual trust between government and citizens (Osborne & Strokosch, 2013;Tuurnas, 2016). One of the key challenges to co-creating public services is the need to redefine the roles of public and private actors in the public service creation process (Hartley, Sørensen, & Torfing, 2013).

Methodology
A case study allows for a contemporary phenomenon to be observed and investigated in context and is often viewed as an acceptable research method for generating theory and understanding of issues in the social sciences such as in the public administration or societal domain (Yin, 1989). This study was designed as a single exploratory case study (Yin, 1989). This sort of exploratory case study is conducted when there is a need to construct new theory or generate propositions for current and contemporary events that are thus far understudied or not understood (Yin, 1989). The holistic aspect of the case study implies that the case is not divided into smaller sub-units or specific variables. Instead, there is a single unit of analysis, and in the context of this research the unit of analysis is the whole process that was undergone to co-create the OGD-driven public service in the City of Chicago. Single case studies are often criticized for having low levels of external validity, but it is also argued that case study research can be a valuable method to gain insights that allow researchers to make theoretical postulations and analytic generalizations about the phenomenon that is being investigated (Yin, 1989). In order to strengthen internal validity, multiple sources of evidence are used (semi-structured interviews, news articles, government document analysis) which can be used to triangulate conclusions and ensure higher levels of internal validity. Furthermore, triangulation also helps to mitigate against subjectivity in the research and thus strengthens the overall research design.

Empirical data collection
The first evidence that was collected came from reviewing the executive orders, laws, and government mandates in effect within the City of Chicago that related to OGD; these records were documented and then analyzed using thematic analysis. After the initial legal overview was conducted, news articles related to OGD in Chicago and also to the case specifically were reviewed for relevance, documented, and then coded and analyzed using thematic analysis. In addition to that, the researchers consulted the video material available on the case on Youtube, taking notes and extracting background information on the project's goals, service development process, and stakeholders involved. This initial step allowed for a broad overview of the case to be gathered, and set the foundation for follow-up semi-structured interviews.
Six semi-structured interviews were conducted with stakeholders representing the different parties involved in the co-creation of the food safety inspection forecasting model for the City of Chicago. Though six interviews is normally a small sample, these six interviewees represent every key player involved in the project and provide an overview of every point of view on the project. An initial list of relevant interviewees was identified based on the key contributors listed on the project's GitHub page (https://github.com/Chicago/food-inspectionsevaluation). The first interview was conducted with Tom Schenk, the main person behind the case, and then, using snowballing, other interviewees were selected. As a result, the final list of interviewees included one person from Chicago Department of Innovation and Technology (CDoIT) (Tom Schenk, Chicago CDO), Civic Consulting Alliance (CCA) (Anonymous), Allstate (Stephen Collins, Model Developer), and Montgomery County (Daniel Hoffman, Chief Innovation Officer), whereas two members were interviewed from the Chicago Department of Public Health (CDoPH) (Gerrin Butler, Head of Department and Raed Mansour, Health Analytics). The interviewee from the Montgomery County Department of Innovation was selected following the previous interviewees' suggestions due to the county's relationship with the project (Montgomery County implemented Chicago's code with the help of a private sector partner), though they were not directly involved in the initial model development. The interviews were conducted during April and May 2017 over the phone or through Skype and lasted from between 15 and 40 min each. All interviews were conducted by two different members of the research team, recorded, and then transcribed.
The interview questions aimed to provide a better understanding of the interviewee's role in the project, how they got involved, how the process unfolded, and the factors behind the success, building on the theoretical base introduced in the theoretical sections of the current paper. The main guiding questions for the semi-structured interviews can be found in Table 2.
After the interviews had been transcribed, two researchers went through the transcripts and individually coded them. To strengthen the reliability of the coding, the researchers then compared the results and discussed the differences to come up with an agreement. A theme-based coding scheme was used to identify key motivations and drivers, important aspects of the co-creation process, contextual factors, enablers and barriers and key success factors.
Commonly mentioned themes and facts were further used to draft initial propositions on what seems to influence the success of a cocreated OGD-driven public service, as well as what factors seem to be needed to allow OGD-driven public service co-creation to take place. The interviews served as a way to gain further insight into the case, provide new information to the researchers, and also to clarify information that had been found during previous evidence collection.

The case
In the realm of OGD, Chicago has been described as one of the leaders in the field (Kassen, 2013). Because of the City's efforts to push OGD, it now has one of the best examples of a high-usage and value generating OGD portal. The OGD portal has grown rapidly and currently provides access to over 550 datasets, applications built by the city and private developers, provides tutorials on how the available data may be exploited or analyzed, provides tools that allow for easy visualization of data, and has been visited over 38 million times (City of Chicago, 2018b).
The OGD portal has been consistently improved since 2012 when executive order 2012-02 was issued, which established the legal requirement for Chicago to establish and maintain an OGD portal, the creation of a chief data officer (CDO) position, and chief information officer (CTO). This executive order stated that every city agency must "make available online… at a level of granularity acceptable to DOIT (Department of Innovation and Technology), all appropriate datasets and associated metadata under such agency's control" (Emanuel, 2012).
The use of OGD for predictive analytics in the City was a political priority and discussed in the Technology plan for the City; for example, Initiative 19 -'Research Data-Driven Solutions to Major Urban Challenges' aims to "use data analytics to help managers across the City exploreand solve -some of the most vexing problems facing municipalities" (City of Chicago, 2013). In addition to the high-level influence pushing for the use OGD-analytics and citizens and civic hackers demanding more data (Chi Hacknight, 2018), there was also a third strong influential force, that of philanthropic funding. Bloomberg Philanthropies organized a competition that would "inspire American cities to generate innovative ideas that solve major challenges and improve city lifeand that ultimately can be shared with other cities to improve the wellbeing of the nation" (Bloomberg Philanthropies, 2013). The City of Chicago entered this competition and was awarded a grant for one million USD to develop a new "SmartData" platform that would allow government agencies easier access to predictive analytics tool; one condition of this grant was that all software developed would be open source (Ash Center Mayors Challenge Research Team, 2014). Specifically, Chicago was selected to "create an open-source platform to harness the power of data to understand underlying trends and better direct limited resources" (Bloomberg Philanthropies, 2013). This grant provided the CDoIT funding to begin to undertake more ambitious OGD-driven predictive analytics models. This combination of factors set the stage for the development of a new and innovative OGD-driven public service: funding was available, stakeholders were motivated, there was political support from the City's executive office, and there was a desire to demonstrate the power of predictive analytics in the public sector.

First iteration
The City of Chicago and the CDoIT made it a mission to continue to expand their use of OGD and predictive analytics; it was believed that a movement in this direction would allow for increased efficiency of many government agencies' day-to-day operations and provide increased levels of public value. In order to do this, an initial list of potential use cases where OGD-driven predictive analytics capabilities could be used was drafted in 2014. Though there was high level political interest in these projects and funding was available, the CDoIT still lacked a full data science team, and outside technical assistance was needed; the CCA was an organization that was in a position to mediate and obtain such help for the CDoIT. The CCA is a Chicago based organization that aims to improve the quality of life in Chicago by bringing together stakeholders from public, private, and non-profit sector to work on new and innovative solutions for problems facing the City. Due to the organizations extensive public and private sector networks, they were able to provide the data science team at Allstate What was your role in the project? Who were the other main stakeholders in the project and how were the roles divided? Were all stakeholders involved in all stages of the creation of the model?
Co-creation process, roles of stakeholders 4. What did the planning and development process look like? How was the initial problem identified and described? Co-creation process, co-creation phases 5.
Was this the first attempt for the city of Chicago to turn open government data (together with external partners) into a service that creates public value? Is there an overall willingness to use open government data in the city of Chicago?
Enablers, drivers 6. Do you feel including people from outside of your organization benefited the creation of the model? If yes, In which ways? If no, why not?
Benefits of a co-creation approach 7. How do you evaluate the success of the service? How are the users involved in the evaluation process and what do you do to react to the feedback?
Outcomes of co-creation process, end user involvement in co-creation K. McBride et al.

Government Information Quarterly xxx (xxxx) xxx-xxx
Insurance with the list of use-cases initially drafted by the CDoIT. Ultimately, the Allstate team decided that a use-case that focused on predicting food safety violations was both possible based on the data available on the OGD portal and interesting, as these data scientists lived in the City of Chicago they felt that it was in their best interest for food served to them to be as safe as possible. The CDoIT agreed that this was also an interesting use-case for them and agreed to approach the CDoPH with the idea. The initial development phase began in 2014, and, at the time, the City of Chicago had over 15,000 food serving establishments. To inspect these establishments, the City had only 36 food inspectors. There was a general requirement for each establishment to be inspected roughly twice a year, though there were some establishments with exceptions that required them to be inspected less and others needed to be inspected more; but, twice a year is the general rule (Interview, Gerrin Butler). Traditionally, food inspectors were sent out to establishments on a pseudo-random basis; there was minimal attempts to optimize or predict inspections. The lack of optimization combined with the high workload for each individual inspector led to a situation where critical food violations were not being detected as fast as the City would like and thus some preventable outbreaks were missed (Schenk, 2017). The CDoIT approached the CDoPH with a potential solution. They were going to build a model that would predict when critical food safety violations would occur, at what restaurants, and then send out food inspectors to the highest risk places first. The head of the CDoPH at the time was originally hesitant, unsure of whether or not technology could really do the job of a human or whether or not data could actually lead to these predictions. However, as she had originally read reports about a data-driven model to prevent rodent outbreaks in the City, she decided to embrace, trial, and test this new and innovative solution. At the initial stages, the head of the CDoPH proclaimed "I don't know what the hell an algorithm is, but let's go ahead and try it".
At the beginning of the project, the CCA organized meetings between the relevant parties (Allstate, CDoIT, and CDoPH) and acted as a project manager. At these meetings, the business requirements of the CDoPH were discussed and presented to the developers and data scientists. It is also important to mention that the CDoPH completely hid this project from the food inspectors so that their behavior would not be altered based on the knowledge of a new solution being developed. After playing with different datasets on Chicago's OGD portal an initial list of variables was prepared and a General Linear Model was constructed by Allstate's data science team. However, due to a misunderstanding of how the inspectors conducted their inspections (it was not originally understood that inspectors served in the same geographic area), the first iteration was not successful and needed to be adjusted and updated. This failure ended up being a major learning point for all involved stakeholders and demonstrated the importance of thorough communication early on in the project as well as the importance of constant communication throughout the duration of the project.

Second iteration
Though the first implementation was not able to provide accurate results, the general model and structure was already in place. During this iteration, Allstate did not participate, but CDoIT and the CDoPH had the ability to make the needed changes to bring the model to production. This second iteration of the model is open source and the code is freely available on GitHub (see: https://github.com/Chicago/ food-inspections-evaluation). This repository includes all the data that was used for the training and testing of the original model thus allowing for new versions to be tested by others under the same conditions. At the beginning, approximately sixteen different datasets on the portal were considered for use in the model (Interview, Stephen Collins). The developers selected datasets that they thought would provide interesting or useful predictive capability, such as rat sighting reports or sanitation complaints, and these were then tested and discussed in an iterative fashion with the best predictive ones remaining and those with limited predictive capability being dismissed. After playing with the data, nine variables were selected for the final model: " 1. Establishments that had previous critical or serious violations 2. Three-day average high temperature. 3. Nearby garbage and sanitation complaints. 4. The type of facility being inspected. 5. Nearby burglaries. 6. Whether the establishment has a tobacco license or has an incidental alcohol consumption license. 7. Length of time since last inspection. 8. The length of time the establishment has been operating. 9. And the assigned Inspector." (Schenk, 2017).
The model was tested over a two-month period (September and October 2014) during this time, assignments were given out following normal operations, but, in the background, the model was running simultaneously to see how it would compare to normal operations. After the two-month period was up, the results were analyzed and showed that there was a clear advantage in switching to the data-driven model. The model would have sent inspectors to restaurants with critical food safety violations, on average, 7.4 days earlier over the eight week test period (Schenk, 2017). Thus, this would allow for potential food borne illness outbreaks to be prevented, or have their severity limited, as the violations responsible were being caught and addressed earlier. The model was thus validated and went into the CDoPH's live operations in February of 2015.
The model works by classifying food serving establishments based on how likely they are to have a critical food safety violation. This ranking of establishments based on order of risk is then accessed by the head of the CDoPH through a simple Shiny Application and the department head is then able to assign food inspectors to the highest risk restaurants first. Fig. 1 was created by the CDoIT and demonstrates how this process works.
Interestingly, the developers wanted to make a more complex model and create an application that allowed for in-depth statistical analysis and visualizations, but, as one interviewee stated, at the end of the day all the CDoPH wanted was a simple list (Interview, Tom Schenk); so, a minimalistic approach was adopted that simply solved the problem of being understaffed and allowed for the CDoPH to do their job better.

Impact and future
Though the model was operational in February of 2015, the improvement did not stop there. Once the code had been made open source, citizens and outside stakeholders have also been able to get involved. For example, on Feb 3, 2017 a citizen of Chicago made a pull request demonstrating how the XGBoost model was finding critical violations, on average, 7.79 days earlier; this represented an improvement on the current model in use. Four days later the Chief Data Officer of Chicago had commented on it and a code review has been initiated and stated, "If the results hold, we will incorporate your contributions to the model that drives food inspections in the city. Thank you and we will be in touch soon". 1 This provides a clear demonstration of how outside stakeholders are able to play a role in the co-creation of OGDdriven public services. The model is still in use by the CDoPH today and it is still actively maintained by the CDoIT, and stakeholders are still able to suggest improvements to the model through GitHub.
This case is truly interesting as it represents one of the only examples that the authors found that seems to represent a co-created OGD-driven public service. It is also a service that is able to continue to provide value moving into the future. As more data is generated, the model is likely to become more accurate in predicting critical violations. It will be interesting to follow up on this case in the future to see how the co-creation of the service progressed as well as how the accuracy improves over time.

Observations
The case at hand has interesting implications for the current understanding of the role of OGD in public service co-creation, and also for enhancing the current knowledge on what factors influence or play a role in OGD-driven public service co-creation. While conducting interviews for the case, many interviewees made it clear that it was not just about the availability of data, the environment that the development of the model took place in was what was key here. One interviewee described it as a 'perfect storm': "the data was out there to increase transparency, the food inspections were already happening, the licensing was there, the civic tech community was motivated, the potential was already there, but it was the environment that was key" (Interview, Raed Mansour). In order to better understand this environment or 'perfect storm' better, different key factors that were brought up by the interviewees were analyzed and discussed. The picture that emerged was a 'perfect storm' consisting of six factors: motivated stakeholders, innovative leaders, proper communications, existing OGD portal, external funding, and agile development. Though it is always possible to argue that other factors may also have played a role, these six appear to be the factors that were crucial for the cocreation of Chicago's food safety inspection forecasting model to take place; that is to say, if any of these factors were changed or missing this case would not have happened. The rest of this discussion section is dedicated to describing each of these factors in detail.

Motivated stakeholders
One of the key factors of the "perfect storm" seemed to be motivated stakeholders. The theoretical literature also views internal motivation of participants as one of the drivers of co-creation (Juell-Skielse et al., 2014). This internal motivation came from different places for different stakeholders. For example, the Allstate data science team was motivated to create this model as it allowed for their city to have higher levels of food safety, and, additionally, it provided an interesting learning opportunity. This differs, for example, from the motivation for the head of the CDoPH who wanted to improve her department's business processes and do more with less. Though originally hesitant, to get on board with the model development the head of the CDoPH needed to be convinced. To do this, Tom Schenk contextualized the problem to the CDoPH by saying that "you have 100,000 decisions to make a day, let us just make this one easier for you" and then noted in the interview that when you "focus on needs, it helps foster cooperation" (Interview, Tom Schenk). In regards to the motivation for citizens to participate, this motivation appeared to be rather intrinsic and was driven by the desire to do something positive for the city and make it a safer place. Since stakeholder motivation appears to play a key role in OGD-driven public service co-creation, and one of the main motivational factors is having a pain point or problem, it seems likely that OGD-driven public service co-creation is more likely to occur when providing an innovative solution to an old problem rather than during the development of a new service.

Innovative leaders
Having innovative leaders seems to be an important part of success. While conducting interviews, two names were always stated as playing a critical role in the success of the project; Tom Schenk (Chief Data Officer of Chicago) and Gerrin Butler (Director of Food Protection for the City of Chicago). Tom was said to be the main driving force behind the model and had it in mind for the code to be open source since the idea was conceived. Gerrin was the owner of the food inspection process and from the beginning she agreed to work with the plan for datadriven food analytics. Gerrin did not initially understand what or how a data analytics model would work and improve current operations, but was willing to try and played an active role throughout. As one of the interviewees mentioned, "Gerrin said that she did not think this would work, but she was still willing to actively participate" (Interview, Tom Schenk). Without the work and willingness of these leaders to push for and try new things, this case would not have been possible.

Proper communication
Another key success factor identified was good communication. It was highlighted by several interviewees that in order to improve the existing processes, full understanding of how the CDoPH operates was needed. It was crucial to understand how the food inspectors are assigned to their tasks, what and why do they do, what kind of data they used and also, what kind of municipal codes and state and federal laws are involved. At the beginning stages of development, the different stakeholders did not understand each other's terminology. The head of the CDoPH painted a picture of this during her interview: "They would use very technical words. I would ask for their meaning and write down what it meant. I was also doing the same for them, I would explain how our system and business processes worked and provided them with a list of important definitions and risk factors. Basically, we created and used 'cheat-sheets'" (Interview, Gerrin Butler). These cheat-sheets allowed for effective communication and ensured that all partners understood each other during communications and helped to prevent misunderstandings, as stated by Raed Mansour, "the power often lays in the terminology".

Existing OGD portal
In the current literature on OGD, it is often pointed out that OGD available from portals is often not clean, accessible, or user friendly (Young & Yan, 2017). However, in the case of Chicago, this barrier did not occur. In fact, multiple interviewees praised the OGD that was available on Chicago's OGD portal and all the data that was used in the development of the model was freely available on the portal to any user. Due to the high quality of the available data, it was possible to focus on developing innovative use cases for the data rather than working on data cleaning or finding what datasets were available. One reason for the high quality of the data on Chicago's OGD portal is because the data that is available on the portal is the same data that government agencies use in production. The ETL is linked up to the core database of the data provider, it is extracted, transformed, and then uploaded to the Chicago OGD portal; 99% of the data in the Chicago OGD portal arrives there automatically via the ETL (Goldstein, 2013). Architecturally, the City decided that it was more important to "focus on building direct pipelines rather than building data warehouses or datamarts to power the portal" (Schenk -Email Communication). The ETL ensures that the OGD portal is sustainable as it enables accurate and up-to-date information to be always available. Furthermore, it also allows the traditional data silos to be dealt with and ensure that data that should be open is opened. The use of this ETL also integrates the provision of OGD to the regular business process and does not require a large amount of human overhead to deal with.

External funding
Maintaining an OGD portal, developing new services, and maintaining a team focused on OGD innovation requires funding. Interestingly, lack of funding is often pointed to as a barrier for OGD availability in the current literature (Barry & Bannister, 2014). Though the City of Chicago did devote some funding to the CDoIT, the availability of External funding was a major driver in this case. Funding from Bloomberg Philanthropy provided the necessary extra funds for developing the City's predictive analytics capabilities, without this funding the service may not have been created.

Agile development
When talking about the development of the service, one interviewee noted that all the participants were working on multiple projects and thus could not focus on one project for a long period of time (Interview, Tom Schenk). This led to a situation where development was done in an incremental way over time by a wide variety of stakeholders. However, as noted by one interviewee, "if you're running an innovation program, you need to be flexible and agile. If you have one hypothesis and it is proven wrong, you must be able to pivot" (Interview, Raed Mansour). Though the service did not follow a traditional agile development methodology with sprints, many aspects of agile development were present and highlighted as key to the success of the project by interviewees. These aspects included rapid development, release of an MVP, validated learning, incremental development, constant testing, and being able to respond quickly to feedback and evaluations.
Reflecting back on the literature, many barriers that normally hinder OGD usage and development of new services were present, but overcome. For example, the quality of OGD on portals is often described as poor quality, messy, and not user-friendly (Young & Yan, 2017). However, the City of Chicago was able to overcome this by focusing on the sustainability of their OGD portal and automating the movement of data from government agency databases to the OGD portal. Barriers such as lack of personal or political interest were dealt with by motivating leaders and stakeholders through problem contextualization, focusing on one problem at a time, and ensuring that development did not require a large amount of effort from the co-creators or a major change in the service users' business processes. Interestingly, the drivers and barriers are often viewed as independent of each other, but what this case seems to drive home is that this is often not the case; the drivers and barriers are almost always linked and, in the case of OGDdriven co-creation, it does not take place within a vacuum, but rather, within a system where everything is connected. Thus, it does appear to be the case that it was really a 'perfect storm' of factors that allowed for the City of Chicago to fight past traditional and expected barriers.

Conclusion
This paper presents an empirical example of a co-creation OGDdriven public service, however, it must be stressed that it only is only one example of an OGD-driven co-created public service and future research should be carried out exploring different sorts of applications (such as web or mobile applications), co-created with different stakeholder configurations (for example, with citizens as the driver of the service), and with different target groups (such as citizen focused or business focused). Though this paper represents a single exploratory case study, there appear to be some interesting implications for the current scholarly debate on OGD and co-creation, and, additionally, new insights have been gained that may be beneficial for any stakeholder interested in OGD and/or co-creation.
The first major contribution of this paper is its conceptualization and initial work in defining what is meant by OGD-driven co-created public services. The initial definition has three requirements: the service must utilize OGD, it must be co-created, and it must produce public value for society. This definition is summarized by Fig. 2.
A second contribution of this paper is that it demonstrates that OGD appears to have the potential to play a catalytic role in driving and enabling the co-creation of new public services. In the presented case, it was the presence of OGD that allowed for this new service to be cocreated. Of course, it is also possible to co-create services without utilizing OGD, but it does seem that the presence of OGD breaks down the barriers for those who want to engage in public service co-creation. However, it is also well known within the literature that in many instances OGD goes unused and does not produce value. How then, did the City of Chicago create an environment that was conducted for OGDdriven co-creation and public value creation? Based off of this initial exploration, it appears to be the case that a 'perfect storm' existed within the City of Chicago that when exposed to OGD allowed for something new to occur. This perfect storm consisted of motivated stakeholders, innovative leaders, proper communication, an existing OGD portal, external funding, and agile development. This is interesting as if OGD-driven co-creation of public service does indeed quire a 'perfect storm', it is actually quite a complex process that is sensitive to many different conditions. It is not only about the data, but the system as a whole must be taken into account when studying OGD-driven cocreated public services.
For stakeholders who want to try to duplicate Chicago's approach to OGD-driven public service co-creation, there are a few different suggestions that can be made to try to foster or develop this same 'perfect storm' in other locations.
• Make the problem relatable to the stakeholder, if stakeholders are directly involved or related to the problem they may become more motivated.
• Start simple, only develop what you need and try to integrate it with the traditional business process.
• Communicate, make sure that everyone has an understanding of key definitions and business processes. Having a clear list of definitions among stakeholder groups decreases the risk of misunderstandings.
• Be open, develop services in a way where they are easily shareable and accessible, for example, host code on GitHub and be responsive to issues and pull requests. This paper sets the initial groundwork for future research on OGD- Government Information Quarterly xxx (xxxx) xxx-xxx driven co-created public services. However, it is limited by the fact that it is only a single exploratory case study that focuses on one specific configuration of an OGD-driven co-created public service. When analyzing other OGD-driven co-created public service configurations, the following questions may be of interest: What type of service is it (web application, mobile application, data analytics model, etc)? Is the application internal or external facing? What is the leading stakeholder group (citizen, government, business, etc) and what other stakeholders are participating in the co-creation? What is the scale of the service (municipal, state, national)? What is the value being created? It may be the case that different configurations have a different 'perfect storm' and this should be empirically analyzed and tested.
Gerli Aavik, MA is a public official in the Estonian public sector, currently undertaking a PhD at Ragnar Nurkse Department of Innovation and Governance at Tallinn University of Technology. Her main research interests include e-governance, social innovation, open governance, co-creation and person-centred service design.
Maarja Toots, MA is a junior researcher and PhD student at the Ragnar Nurkse School of Innovation and Governance at the Tallinn University of Technology, Estonia. Her research topics mainly concern the collaborative aspects of e-government and public sector innovation, from e-participation, co-creation and open government data to cross-border egovernment services.
Tarmo Kalvet is a Senior Research Fellow at Ragnar Nurkse Department of Innovation and Governance at Tallinn University of Technology, Estonia. Previously, Tarmo was the director of innovation policy research program at PRAXIS Center for Policy Studies. His main research interests are related to technology and innovation management, and policy.
Robert Krimmer is Full Professor of e-Governance within Ragnar Nurkse Department of Innovation and Governance at Tallinn University of Technology, Estonia. His research is focused on electronic participation and democracy, as well as e-voting, the transformation of the public sector, and all issues further developing a digital society.