Advancing innovation in the public sector: Aligning innovation measurement with policy goals

There is su ﬃ cient evidence, drawn from surveys of innovation in the public sector and cognitive testing in- terviews with public sector managers, to develop a framework for measuring public sector innovation. Although many questions that are covered in the Oslo Manual guidelines for measuring innovation in the private sector can be applied with some modi ﬁ cations to the public sector, public sector innovation surveys need to meet policy needs that require collecting additional types of data. Policy to support public sector innovation requires data on how public sector organizations innovate and how a strategic management approach to innovation can in ﬂ uence the types of innovations that are developed. Both issues require innovations surveys to delve deeply into the innovation processes and strategies that are used by public sector managers. Implementation of the measurement framework proposed in this paper would open up opportunities for a new, policy-relevant research program on public sector innovation.


Introduction
The public sector consists of general government (public administration entities at all levels of government, regulatory agencies, and government entities that provide services such as education, health, security, etc.) and publicly-owned corporations (System of National Accounts (SNA), 2008, chapter 22). In high-income countries the public sector contributes to between 20% and 30% of GDP, with the highest shares observed in Scandinavia. 1 This is considerably more than the share of manufacturing in GDP, which is less than 10% in many OECD countries, including Australia, Canada, the United Kingdom, and the United States.
Given its economic weight, there is growing policy interest in how to encourage innovation in the public sector to improve the efficiency in how resources are used, the quality of public services, and address a diverse range of societal challenges, including climate change, demographic pressures, urban congestion and social and economic inequality (Torfing and Ansell, 2017). Following common practice in the management literature, we limit the term 'public sector' to general government organizations and exclude publicly-owned corporations.
Two separate disciplines, based in management and the economics of innovation, have conducted research on innovation in the public sector. The management discipline includes researchers from public management and public policy (de Vries et al., 2018). The majority of these studies have used case studies or interviews to examine different aspects of public sector innovation and to develop theories to explain public sector innovation. The economics of innovation discipline is mainly concerned with innovation in the business sector, but researchers from this discipline have recently experimented with measuring public sector innovation through large-scale, representative surveys of public sector organizations in Europe and Australia. These surveys were initially inspired by the OECD's Oslo Manual guidelines for measuring innovation in the business sector. The strength of case or interview studies is that they can provide indepth understanding of causal relationships, such as the effects of different strategies on the types of innovations that are developed. The disadvantage of case and interview studies is that they are usually unrepresentative of all public sector organizations. In contrast, statistically representative surveys can provide an industry or country-wide overview of the prevalence of specific innovation activities, the use of strategies to support innovation, and some types of innovation outcomes. These data can be used for benchmarking or tracking innovation performance. The application of econometric methods to survey data can also test theories, developed through case study and interview research, on the types of activities and strategies that support innovation. In this respect, surveys of public sector innovation can complement research based on case and interview studies.
This paper provides a framework for measuring public sector innovation through representative surveys, with a goal to guide policy to support public sector innovation and facilitate policy learning. After a brief overview of experience with surveys to measure innovation in the public sector, this paper evaluates the appropriateness to the public sector context of the Oslo Manual's definition of innovation and its guidelines for measuring innovation inputs, activities, and outcomes. While the general definition of innovation that underlies the Oslo Manual, and many of the manual's guidelines, can be applied to the public sector, research indicates that the guidelines are insufficient, largely because the types of data proposed for collection cannot address the full set of policy issues identified in the public management literature. In comparison with the private sector, policy interest in public sector innovation is more concerned with the innovation process, or how innovation occurs, in order to increase the use of innovation to solve problems and to improve innovation outcomes. This requires data on the strategic management of innovation, its governance, on how and where knowledge and ideas for innovation are obtained, the innovation culture, the personal characteristics of individual managers, and the capabilities and tools available to support innovation (Eggers and Singh, 2009).
We outline a measurement framework that identifies key policy needs for the promotion of public sector innovation and the types of data that are required to test theories and to help policy makers and public sector managers in their efforts to improve the innovation practices and capabilities of public sector organizations. In doing so, we build a bridge between the economics of innovation and public management disciplines, showing how their research methods and insights can be combined to better understand public sector innovation and the role of policy in promoting it.

History of measuring innovation in the public sector
Until the late 2000s, research on innovation in the public sector was primarily conducted by researchers interested in public administration or management, using case studies, interviews, and data on specific innovations within public sector agencies. Much of this literature focused on the effects of governance, organizational factors, and the personal characteristics of managers on innovation (Bernier et al., 2015).
In addition to case studies and interviews, management researchers evaluated awards or other data for individual innovations or asked public sector managers if their organization had adopted any of a predefined list of innovation practices or technologies. Relevant studies of individual innovations used innovation award data for the United States and for Canada and other Commonwealth countries (Borins, 2001;Bernier et al., 2015), innovation projects by American State governments (Vanagunas and Webb, 1994), pre-identified innovations in European countries (Pärna and von Tunzelmann, 2007), and innovations chosen by respondents to a survey of British public sector agencies (National Accounting Office (NAO), 2006). Research on the adoption of innovations in the public sector covered public administration agencies in Canada (Lonti and Verma, 2003;Earl, 2004), municipalities in the United States and Spain (Damanpour and Schneider, 2006;Moon and DeLeon, 2001;Gonzalez et al., 2013) and local authorities in England (Walker, 2006;Damanpour et al., 2009).
Dedicated innovation surveys and surveys that include a module of questions on innovation have only been used since the 2000s. They include a survey of 121 public sector managers in Norway and Finland conducted in 2002 (Laegreid et al., 2011) and an Audit Commission (2007) survey of 275 local authorities in England. These studies produced a number of valuable insights on public sector innovation and how to measure it. They were followed after 2008 by several large-scale surveys that used the Oslo Manual guidelines for measuring innovation in the private sector as a point of departure for measuring innovation in the public sector.
The first edition of the Oslo Manual only covered technological innovations in manufacturing. In order to cover services as well, later editions of the Oslo Manual had to address issues of demarcation, assimilation, inversion and integration between manufacturing and services in the business sector. Djellal et al. (2013) applied the same analytical lens to differences between service innovation in the business and public sectors. They found that the literature on innovation in public services had adopted several perspectives, ranging from a view that public services are fundamentally distinct from services provided by businesses to a view that they share many common features. Other research identified general differences between the public and business sectors that were likely to influence innovation activities, such as the lack of a market selection mechanism in the public sector (Bloch and Bugge, 2013). However, Djellal et al. (2013) also noted that most research on differences between the public and business sectors was based on ad hoc evidence, suggesting that more systematic data were needed to properly understand and characterize public sector innovation.
The first large-scale representative survey dedicated to public sector innovation was the 2008-2009 MEPIN survey of public sector organizations in Scandinavia, which obtained over 2000 responses from public sector managers. As far as possible, the MEPIN survey followed the guidelines of the third edition of the Oslo Manual (OECD/Eurostat, 2005), but several changes were required to the manual's definition of four innovation types (product, process, organizational and marketing) to account for the characteristics of innovation in the public sector (Bugge et al., 2011). For example, the manual's category of marketing innovation was changed to 'communication' innovation.
The MEPIN survey was followed by other public sector innovation surveys (see Table 1). There has been a continual learning process in which each survey has influenced subsequent surveys. For instance, the MEPIN survey influenced the design of the 2010 European Innobarometer survey, which in turn influenced the design of the 2012 Australian APSII survey. Table 1 also includes a 2015 survey of product, service, organizational and communication innovations in the healthcare sector that applied the lessons learned from many of the previous studies listed in Table 1. 2 The public management literature made a substantial contribution to the studies listed in Table 1 by identifying the factors that need to be measured in order to understand how innovation occurs (Djellal et al., 2013). Many of these factors are either not relevant to the business sector or taken for granted and consequently not covered in the Oslo Manual. For example, a key difference between the public and private sectors lies in the governance of control and decision-making, which is considerably more heterogenous in the public sector than in the business sector.
The majority of the surveys listed in Table 1 covered innovation in public administration and other entities, with few studies including publicly-owned corporations, in part because the latter are often covered in business surveys. Public administration is a useful target for innovation surveys because these organizations provide the framework for how public services are provided. For example, educational and health ministries are responsible for many (though definitely not all) of the major innovations that are implemented in subsidiary service providers such as schools and hospitals. Yet specialized surveys of the providers of public services are also necessary, particularly to identify innovations that are suggested and developed by front-line staff. Three of the surveys in Table 1 covered service providers, including MEPIN, a 2016 study of Australian universities, and a 2014 survey of public workplaces in Denmark, including schools, hospitals, nursing homes, daycare centers, police stations, etc. 3 In addition to surveys, we draw on extensive cognitive testing in Scandinavia, Spain, Belgium, Italy and Australia, with 118 face-to-face interviews with individuals drawn from the same population as potential survey respondents. The two main goals of cognitive testing are to ensure that respondents understand survey questions as intended and can provide reasonably accurate responses (Willis, 2005). With a few exceptions, the cognitive testing results listed in Table 1 have not been published, but the authors either conducted the testing themselves or had access to the interview transcripts or summaries for all but one study. The cognitive testing results are generally reassuring as they show that public sector managers in multiple high-income countries share a similar understanding of innovation concepts (Arundel and Huber, 2013;Arundel, 2014).

Applying the Oslo manual guidelines to the public sector
The Oslo Manual guidelines are the responsibility of the OECD Working Party of National Experts on Science and Technology Indicators (NESTI). The purpose of the Oslo Manual is to support the collection of statistically representative and internationally comparable data on innovation in the business sector. The fourth edition of the Oslo Manual (OECD/Eurostat, 2018) provides a universal definition of innovation that is applicable to all sectors covered by the System of National Accounts and includes a brief discussion of the value of collecting data on public sector innovation.
Although differences between innovation processes in the public and business sectors are widely recognized, there are several advantages for the measurement of public sector innovation to follow, where possible, the Oslo Manual guidelines.
First, National Statistical Offices (NSOs) have extensive experience with measuring innovation and the expertise to conduct large-scale surveys of innovation in the public sector. NSOs in more than 60 countries conduct innovation surveys in the business sector that comply with the Oslo Manual. Their interest and support in measuring public sector innovation is likely to be enhanced by using language and definitions that are similar to those used in the Oslo Manual. Indeed, the adoption of a universal definition of innovation in the fourth edition of the Oslo Manual means that National Statistical Offices are unlikely to implement surveys of innovation in the public sector that do not use a comparable definition.
Second, many innovation activities involve several sectors, with multiple linkages between public and business sector organizations in the development and diffusion of innovations. Collecting comparable data on innovation for the public and business sectors would provide insights into how these sectors are connected and identify the contributions of the public and business sectors to innovation. This would be particularly valuable for research and policy on innovations that address social challenges, which can require inputs from actors in several sectors.
Third, in some countries both public sector organizations and businesses are active in industries such as education, health and social services. In these sectors, comparability in the measurement of the innovation activities of both public agencies and businesses would improve research on the advantages and disadvantages of public and private sector provision of service innovations. However, many countries do not collect innovation data for businesses in sectors dominated by the public sector. In this case, a questionnaire designed for public sector organizations, with some modifications, could also be sent to businesses.
Fourth, the Oslo Manual is designed to collect innovation data for highly heterogeneous businesses that vary by size, ownership and sector of activity. Consequently the focus is on a common measurement framework that uses questions that are relevant to all businesses. The same approach can be applied to heterogenous public sector organizations, which vary considerably by size, area of responsibility (local, regional and national), function (regulatory, policy, internal services, etc.), and industry (education, transportation, health care, etc.). Recent experience indicates that it is possible to use a common framework to measure innovation in different types of public sector organizations, spanning different functions and levels of government (see Table 1). A focus on specific sectors such as healthcare can be accommodated through the addition of sector-specific questions. Silvander and Hagen (2015) provide an example of this approach for healthcare services while Arundel et al. (2016) provide an example for tertiary education.
However, there are substantial disadvantages to applying the Oslo Manual guidelines to the public sector. The following sections identify areas where the Oslo Manual is sufficient and areas where a new framework is required to guide the measurement of public sector innovation.

Definitions of innovation
Innovation measurement requires definitions to ensure that comparable data are collected for phenomena of interest. The Oslo Manual defines innovation in relation to the innovative organization: an innovation must be novel (new or significantly improved) to the organization, but there is no requirement for an innovation to be a market novelty. Applied to the public sector, the definition encompasses a broad range of innovations, from minor incremental improvements (Bugge and Bloch, 2016;Fuglsang, 2010) to disruptive or transformative innovations that completely alter or replace processes or services (Osborne and Brown, 2011). In addition, the fourth edition of the Oslo Manual recommends collecting data on different types of innovations, primarily products (goods or services) and processes.
The Oslo Manual's general definition of innovation that is applicable to all economic sectors in the System of National Accounts, including the public sector, is as follows: "An innovation is a new or improved product or process (or combination thereof) that differs significantly from the unit's previous products or processes and that has been made available to potential users (product) or brought into use by the unit (process)." (OECD/ Eurostat, 2018, p. 60).
The term 'unit' refers to the innovative organization, which can be a business, government ministry, municipality, etc. There is no requirement in the definition for an innovation to be normatively better than existing processes or products. An innovation only needs to provide a significant change compared to what was previously in use.
Cognitive testing has shown that public sector managers have few problems with the Oslo Manual's concept of novelty as something that is new or significantly improved for the entity itself or with the categories of service and process innovations, but managers' personal understanding of innovation conflicts with the implementation and normative requirements. Although implementation is understood, public sector managers struggle to follow this definition in practice because many public sector innovations are services or processes that are introduced over a long period of time (Arundel et al., 2016). In respect to the lack of a normative requirement, many public sector managers believe that an innovation must 'make something better' or have a goal of 'delivering better outputs'. Cognitive testing in Australia and Europe finds that the resistance to a definition that excludes a normative component is due to the frequency of restructuring in the public sector. Public sector managers do not view "rearranging organizational structures" as an innovation unless there is an improved outcome or benefit (Arundel, 2014).
The cognitive testing results suggest that it is possible to use a definition of public sector innovation that is compliant with the Oslo Manual (thereby permitting comparisons with innovation data for the business sector), but it may need to be followed by an explicit exclusion of restructuring or a reference to making something "better". Gault (2018) proposes the concept of 'restricted' innovation, in which the broad definition of innovation is restricted by additional requirements, such as a requirement for environmental sustainability. In the public sector, one method for maintaining comparability with the Oslo Manual would be to first use a universal definition that is compliant with the Oslo Manual, followed by a question on whether any of the entity's innovations made something 'better'. This would be an example of a restricted innovation. An ability to identify 'better' innovations is also of value for research on the management and measurement of strategically-driven innovation, where "success" is required to attract resources on an ongoing basis.
Researchers in public management have not used the Oslo Manual's definition of an innovation, possibly because they are not aware of its existence or because they prefer definitions based on restricted definitions of novelty. For example, Osborne and Brown (2011) argue that the emphasis of policy and research should be on disruptive or transformative innovations with more substantial outcomes than incremental innovations. However, although the theoretical concept of transformative innovations in the public sector has been verified in case studies (Lee and Lee, 2014), little is known about the prevalence of transformative innovations or the relative outcomes of transformative and incremental innovations. The value of using the Oslo Manual's broad definition of innovation is that it captures all types of innovation and consequently a broad range of outcomes. It is a relatively simple matter for surveys to include questions to identify incremental and transformative innovations and consequently support analysis of the types of inputs and strategies that are required for innovations of varying novelty and impacts. Limiting survey research on innovation to transformative innovation would also fail to provide useful data to support strategic management goals to encourage an "innovation culture" at all levels of a public sector organization.
Another challenge for applying the Oslo Manual definitions of innovation to the public sector lies with the types of innovation that have been identified by public management theorists. For example, Windrum (2008) identifies six types of public sector innovations. Four types (service, service delivery, administrative / organizational innovations, and systemic innovations) fit within the Oslo Manual definitions for service and process innovations, but policy innovations and conceptual innovations have no equivalent in the Oslo Manual. Windrum defines conceptual innovations as the "development of new world views that challenge assumptions that underpin existing service products, processes and organizational forms", while policy innovations include "incremental innovation based on policy learning by government and radical innovation sparked by conceptual innovation".
Innovation types need to be defined in a way that can be understood by all survey respondents. Questions on service and process innovations are likely to be understood by most potential respondents, including those with non-managerial positions. Policy innovation was included as an innovation type in Australian innovation surveys and was well understood by senior managers (Arundel and Huber, 2013), but conceptual innovations pose a greater challenge to define and may only be relevant to the heads of government organizations.

Relevance of Oslo manual themes to public sector innovation
In addition to recommending key definitions for innovation and innovation types, the Oslo Manual identifies thematic areas for coverage in an innovation survey: activities within the firm for the development and implementation of innovations, the use of internal and external sources of knowledge of value to innovation, innovation drivers, objectives and outcomes; and obstacles to innovation. For each theme, the Oslo Manual provides detailed lists of relevant items for measurement, such as the use of different knowledge sources or the importance of different innovation objectives.
All of these thematic areas are relevant for innovation in the public sector. As an example, research on e-government, public policy and public management has evaluated the role of factors such as collaboration, knowledge sources, innovation activities such as training, and innovation objectives (de Vries et al., 2018). However, the importance of specific items under a theme varies between the business and public sectors due to differences in what is measurable, what is contextually relevant, and what is important to policy. Table 2 summarizes differences in comparability between Oslo Manual themes and the requirements for public sector innovation surveys.
In respect to measurability, several types of data that can be collected for the business sector are not presently measurable in the public sector. An example is interval level data on innovation expenditures for different innovation activities, such as R&D, marketing, and training for innovation. Public sector managers can provide nominal (yes or no) data for different innovation activities, but they seldom collect data on expenditures and consequently are unable to report expenditures with an acceptable degree of accuracy (Bugge et al., 2011).
Second, there are stark differences in the relevance of some thematic areas between the business and public sectors. A basic difference in innovation drivers is the absence of a profit motive for most public sector organizations. In addition, many of the Oslo Manual themes include activities of low relevance to public sector organizations, such as R&D, engineering, and the acquisition of intellectual property rights. These three activities are less relevant in the public sector due to a focus on services and low internal capabilities for technological innovation (Earl, 2004).
Conversely, other activities that are of high importance to public sector innovation receive only limited coverage in the Oslo Manual. These include knowledge sourcing and collaboration with other public sector organizations (Sorensen and Torfing, 2012), management capabilities (Audit Commission, 2007;Pärna and von Tunzelmann, 2007), the personal characteristics of managers themselves, such as their level of education, previous experience in the private sector, and attitudes to risk (Damanpour and Schneider, 2006); workforce experience with innovation (Boyne et al., 2005), external factors such as citizen demand for service innovations (Hartley et al., 2013), and the involvement of customers or end-users in the co-creation of service innovations (Skalen et al., 2018;Svensson and Hartmann, 2018).
The third difference between the business and public sectors is relevance to policy. This has the largest effect on creating differences between the Oslo Manual recommendations for the business sector and the types of innovation data that need to be collected for the public sector. In many cases public policy needs can be met through adding new items to the main Oslo Manual themes, but in other cases data for new thematic areas are required, for instance to collect information on a public organization's innovation culture. The next section draws on the public management literature and survey experience to identify essential thematic areas for measurement that are not covered in the Oslo Manual.

A measurement framework for addressing public policy needs
Public policy requirements for innovation data depend on the potential or actual policy instruments for influencing innovation. For the private sector, the public policy focus is on supply-side instruments to create new knowledge of value to innovation, such as subsidies for research, training, external knowledge sourcing and collaboration activities; the maintenance of supportive regulatory systems such as for intellectual property, and more recently, policies to involve the private sector in addressing societal challenges (Kuhlmann and Rip, 2018) through demand-oriented instruments such as government procurement policies (Boon and Edler, 2018).
In contrast, policy interest in public sector innovation is increasingly concerned with how innovation occurs within public sector organizations, as part of a goal to increase the use of innovation to solve problems and deliver beneficial community-based outcomes. Public policies can directly influence the inner workings of public sector organizations through government or management direction, for instance through adopting a strategic management approach to innovation, or through actions that strengthen organizational innovation capabilities. In some cases improving outcomes could require changes to both governance and actions to promote innovation. An example is the roll-out of new assisted living technologies in Norway, which required policy actions to develop and implement new services and a new governance strategy to influence how public sector actors work with health care innovations (Bugge et al., 2018).
Key factors that influence the innovation process in the public sector include the role of governance in shaping innovation, the sources of ideas for innovation, the innovation culture of the public sector organization, the capabilities and tools available to managers to support innovation, and objectives, outcomes, drivers and obstacles. Each is discussed in turn below. In addition, we describe the use of questions on a single innovation that can obtain higher quality responses for some innovation inputs, outcomes and obstacles.

Governance and innovation
Public sector organizations can be under pressure to innovate for a variety of reasons, ranging from public demand for new or improved services to budgetary constraints (Walker, 2006;Hartley et al., 2013). How these diffuse pressures filter through to public sector organizations and the ability of these organizations to respond are influenced by the governance model. Although there is no universally accepted definition of governance (Bovaird and Löffler, 2003;Stoker, 1998), the term is commonly used to refer to processes or methods of control (Rhodes, 2007). For example, Briggs (2007) defines governance as "the set of responsibilities, practices, policies and procedures, exercised by an agency's executive, to provide strategic direction, ensure objectives are achieved, manage risks, and use resources responsibly and with accountability." Governance determines the relationship between the elected government and the administrative arm or public service. These two parts of the public sector are subject to different pressures and motivations and generate different types of innovations. Innovations from the elected arm are likely to be relatively large in scale and determine the tasks of the public service. Innovations developed and introduced by Table 2 Comparability between Oslo Manual topics for innovation measurement and data collection requirements for the public sector.

Oslo Manual topic Public sector comparability Comments
Innovation definitions Moderate Workable general definition of innovation available for both the business and public sectors, but the public sector includes innovation types (conceptual and policy innovations) that are difficult to fit within the Oslo Manual typology for innovation. Innovation activities Moderate Some of the activities covered in the Oslo Manual (R&D, acquisition of external knowledge such as intellectual property, engineering) are less commonly used in the public sector, while other activities (training and purchases of equipment) are frequently undertaken in the public sector. Innovation expenditures Low Difficult to obtain expenditure data for innovation in the public sector because internal investments focus on staff, with measurement in terms of personnel numbers or person-months for innovation. Knowledge sources High Good comparability, but public sector surveys need more details on government sources. Collaboration High Good comparability, but public sector surveys need more details on government sources. Drivers Moderate Common drivers for the business sector (profit and competitiveness) are less relevant for the public sector, but both share consumer demand as a driver of service innovations. Objectives / Outcomes Moderate The public sector lacks a sales measure for services, but shares qualitative outcomes such as quality, lower costs, speed of delivery, etc. Obstacles Low Similar interests in insufficient resources, but the public sector potentially faces many internal obstacles that are not discussed in the Oslo Manual, such as staff resistance, a negative innovation culture and risk aversion.
the public service are more likely to focus on how these tasks are met (Arundel and Huber, 2013). Governance also influences how innovation occurs. Under the traditional Weberian governance model, the decision to innovate is taken by elected politicians and the main method of innovation is technology adoption. In theory, this leaves little room for public sector managers to influence innovation processes and no role for middle managers or front-line staff. Due to fiscal pressures, the traditional governance model was gradually replaced during the 1970s and 1980s in many high-income countries by New Public Management (NPM). Authority was devolved to senior departmental managers who were encouraged to introduce organizational innovations copied from the private sector, such as performance bonuses, competitive tendering, outsourcing, and privatization to generate service delivery efficiencies (Hartley et al., 2013).
Experimentation with other forms of governance to encourage innovation has led to 'joined up government' and 'networked' governance models (Bysted and Jespersen, 2014;Hartley et al., 2013;Sorensen and Torfing, 2012). Joined up government requires collaboration within and across ministries (Hood, 1991;Laegreid et al., 2011), while the 'networked' governance model includes collaboration with external parties (Rhodes, 1996;Peters and Pierre, 1998;Borzel and Ruisse, 2010). Both joined-up government and networked governance models can support 'bottom-up' innovation in which middle management and front-line staff can contribute to innovation.

Bottom-up, top-down and strategic innovation
Governance methods can co-occur, such that different ministries follow different governance models. When viewed as a suite of alternative management models for specific contexts, governance models provide a range of options for managing innovation. Top-down innovations, which are directed by the political arm or senior management, are often associated with changes of government, new mandates or large scale initiatives, and can involve a combination of new policy goals and frameworks that are sometimes associated with new ideologies, as well as new concepts of services and service delivery. A key policy issue for top-down innovation is the conditions for successful implementation (and in many cases further development) of innovations. Bottom-up innovations are frequently created at lower levels of public sector organizations or individual work-units and rely on the initiative of individuals and work-units to identify and pursue innovations. For bottom-up innovations, key policy-relevant issues relate to employee competences, conditions that can foster innovation contributions by staff and middle management, and how innovations are diffused to other organizations.
While there is some evidence that governance methods that encourage 'bottom-up' innovation result in better outcomes than traditional policy-driven innovation (Arundel et al., 2015), the context is likely to matter. In some contexts, traditional governance methods or new pubic management could outperform governance methods that encourage bottom-up innovation.
Under all forms of governance, innovation in the public sector is often undertaken on an ad hoc basis, for instance in response to new policies, expectations of declining budgets, or demand for new or improved services. The alternative is to create a governance model that supports the strategic management of innovation, where innovation is an ongoing strategy for addressing service and efficiency goals throughout the public sector. The underpinning philosophy is to develop an organizational capability to systematically raise innovation outputs over time, instead of focusing on ad hoc projects or a reliance on external contractors that do little to enhance the organizational capability for innovation.
The development of a strategic management approach to innovation would benefit from a measurement framework that can identify how innovation is managed and the relationship between innovation management and governance on the one hand, and innovation capabilities and outcomes on the other. Research suggests that incremental and selfdirected innovation developed within work groups are likely to occur regardless of the form of governance (Bugge and Bloch, 2016;Fuglsang, 2010;Brown and Osborne, 2013), but the probability of producing transformative or 'breakthrough' innovations could be enhanced by governance that supports strategic innovation management.
Innovation surveys have not collected data on governance models or the presence of strategic management, partly because an announced governance method may not be fully implemented (as was often the case for joined-up government) and partly because popular theories of governance can influence the perception of senior managers without corresponding changes in how decisions are managed. Instead, governance methods can be indirectly identified through an analysis of data on strategies and capabilities, as in the study by Arundel et al. (2015), or specific questions can be included on the use of mechanisms that indicate the presence of a strategic management approach to innovation. Innovation activities that are influenced by governance, and which can be used to identify different governance methods, include the sources of ideas for innovation, ranging from front-line staff to politicians (Borins, 2001); the division of responsibilities, the innovation culture, managerial characteristics, and where innovation capabilities lie (internal to the organization or external). Strategic management could also be identified through the presence of mechanisms such as a written innovation strategy, the inclusion of innovation targets in annual reports, the participation of managers in an ongoing innovation task force, and the active participation of politicians in innovation (Torfing and Ansell, 2017).

Sources of the ideas for innovations
The sources of ideas for top-down innovations instigated by politicians and senior managers include new mandates due to a change in government, large-scale policy initiatives, new policy goals associated with new ideologies, or new concepts of services and service delivery. The ideas for 'bottom-up' innovations can be obtained by front-line staff and middle managers interacting with users of services or processes (Arundel and Huber, 2013;Borins, 2001;Bugge et al., 2011).
Policy can benefit from data on the source of the ideas for innovation and the benefits and limitations of top-down and bottom-up innovations, for instance if the value of bottom-up innovation is largely limited to incremental improvements to processes or services, as suggested by Fuglsang (2010), or if bottom-up innovations involve or make a useful contribution to innovation. This leads to an additional issue of high policy interest: the ability to differentiate between what is required for incremental innovation and what is required for transformative or disruptive innovation in the public sector (Bugge and Bloch, 2016). The answers to these questions have important implications for the governance of public sector organizations.
Measurement of the ideas for innovation should collect data on the source of ideas, including external sources outside the public sector (Lagunes and Rubalcaba, 2015), sources from different levels within a public sector hierarchy, and details on the characteristics of innovations (type, novelty, level of investment) by the source of the idea.

Innovation culture
An 'innovation culture' can be defined as "a culture where a group of people's shared values, customs and assumptions are conducive to new ideas and organizational change" (Bason, 2010). The literature suggests that the organizational or workplace culture for innovation is a significant factor in either supporting or hindering public sector innovation (Albury, 2005;Laegreid et al., 2011).
Many factors can influence the innovation culture, including the governance method and the personal characteristics of managers, such as whether or not a manager has an 'entrepreneurial mindset' (Damanpour and Schneider, 2006) or previous experience with innovation (Boyne et al., 2005). The motivation for senior and middle managers to innovate is an important factor in public sector innovation and closely related to the personal and professional development that flows from successful innovations (Halvorsen et al., 2005).
A lack of organizational support for innovation could depress public sector innovation or limit innovation to minor, incremental improvements (Osborne and Brown, 2011). Both within and across organizations, the context matters, with critical elements including the organizational level at which innovation is pursued (across the entire public sector, across specific ministries or departments, or only within specific work units), the nature of the challenge (if it can be dealt with through incremental innovation or if it requires transformative innovation), the amount of time available (if innovation is required because of a crisis or other externally imposed event (Kay and Goldspink, 2012)) and the function or purpose of the organization. A department of legal affairs could justifiably discourage innovation to retain the integrity of a riskaverse legal system, while a department of infrastructure or agriculture could have greater flexibility to pursue risky innovations (Williamson, 1999).
An innovation culture needs mechanisms for managing the risk of innovation failure, which can occur due to technological risks (Pärna and von Tunzelmann, 2007), rejection by potential users, or a lack of resources and capabilities for developing and implementing an innovation (Kay and Goldspink, 2012). A risk-averse organizational culture can hamper innovation by preventing experimentation (Borins, 2001;Brown, 2010;Kay and Goldspink, 2012;Brown and Osborne, 2013;Potts and Kastelle, 2010). Risk-averse conditions can be created by political damage to elected officials from media scrutiny of failed policies (Albury, 2005;Borins, 2001;Potts and Kastelle, 2010), reputational risks for managers (Pärna and von Tunzelmann, 2007) and self-selection effects whereby risk-averse individuals are more likely than 'risk tolerant' individuals to seek employment in the public sector (Buurman et al., 2012;Noussair et al., 2014;Pfeifer, 2010). Self-selection could contribute to higher levels of risk aversion among public sector managers compared to private sector managers (Chen and Bozeman, 2012;Hartog et al., 2002;Roszkowski and Grable, 2009).
Measures of an innovation culture include the level of support, leadership and experience of senior managers for innovation and the attitudes of both management and other personnel towards risk and change. Questions can also ask if specific strategies are 'supportive of innovation' or a 'hindrance to innovation', as in the survey by Silvander and Hagen (2015).

Capabilities and tools
Surveys of innovation in the business sector collect data on a range of activities for innovation, but rarely collect data on the methods used to innovate. The assumption is that business managers know how to innovate, for instance by drawing on trial-and-error methods developed in the manufacturing sector or scientific methods learned in higher education. Conversely, in the public sector we can't assume that most managers know how to innovate, although this is probably changing, with the adoption of design thinking and co-creation (user-centered design) by public sector agencies in many countries, including Scandinavia, the UK and the United States (Mergel, 2018;Sangiorni, 2015;Voorberg et al., 2017).
Measurement should capture the use of a range of capabilities and tools for innovation, including actions to encourage creativity and learning by public sector staff, incentives and rewards for personnel to suggest ideas for innovation and participate in their development, training and the use of innovation teams, collaboration, strategies that managers can use to manage risk, and the development of public services through co-creation with potential users. It is also of value to collect data on how innovation capabilities are managed, for instance if innovation activities are guided by design-thinking or change management methodologies that provide a structure for innovation activities.

Objectives, outcomes, drivers and obstacles
Innovation objectives and outcomes are often two sides of the same coin, for instance an innovation objective to reduce costs is matched by the observed effects of the innovation on costs. Similarly, drivers such as the availability of skills or funding can, through their absence, act as obstacles to innovation.
The Oslo Manual includes lists of relevant objectives, outcomes, drivers and obstacles to innovation for the business sector. Many of these factors are also relevant to public sector organizations, such as objectives to reduce costs, improve product quality, and improve workplace safety; drivers such as the need to meet regulations, citizen demand or address social challenges, or obstacles such as a lack of human or financial resources. Other factors are either unique to the public sector or should be phrased differently. These include political actions that drive innovation, including mandates, new laws, regulations and policy priorities; a problem or crisis that requires an urgent response, and organizational restructuring. Obstacles that are particularly relevant to the public sector include a lack of management support for innovation, a lack of incentives, staff resistance, a poor innovation culture, and different factors related to the risk of innovation, including a risk-averse organizational culture or high or unanticipated risks (Arundel et al., 2015;Bugge et al., 2011;Demircioglu, 2017;National Accounting Office (NAO), 2006;Pärna and von Tunzelmann, 2007;Torugsa and Arundel, 2016a).
A crucial need is to measure outcomes such as the value created (or destroyed) by public sector innovation, where value can be measured through changes in the efficiency of processes or changes in the objective or subjective value of services, as experienced by users. Outcomes are a challenge to measure in the public sector because of the absence of a single unified outcome, shared by all public sector organizations, that is equivalent to sales revenue from product innovations in the business sector. Most of the innovation outcomes that have been measured in public sector innovation surveys address generic subjective outcomes that are applicable to many types of public sector organizations, such as simpler administrative processes, faster delivery of services, improved user experience or access to information, and improvements to service quality. Attempts to obtain numerical data on the cost savings from process innovations have been unsuccessful, but cognitive testing shows that public sector managers are capable of providing categorical estimates, such as a '5% to 10% reduction in costs' when asked about a single, specified innovation.
A second challenge is potential bias from asking managers to report on their organization's innovations, which could lead to over-estimates of positive outcomes. Damanpour et al. (2009) avoid self-reporting by linking survey data on the adoption of innovations to an independent source of data on each organization's service performance score. This provides a measure of outcomes that avoids a possible bias towards successful outcomes, although it may not be known if innovations or other factors contributed to service performance. An alternative is to estimate the degree of possible bias by collecting data on whether reported outcomes are based on internal or external evaluations of the innovation (Center for Offentlig Innovation, 2015) and comparing outcomes for internal and external evaluations.
A third challenge is to collect data on outcomes after a sufficient period of time has passed for outcomes to be observable. To prevent memory recall biases, the Oslo Manual recommends asking about innovations that occurred in an observation period of the previous one to three years. This can be insufficient time to measure outcomes, particularly for innovations introduced in the most recent year of the observation period.
A method for reducing possible biases from self-reporting and to provide sufficient time for the observation of outcomes is to collect outcome data for a single innovation identified by the respondent (see below). This method could reduce bias by forcing the respondent to concentrate on a single innovation. If a description of an innovation is obtained, the information could be used in follow-up questions on outcomes after a suitable time lag (Gault, 2018). It may also be possible to independently survey the users when respondents report on a widelyused service innovation.
None of the methods for measuring outcomes through surveys are likely to be effective for long-term outcomes that occur more than three to five years after implementation, due to staff turnover or additional modifications over time that replace the original innovation. Long-term outcomes are best addressed through case studies. Case studies may also be required to assess complex outcomes that cannot be described in a questionnaire, where best practice limits questions to one or two sentences.

Collecting innovation data for single innovations
Many types of innovation data are relevant to the entire public sector organization (the subject), such as data on an organization's innovation capabilities and strategies, or the factors that drive an organization to innovate. In addition, data can be collected for a single innovation or 'object' (Bianchi et al., 2018), similar to the studies by Borins (2001) and Pärna and von Tunzelmann (2007). Surveys of innovation in the public sector have increasingly asked some questions at the level of the organization, while other questions are limited to a single innovation, which can be the 'most important', 'most successful', or 'most recent' innovation. It is also possible to ask about a single under-performing or failed innovation in order to obtain data on obstacles that lead to unsatisfactory outcomes. Methods for including questions on a single or 'focus' innovation in surveys are discussed in Chapter 10 of the Oslo Manual (OECD/Eurostat, 2018).
The inclusion of questions on a single innovation has substantial advantages: it permits questions that may be too difficult for respondents to answer for all of their innovations combined, for instance on the resources invested in the innovation, the source of the idea for the innovation, the outcomes of the innovation (including interval measures for efficiency), the contribution of other actors to the development or diffusion of the innovation, including businesses or the use of public-private partnerships (Bianchi et al., 2018), and possible obstacles faced during its development. If there are adequate resources for coding, a written description of a single innovation can also be requested. An alternative approach is to ask a series of questions about a pre-defined innovation, such as new types of health care services. This method could also be applied to both public and private sector health care providers, permitting comparisons between innovation activities and outcomes in both sectors, and to the users of these innovations.
With some exceptions, novelty is best addressed through questions on a single innovation. It is possible to collect data that can be used to determine if the innovation is a transformative innovation, the amount of person-months or number of staff members that were required to develop and implement it, or if it is the first use of this type of innovation in their country by public sector organizations. The latter question is relevant to research and policy on the diffusion of innovations.

Conclusions
We draw on public management theory and empirical research on public sector innovation to propose a framework for the collection of micro-level data of value to research and public policy to support public sector innovation. A public policy issue shared with business innovation surveys is to collect data for benchmarking innovation prevalence and activities. The framework proposed in this paper, such as the use of a general definition of innovation that is compatible with the Oslo Manual definitions, would permit benchmarking innovation activities between the public and business sectors. However, policy interest in public sector innovation extends to topics that are not covered in the Oslo Manual, including the governance and strategic management of innovation and the nuts and bolts of how public sector organizations innovate.
The policy interest in the nuts and boltswhich includes the innovation culture, the strategies and tools that managers use to innovate, and innovation incentives -is based on the effect of these factors on innovation outcomes. Innovation surveys need to provide sufficient data to identify 'what works' and what hinders innovation. Analysis of these factors is also required for specific types of innovations, as what works for developing a new service could differ from what works for improving an internal process.
Better data collection on innovation in the public sector would create a new research program, in the same way that the implementation of the Oslo Manual guidelines in multiple business innovation surveys resulted in a new research program on innovation in the business sector, leading to valuable insights for innovation theory and policy. The number of academic papers using European Community Innovation Survey (CIS) data on business innovation increased from less than 10 per year in the 1990s to close to 100 in 2013, in part due to CIS data creating new opportunities for innovation research .
The creation of a new research program on public sector innovation would be greatly assisted by government support for data collection. A first step in this direction would be the formal establishment of international guidelines on the measurement of public sector innovation by the OECD's Working Party of National Experts on Science and Technology Indicators. This paper is intended to contribute to possible future deliberations by this expert group.
There is also a need for additional experimentation in the collection of data on public sector innovation. While there is good evidence for how to collect high quality data on the strategies and tools that managers use to innovate and the innovation capabilities of their organizations, so far surveys have not attempted to capture the existence and effectiveness of high-level strategies to support innovation, such as a strategic management policy for innovation. Adding a strategic and systematic perspective to data collection would permit analyses that can link innovation capabilities and practices to the motivation or 'why' of innovation activities, the types of innovations that are produced with and without a strategic approach to innovation, and differences in innovation outcomes by management strategies to support innovation. An ability to measure management strategies for innovation and how they are implemented at different levels in an organization would assist both theory and practice (de Vries et al., 2018). Furthermore, a strategic approach to public sector innovation would support research on the optimal boundaries for the division of responsibility for innovation between the political and administrative wings of the public sector.
While the focus of this framework is innovation in the public sector as opposed to all providers of services such as education or health care, it would be worthwhile experimenting with using the same (or similar) survey questionnaire to collect data from businesses active in the same service industry. This would permit direct comparisons between service innovations provided by the public sector and by private businesses.
Experimentation should also further explore the use of questions on a single innovation, which could illuminate differences in the source of ideas for different types of innovations and potentially provide high quality data on innovation investments. A crucial area for experimentation is to improve data on innovation outcomes in the public sector. This requires better subjective data on the value of service innovations to citizens and interval level data on the contribution of public sector innovation to efficiency improvements. Surveys are not the only possible source of representative microdata on innovation. Under some conditions, web-scraping can provide timely representative data at lower cost than surveys. Web-scraping uses textual analysis to identify innovation activities that are posted on the websites of public sector organizations. This method has been used to identify the adoption of pre-identified innovations in waste treatment and health care by public sector organizations (Bianchi et al., 2018) and the knowledge transfer activities of public universities (NESTA, 2018). The technique, however, faces several challenges when used to collect representative data on innovation activities and strategies, due to a lack of consistency in the information provided by public sector organizations on their websites. Divisions within a government ministry, for example, may lack a website of their own and lack space on a ministry website to report their innovation activities. Organizations are also likely to post different types of data. For instance, a few public sector organizations could post all of their collaboration partners for their innovations, while others could fail to mention any collaboration partners.
Of note, experience with using surveys to measure public sector innovation is largely limited to the European Union and Australia.
Research is required to determine if current methods are applicable in other regions and in low and medium income countries.
Representative data on public sector innovation could support a diverse range of policy and research interests, including the different motivations, roles, responsibilities, and contributions of various actors. These data would draw on and complement the findings of in-depth case and interview studies on similar topics. Public policy would also benefit from data-driven research on the differences between change and innovation and the potentially destructive nature of innovation, the nature of leadership, the internal resource allocation process, the optimal balance between exploitation and exploration (March, 1991), the effect of organizational structures on capabilities, and the achievement of optimal outcomes. The overarching goal is to collect data that can better direct public policy-makers' efforts to building a more innovative public sector.