The State of Policy Instrument Analysis: Knowns, Unknowns and the Current Research Agenda

It is time to take inventory of the knowns and unknowns about policy tools. The study of this field dates back to Lowi and others who have developed many typologies and theories in classic works such as Hood, Salamon, Linder and Peters, Peters and van Nispen, Schneider and Ingram, Lascoumes and Le Gales. Furthermore, policy instruments have become a relevant topic in many policy fields due to their theoretical and empirical relevance (overall, policies are made and pursue their goals through policy instruments). This is important work that is linked closely to current research on policy design but, despite much effort, many fundamental issues remain unknown or under-studied with respect to the topic. The current paper examines four clusters of basic issues in the field which require further research. Presented at the International Conference on Public Policy 4 Montreal, Canada June 26, 2019


Introduction
Policy tools, or the techniques through which governments generate, evaluate and implement policy options, have been the subject of research throughout the history of the policy sciences. The study of the field dates back to Lowi and others who developed many typologies and theories on the subject in the period 1950-1980 as well as works such as Hood, Salamon, Linder and Peters; Peters and van Nispen, Schneider and Ingram, and Lascoumes and Le Gales in the era since then.
This work is important work for policy scholars interested in explaining policy dynamics as well for those adopting a policy design perspective (Howlett 2019). In fact, it can now be argued convincingly that an instrument-based approach can better enlighten the nature of policy dynamics, from a prescriptive point of view, and assist policymakers in taking more effective decisions than can a purely input-based approach.
Despite much effort, however, many fundamental issues remain unknown or under-studied and there are key elements concerning individual policy tools. Central to all these studies is the need for a clearer understanding of the policy instruments arrayed in a policy mix, as well as how these tools relate to each other when combined, what impact this combination has on the effectiveness of policies both when enacted and how these packages or portfolios evolve and change over time.
It is time to take inventory of what is known and unknown about policy tools. This paper presents a survey of informed thinking about policymaking, policy processes, the manner in which tool choices are made, and the ways that tool bundles evolve over time. It examines four clusters of basic issues in the field which require additional research. The level of knowledge about each of these issues is established, forming the basis for a multi-pronged research agenda into the subject.

The Emergence and Consolidation of Policy Instrument Research
There are many ways to study public policy which, given its processual nature has always involved consideration of policy dynamics and change. This has most commonly been approached through an 'input' lens, in which the actors and their interactions in the policy process have always been at the core of studies in the discipline (Araral et al. 2013;Capano 2020). The most commonly adopted theoretical frameworks-from the advocacy coalition framework to the punctuated equilibrium theory, from the multiple stream approach to the narrative policy framework (Weible and Sabatier 2017)-all examine the dynamics of actors in attempting to make sense of policymaking.
However, there has been a parallel stream in policy studies that has been more focused on the examination of the outputs of policy-making, analysing in detail the content of the adopted decisions (Salamon 2002). These are conceptualized in terms of policy instruments with scholars working backwards from outputs to inputs in assessing policy-making not as an open-ended struggle between ideas or interests but rather as a process of choosing or selecting tools to address policy goals. This stream has its roots in the work of authors such as Salamon and Vedung (Vedung 1998;Eliadis et al. 2005;Bendor et al.2009;Sidney 2007;Radaelli and Dunlop 2013;Howlett 2014;Howlett et al. 2014), although Theodor Lowi's theoretical work (1972; can be considered the precursor to these works given his analytical focus on the content of governmental decisions and their capacity to address policy targets' behaviour in some fashion. This field has recently added an additional degree of complexity as there is now a shared scholarly view that the actual sets of adopted policy instruments, in every policy field, have a mixed nature and feature different patterns and trajectories of change over time (Gunningham and Sinclair 1999;Bressers and O'Toole 2005;Howlett 2005;Del Río, 2010;Grabosky, 1995;Justen et al, 2013b;Leplay and Thoyer, 2011).
The intrinsic and inescapable mixed composition of adopted policy solutions makes the study of public policy instruments based on examinations of single tools particularly problematic and urges a reconsideration of the policy instrument approach, clarifying its actual state and possible solutions to its shortcomings.
These mixes "feature the use of combinations of different kinds of policy tools (market-based, hierarchical, network and others) whose exact configuration changes from location to location" (Rayner et al 2017, 473). Such mixes are complex, given the nature of the tools involved and how they relate to eachother. These include interactive effects among policy tools (Boonekamp, 2006;Justen et al., 2013aJusten et al., , 2013bYi and Feiock, 2012), counterproductive effects among policy instruments, and synergies (Lecuyer and Bibas, 2012;Philibert, 2011). Tools in an instrument mix, for example, can be considered consistent when they work together to support a policy strategy (Kern and Howlett 2009;Rogge and Reichardt 2016).
There is a wide consensus in policy design literature, however, that not all tools are inherently complementary (Boonekamp, 2006;Del Río, Calvo Silvosa, and Iglesias Gómez, 2011;Grabosky, 1995;Gunningham et al., 1998;Gunningham and Sinclair, 1999;Howlett 2017;Tinbergen, 1952) and that some generate counterproductive responses to policy targets Ingram, 1990, 2005). Counterproductive effects may be manifest when command and control regulation is used alongside voluntary compliance (Grabosky 1995). Complementary effects may be recorded when command and control regulation minimizing undesirable modes of behaviour are employed alongside financial incentives to promote more desirable ones (Hou and Brewer 2010). However they can also be neutral, conflicting, or overlapping in the case of renewable energy and building energy efficiency (Del Rio 2010; Rosenow et al. 2016). Further, some combinations of tools may be superior in reinforcing or supplementing an arrangement (Hou and Brewer, 2010), while others may unnecessarily duplicate in one context but advantageous in another (Braathen, 2007;Braathen and Croci, 2005). A key principle of current policy design thinking, therefore, is to maximize supplementary effects while mixes are developed (Daugbjerg, 2009).
But knowledge is limited with respect to how such policy mixes are designed.
Obviously there are many studies that try to conceptually grasp how policy instruments are chosen by policymakers, but there is no real systematic analysis about how designs develops over time. There are some theoretical and empirical studies that focus on the dimensions of policy design whereas the political capacity/will of governments and their technical capacities are taken into consideration to align why and how policy mixes are designed to produce good policy design, that is, policy mixes that are more or less coherent, consistent and congruent Howlett and Mukherjee 2018;Capano 2018). These studies, however, should be considered only the beginning of a potentially relevant research stream.

Studying policy tools: Knowns and unknowns
Because of the recent moves towards the enhanced study of policy mixes and policy design, the lacunas of the instrument literature today are more pronounced than ever before. This new focus on policy instrument mixes underlines the need to pay more attention to the actual way in which policies achieve expected results as well as the nature of the interactions of tools within mixes. More research is needed to order the complex world of designing policy mixes and above all to disentangle how different factors drive the designing of good or bad policy mixes.
In general, policy instruments have been the topic of continuous research and analysis to understand the following, and some progress has been made on most of these issues:

1.
Why and how policymakers choose, recurrently, particular instruments with respect to others, and whether and how they change previous choices (Salamon 2002;Hood 1983;Peters 1989, 1998 Capano and

7.
What are the basic kinds of tools and whether, for example, distinctions such as those mooted between 'procedural' and 'substantive' tools or between 'implementation' and 'non-implementation' oriented tools serve a useful purpose (Howlett 2000) 8.
How well tools work alone and/or whether they work better, or worse, in combination (Taeihagh et al. 2013).
Knowledge generated in this research has created the basis for an improved understanding of the nature of policy instrument choice and enhanced the notion that it is possible to design public policy in a sophisticated way.
However, the pattern of research on policy instruments has developed in a very uncoordinated way, and is uneven. Despite the richness of the literature there are still many analytical "black holes", theoretical lacunas and an excess of descriptivism. This is necessary to address if the policy instruments approach is to proceed towards an effective process of scientific cumulation. In particular, there is a need to deepen knowledge of many relevant dimensions of the policy instrument approach in order to address unresolved questions such as why policymakers choose some instruments over others, whether policy instruments directly impact policy performance, how to study the characteristics and the effects of policy mixes, and how policy instruments truly work when delivering their outcomes. In answering these questions, a clearer understanding of policy tools, mixes and design is required. However, these are lacking at present.

Problems with understanding instrument and mix dynamics
The first category of problems deals with missing empirical studies that, consequently, create or reinforce theoretical shortcomings (in terms of undertheorization and even over-theorization) about important aspects of policy mix design Problems with understanding instrument and mix dynamics 1. Sectorial and National variations and patterns /styles in tool use including links of tool choices to national traditions in administration and implementation 2. Temporality issues (sequencing, trajectories, critical junctures, volatility of policy mixes) 3. Sectoral convergence and intersections and its impact on policy mix coherence 4. Links of tools choices to ideational paradigms and paradigm changes Under-Examined behavioural issues

Identifying types of targets and what motivates
targets (policy-takers behaviour and compliance) 6. How decision-makers ("policy makers") design/discover/decide upon tools Measurement and methodological issues 7. Describing and delimiting the mechanisms which tools activate to attain their effects and impacts 8. Defining and measuring the types of tools found in policy mixes 9. Measuring performance & effectiveness of tool deployment and mixes Implementation Issues 10. Understanding the volatility of tools and mixespredicting risks of failure and perverse outcomes) 11. Understanding government capacity and its impact on tool choices and use 12. Multilevel governance dimensions of tool choice and programme creation 13. Calibrations (substantial or procedural ones) -what they are and how they are selected 14. Procedural tools and how they relate to mixes and evolution. Despite the very large amount of work which has been done recently on policy tools and mixes, several large gaps remain.

Better understanding sectorial and national variations and patterns /styles in tool use including links to national traditions in administration and implementation
Sectoral variation is one of the most evident problems in policy instrument research which current research has described but failed to address theoretically or The key point which has not been systematically addressed has to do with the persistence of specific patterns of instrument preferences and adoptions either by sector or jurisdictionally, or both. The insights of Freeman (1985) concerning sectoral policy styles and patterns of instrument deployment have been under-investigated. Are there can be different types of policy instruments adopted in different policies according to the characteristics of the policy issues? Can we expect to, for example, have more incentive-driven or cooperative-based instruments in environmental policy (Bouwma et al. 2015) than in education policy (Hannaway and Woodroffe 2003)?
Similarly, we do not know much about why and how national variations develop in a comparative perspective and thus whether and how these variations are due to national policy styles, national administrative traditions, or to the characteristics of bureaucracy. We know that policy styles exist and that national traditions in administration and implementation are crucial. There is, in fact, a stimulating empirical literature showing that there are national policy styles of formulation (Howlett and Tosun 2018) and of implementation (Tosun and Treib 2018), and that these different ways of designing and implementing policies must be contextualized into the related types of politico-administrative regimes (Pollitt and Bouckaert 2011). In addition, these theoretical lenses and empirical evidence only design the borders of the contexts in which policy instruments are selected over time. We know that there are particular styles and politico-administrative traditions and assume that policy instruments are handled in different ways accordingly. There is also some empirical evidence that these styles tend to become highly institutionalized and persist over time.
However, there is no significant empirical research on the instrument-based dimensions of national styles and politico-administrative regimes, although it is intuitive that these styles should influence the way instruments are designed and then implemented. Here, the most intuitive hypothesis is that the institutionalization of policy/implementation styles as well as the characteristics of politico-administrative regimes should create permanent effects and path dependency in terms of instrument adoption. However, we also know that new instruments have also been adopted in countries where they could be considered novel (as a consequence, for example, of the diffusion of New Public Management, New Environmental Tools, and Evaluation) as different methods of implementation have also been introduced.
Thus, there are many unanswered empirical questions about the relationships between policy styles and politico-administrative regimes. All in all, this means that "comparative policy instruments research" is quite undeveloped and consequently there is often a mismatch between empiricism and conceptualization and between the descriptive nature of instruments and the typologies offered by the literature, as well an under-theorization of the causes of the variations between sectors.
Temporality issues (sequencing, trajectories, critical junctures, volatility of policy mixes) In addition to these spatial issues, the temporal aspect is obviously a constitutive dimension of policy instrument research. We know that time makes a difference in how instrument choices and patterns evolve and that temporal sequences is a crucial (Taeihagh et al. 2013;Justen et al. 2013aJusten et al. , 2013b component of policy mix design and evolution. Through a specific organization of events over time, for example, policy instruments can undergo permanent effects and thus becoming highly institutionalized and/or generate positive/negative feedbacks. And policy solutions -a set of adopted policy instruments -at a given point in time may enshrine problems as well as specific tools that actors must eventually confront in the next decision-making moment. Thus, negative and positive feedbacks of the adopted policy instrument(s) will inform the policy debate such that the adopted instruments will either be adopted or will result in a crisis at the critical juncture which could be then an occasion for a shift from the adopted set of policy instruments. Policymakers, for example, can proceed by layering, conversion and drift (Thelen 2004).
All of these terms, however, are not clearly defined or consistently employed by various authors working in the field and, in some studies, are used without an appropriate understanding of the underlying concepts or methods necessary to analyse and evaluate a policy mix.
Layering is thought to be the most commonly adopted process but layering can be done in different ways. That is, policy instruments can be assembled through processes such as packaging, patching, stretching, and bricolage (Howlett and Rayner 2013;Capano 2018) which range in terms of coverage and deliberativeness. At the same time, related to the produced effects, layering can lead to policy instruments mixed in a consistent, counterproductive and tense way (Capano 2019) or not.
However, if we know something about these general modes and types of policy instrument design, we are missing reliable empirical knowledge about the microcomponents of these different kinds of processes. There is a need to consider the effect of policy formulation processes on the character and effectiveness of complex policy mixes (Feindt and Flynn, 2009;Kay, 2007;Larsen, Taylor-Gooby, and Kananen, 2006), for example when existing mixes developed unsystematically through processes of policy layering (Thelen, 2004;Van der Heijden, 2011;Carter, 2012;Howlett and Rayner, 2007;OECD, 1996). From this point of view then, we require better empirical evidence of the foundations of design/non-design continuum. This would mean, for example, better operationalizing concepts such as package and patching, as well tense layering, in terms of instrument-based content.
There are thus many empirical gaps when the temporal dimension is considered.

Sectoral convergence and intersections and its impact on policy mix coherence
A related item refers to change processes that occur in policy mixes when activities in otherwise distinct subsystems transcend old boundaries and affect the structure or behaviour of other subsystems (Dery, 1999;Lynggaard, 2001;Djelic and Quack, 2007;Kay, 2006). Instances such as those that have occurred when Internetbased computing collided with existing telecommunications regimes and when longestablished natural resource policy actors find it necessary to deal with Aboriginal and claims exemplify this phenomenon (Hoberg and Morawaski, 1997;Grant and MacNamara, 1995;Rosendal, 2000;Gehring and Oberthur, 2000;Marion, 1999;Rayner et al., 2001).
This particular process of policy mix change has received almost no treatment in the literature. But a large research agenda exists here. Thus, for example, subsystem interactions can occur in specific issues without any permanent change in subsystem membership (subsystem intersection) or they can be more long-term in nature (subsystem convergence). These general processes affect policy mixes largely through the introduction of new instruments into otherwise stable regimes (Deeg, 2007;May et al., 2007). What is the impact of such changes? Which tools remain and which are removed? How does this vary between intersection and convergence? These are all key questions requiring more research.

Links of tools choices to ideational paradigms and paradigm changes
Policy paradigms, belief systems, and ideas matter when policy instruments are also at stake in policy change (Hogan and Howlett 2015) and their impacts on policy tools little known. Such paradigms provide general guidance to policymakers because their normative and cognitive dimensions structure the goals they pursue. But they also affect considerations of the appropriate tools for achieving those goals. The ideational turn in political science and public policy has generated relevant attention on how paradigms, beliefs and ideas can drive the choice of policy instruments.
However, the results are quite ambiguous and not definitive. Surely the concept of paradigm is useful to analyse policymaking from a macro-meso perspective and thus to outline general trends. In fact, when policymaking is analysed from a microperspective, different ideas, frames, and belief systems are usually competing and confronting each other in relation to the instrument choices. In addition, this variety of ideational drivers can be one of the causes of initiating or the institutionalization of policy mixes. For example, over time new paradigms/ideas/frames can and do emerge, but the older ones are not dissolved (Lieberman 2002;Oliver and Pemberton 2004).
This can lead to obvious conflicts and confrontations affecting tool choices if different instruments are pursued in each approach. Thus, existing policy mixes can be generated by the layering of different paradigms/frames over time or by an agreement between different actors holding different cognitive and normative beliefs with respect to policy problems and the instruments chosen to deal with them.
All in all, we lack understanding of the real role of paradigms, frames, ideas, and beliefs when policy instruments are at stake; this can be considered disappointing if we recall the relevance of the "ideational turn" in public policy.

Under-examined behavioural issues
A second major cluster of issues is related to the behaviour of policy-makers and policy-takers. There are different modes for designing policy and thus different types of the design itself in terms of instrument content and how the behaviour of policy makers intersects with that of policy takers is a key issue which existing research has failed to address systematically.

Identifying types of targets and what motivates targets (policy-takers behaviour and compliance)
Most of the concerns raised above, when they have been examined, have been analyzed in the case of policy-makers. But there is a large second area of concern which also exists: that related to the adverse or malicious behaviour of policy "takers". This issue also has to do with mendacity and/or Machiavellian behaviour on the part of policy-takers, a subject often glossed over in studies of policy compliance and 'target behaviour' (Howlett 2018).
Here the idea commonly found in the policy literature is that the only real issue in policy compliance is merely a matter of "getting incentives (and disincentives)' right" (Howlett 2018). This not only ignores aspects involved in the social and political construction of targets highlighted above Ingram 1990a, 1990b), but also minimizes the complex behaviours which go into compliance, most notably considerations of legitimacy, but also related to cupidity, trust and other social and individual behavioural characteristics as well as the operation of a wide variety of descriptive and injunctive social norms (Howlett 2019;Bamberg and Moser 2007;Thomas et al 2016).
Not the least of the problem with this view is that it has a notion of policy-takers as static targets who do not try, or at least do not try very hard, to evade policies or even to profit from them (Howlett 2019;Braithwaite 2003;Marion and Muehlegger 2007).
Such activities on the part of policy takers, however, are key in determining the success of various government initiatives ranging from tobacco control to bus fare evasion (Delbosc and Currie 2016;Kulick et al 2016) and should be 'designed for' in the sense that determined non-compliance and gaming should be taken into account in designing policies, along with many other such behaviours, such as free-ridership, fraud and misrepresentation (Harring 2016). As it stands, these are often thought of as purely 'implementation' issues and left up to administrators to deal with rather than forming an essential component of policy formulation and design (Doig and Johnson 2001;Kuhn and Siciliani (2013).
But understanding this behaviour on the part of policy takers is central to better understanding policy tool selection and effectiveness.

How decision-makers ("policy makers") design/discover/decide upon tools
Policy design is a specific form of policy formulation based on the gathering and application of knowledge about policy tools to the development and implementation of programmes aimed at the attainment of desired policy ambitions. In a time when policymakers are often tasked with developing innovative solutions to increasingly complex policy problems, the need for intelligent design of policies and a better understanding of the policy formulation processes they involve has never been greater.
In general, a means-ends understanding of policy formulation permeates the policy design orientation (Colebatch 2018). Although policy-making does not always necessarily lend itself to or result in purely instrumental thinking about policy issues, this instrumental orientation is significant in that policy formulators operating in accordance with its strictures are expected to base their actions on analyses which are logical, knowledge and evidence-based (Bhatta 2002).
Conceptually, an instrumentally oriented policy design process begins with an assessment of the abilities of different policy tools to affect policy outputs and outcomes, and considerations of the availability of the resources required to allow a policy to operate as intended. As Linder and Peters (1991) noted, this involves a series of choices which emphasize "not only the potential for generating new mixtures of conventional solutions, but also the importance of giving careful attention to tradeoffs among design criteria when considering instrument choices" (p. 130).
Designing policies in this way requires an understanding of how the use of specific kinds of instruments affects target group behaviour and compliance with government aims (Weaver 2015) and knowledge of the many constraints on tool use originating in the limits of existing knowledge and prevailing government priorities and governance structures (Torgerson 1986). It also requires both analytical and evidentiary capacity on the part of the government as well as the intention to exercise it (Howlett 2015). Whether or not policy-makers have these capacities is an outstanding issue in the field which half the essays in this special issue will address.

Describing and delimiting the mechanisms which tools activate to attain their effects and impacts
Recently, developing a mechanistic approach to policy design has been proposed . Regarding underlying mechanisms, policy design (and thus policy instruments research) can be explanatory without the need for broad-reaching theories while providing a deeper understanding in terms of answers to the why and how questions and on how policy instruments can reach or not reach their effects, that is, on how policy instruments/mixes can directly encourage or structure policy targets' behaviour to achieve the expected results.
Consequently, a mechanistic approach to policy instruments could offer a better understanding, and possibly an explanation, of the potential links between policy design and its effects.
This approach implies that policy solutions should be seen as a set of policy instruments, the adoption of which is conducive to achieving the expected results.
Accordingly, the key analytical point is how the adopted solution can be a genuine driver of the pursued outcome, that is, capable of generating the proper mechanisms and thus the causal chain. From this perspective, policy instruments and their mixes can be considered as "activators" of specific mechanisms conducive to the expected results. A mechanistic perspective could be a very promising way to understand policy dynamics and the development of policy mixes over time that also allow for the inclusion of outcomes.

Measurement and methodological issues
This third cluster of key issues involves a set of methodological issues around measuring and operationalizing policy tools and tool interactions. These cause difficulty in the measurement of many relevant dimensions of policy instruments and instrument mixes needed to understand aspects of their evolution and change.

Defining and measuring the types of tools found in policy mixes
The recent literature on the subject has shown some significant efforts in measuring policy mixes (which can be considered a success). Thus we can measure the "density" (intended as the number of policy instruments enacted in a policy mix) and the "intensity" (intended as the grade of significance/stringency assigned to the adopted policy instruments) of policy mixes. The research on policy intensity has been particularly well-developed; starting from the use of expert panels or media coverage (cite), there has now been a convergence in measuring policy intensity in terms of focusing on "objectives", "settings" and "calibrations" (Knill et al. 2012), and thus assessing the amount of resources, effort and activities invested in the adopted instruments (Schaffrin, Sewerin and Seubert 2015). This focus on intensity has allowed measuring the development of policy dynamics in terms of balance among different policy instruments as well as change in intensity (Schimdt and Sewerin 2018).
Other dimensions of policy mixes that can be measured are related to the way their components (goals, instruments, context) relate to each other; these include the grade of consistency, coherence and congruency of mixes Rogge Kern and Howlett 2017). However, empirical research has not yet sufficiently developed with regard to measuring these characteristics. This can be due also to the fact that there is still a certain disagreement about the definition of these concepts especially regarding the semantical difference between consistency and coherence (Rogge & Reichardt 2016). At the same time, more empirical research on the effects of the level of coherence, consistency and congruency of the implementation on the output, and the outcomes of the policy design would be quite welcome; there is a need to understand whether and how the interaction between goals and instruments, old instruments and newly adopted instruments, and different policy mixes functioning in different policy sectors empirical research. For this, scholars should try to be less speculative on the theoretical definitions of terms and find a functional agreement to develop more empirical research.

Measuring performance & effectiveness of tool deployment and mixes
Governments design policies to reach specific goals, that is, to change the quality of the performance of the addressed policies. Governments reform education policy because they want more well-educated citizens and more citizens with degrees; they intervene in environmental policy because they want to pursue, for example, decreased pollution and better water quality, etc.
While the link between policy instruments and their outcomes is indirect and limited (Koontz and Thomas 2012) and policy performance is co-driven by many other factors, it remains the case that the main method through which governments can steer their policy systems is by adopting specific sets of policy tools that address the behaviour of specific targets and beneficiaries. Thus, the policy mixes that governments design could help to readdress the way policies function as well as their performance. However, we do not know much about this linkage between policy instruments and policy outcomes, although some recent research has shown that some policy instruments and some mixes are associated with better performance .

Implementation Issues around Policy Tools
A fourth set of issues is related largely to the administrative details and considerations which go into putting policy tools into practice, or administering policy mixes. These are quite wide-ranging in themselves but cover many essential questions whose answers should drive design and choice considerations.

Understanding the volatility of tools and mixes -predicting risks of failure and perverse outcomes
Policy design studies to date have focussed almost exclusively on the "good" side of policy formulation, that is, dealing with concerns around ensuring that knowledge is marshalled towards developing the best feasible policy in any given context under the assumption of well-intentioned governments and accommodating policy targets. This work has looked at issues around how policies evolve over time and focused upon understanding how such policies can be made more robust and resilient but without carefully examining or allowing for the possibility that government intentions may not be solely oriented towards the creation of public value, or that policy targets may indulge in various forms of 'misconduct' from fraud to gamesmanship, undermining government intentions of whatever kind.
While self-interested, corrupt or clientelistic policy-making has been the subject of many studies in administrative and regulatory law, even the best of policy intentions can be perverted in implementation and the need to design policies to be resilient against conscious and determined efforts on the part of policy targets to undermine them is pressing. Although the question of intentional efforts to undermine or pervert policies and programmes in less than benign ways on the part of policy targets has only now become a source of interest among policy scholars some lessons can be learned from these other studies of policy tools and instruments about improving policy designs to deal with malicious behaviour.
These aspects of policy-making and policy design constitute the degree of 'volatility', found in a policy area, that is, the likelihood or propensity of certain instruments and certain design situations to lead to unstable policy mixes. While linked to design, this is due to the deployment of instruments and tools which by their nature inherently involve a high risk of failure. This can be contrasted with more stable tools and mixes in which designs are likely to approximate the image often set out in the literature on the subject.

Understanding government capacity and its impact on tool choices and use
The process of selecting policy instruments, as well that of assessing the effects of the adopted policy design, depends on the resources and capacities at the disposal of governments. Regarding resources, Christopher Hood (1983) has shown us that governments can use four main resources in designing and delivering policy instruments: nodality, authority, funding and organization. At the same time, we know that these resources are not sufficient in properly choosing instruments, because it is necessary to have specific policy capacities-analytical, organizational and systemic (Wu et al. 2017)-thanks to which the four resources at governments' disposal can be used to design new policies or new instrument-based interventions.
However, for both resources and policy capacities, there are at least three significant empirical gaps. First, we do not know whether governments are sufficiently aware of their eventual weakness in the necessary policy capacities. Second, there is insufficient empirical evidence on whether and how the characteristics of the actual stock of resources and capacities at governments' disposal influences the choice of policy instruments and eventually their implementation. And third, most research to date has focussed on state resources and capacities and has paid less attention to civil society capabilities and competences, such as those of NGOs, think tanks, pressure groups, lobbysts and others active in the policy process.

Multilevel governance dimensions of tool choice and programme creation
Related to these capacity issues, policymaking has very often a multi-level governance arrangement (MLG); MLG implies that different levels of government are likely to have some common but also different goals and instrument preferences and that reconciling them involves the use of the overt political calculus of intra-or intergovernmental bargaining and decision-making (Bolleyer and Borzel 2010;Kaiser et al. 2012). We know, for example, the MLG arrangements can have different formats through which goals and instruments are related. For example, in the EU MLG, there are some policy fields in which higher levels cannot establish the instruments to be used but can propose policy guidelines and goals. In other cases, MLG can be highly vertical wherein higher levels can impose the instruments to be adopted, however, this depends on the implementation styles of the lower levels with respect their effective functioning.
We know that the design of MLG arrangements is crucial but how the arrangement of MLG impacts tool choice and selection remains to be understood.

Procedural tools and how they relate to mixes
Policies have a substantive element that comprises of the technical arrangements of alternatives that can potentially resolve the policy problem at hand -and a procedural component that entails all the processes and activities necessary to coordinate the activities of policy actors in charge of formulating, making decisions and administering the alternatives (Howlett 2011).
That is, policy actors are arrayed in various kinds of policy communities, and just as they can alter or affect the actions of citizens in the productive realm, so too can they affect and alter aspects of policy-making behaviour (Knoke 1987;1991;1993).
Procedural implementation tools are an important part of government activities aimed at altering policy interaction within policy sub-systems (Klijn et al. 1995). An essential component of modern governance, the range of procedural policy instruments comprises at least half the toolbox from which governments select specific tools expected to resolve policy problems.
These two aspects of policy designing, however, have not received equal treatment from students of the subject. Procedurally oriented implementation tools have received much less attention than substantive ones, even though several procedural techniques, such as the use of specialized investigatory commissions and government reorganizations, are quite old and well-used and have been the objects of study in fields such as public administration, public management and organizational behaviour (Schneider and Sidney 2009).

Conclusion: A call for increased and improved analysis of policy tools and policy mixes
Policy instruments are a highly promising topic of research in public policy, not only for those interested in policy design but for those interesting in policy-making and policy processes as a whole.
There is room for substantial improvement in our knowledge of policies when focusing on policy instruments. There are still many lacunas and gaps that need to be filled and in this paper, we have focussed on those issues we consider crucial in moving policy instrument research forward.
As we have shown, there is much we already know, but much that remains to be studied, some of which needs more empirical research while other topics require more theoretical clarity and scholarly agreement. According to the situation we have presented, policy instruments research has much to offer for improving our understanding on how policies develop and could be better designed. Once this work has been done this approach to studying and understanding public policy-making will be placed on much firmer ground and generate many insights of use to both scholars and practitioners in the field.