The Mobilization of Scientific Evidence by Public Policy Analysts

Research on knowledge mobilization in policy making has been largely focused on identifying relevant factors having an effect on the uptake of evidence by actors and organizations. However, evidence on the magnitude of those effects remains limited and existing methods allowing for this have been scarcely used in this field. In this article, we first provide a rationale for greater investigation of substantive effect sizes, using methods such as mediation analysis and conditional probabilities. Using cross-sectional data from Québec (Canada) government policy analysts, we test an absorptive capacity model and describe direct, specific indirect, and total effects estimated from a path analysis. The results show that some factors have considerable effects, such as physical access and individual field of training, whereas some mediated relations are worth considering. Finally, we discuss some practical implications with regard to policy making and policy analysis but also the methodological standards of empirical research in this field.


Introduction
The field of research on knowledge mobilization is in full swing and has speedily developed over the past years. Although a plethora of studies have been published on various aspects of the phenomenon-mostly theoretical and empirical investigations of "facilitators and barriers" of knowledge mobilization-it is fair to say that the evidence accumulated remains limited in some respect. Some systematic reviews of literature (e.g., Innvaer, Vist, Trommald, & Oxman, 2002;Lavis et al., 2005;Oliver, Innvaer, Lorenc, Woodman, & Thomas, 2014;Orton, Lloyd-Williams, Taylor-Robinson, O'Flaherty, & Capewell, 2011) and some literature syntheses on the matter (e.g., Contandriopoulos, Lemire, Denis, & Tremblay, 2010;Mitton, Adair, McKenzie, Patten, & Perry, 2007) have been published, but it has been noted that this field of research nonetheless largely rests on actors' perceptions (Oliver et al., 2014, p. 13), their understanding of the policy and academic context, their organization's decision-making processes, and their judgment about the likely importance/impact of those factors: There are diverse methods for identifying potential barriers including qualitative approaches (individual interviews, focus groups), surveys and direct observation. However, there are no standard approaches available yet. As a result, those involved with knowledge translation activities need to use their judgment about how best to elicit barriers given their understanding of the context and potential barriers and resources available to them. (Grimshaw, Eccles, Lavis, Hill, & Squires, 2012, p. 5) If this type of method indeed allows researchers to identify potentially important factors (as is the case in studies reported in existing systematic reviews cited above), the fact remains that relatively few studies focus on empirically assessing their relative practical importance. As noted in Oliver et al.'s (2014) systematic review, "this paper has only counted the frequencies with which factors are mentioned without any weighting. Without more research, it is difficult to say what impact different factors might have" (p. 12).
Although it is recommended that "we should in general be more concerned with estimating the sizes or magnitudes of effects (i.e., effect sizes) than with the outcome of statistical tests" (Kline, 2011, p. 13), such recommendations are still rarely taken seriously and applied in social science empirical research. By focusing on conventional test statistics, p values, and regression coefficients, researchers convey only 604193S GOXXX10.1177 1 Durham University, UK part of the statistical information provided by their data and model. Assessing the magnitude of effects is here seen as a tool that allows discrimination between statistically significant but substantially unimportant effects (i.e., nil or almost nil) and substantially significant effects. In the context of research on knowledge mobilization, this distinction is crucial.
Furthermore, an argument is often made with regard to the apparent complexity of knowledge mobilization-related phenomena (see, for instance, Best & Holmes, 2010;Sanderson, 2009). As suggested by Black (2001), "The implicit assumption of a linear relation between research evidence and policy needs to be replaced with a more interactive model" (p. 275). Willing to avoid simplistic conceptualizations, models are put forth with the intention to reflect the complex conceptual dimensions of the phenomenon, but, in return, appropriate methods to test these models are lagging. This is manifest in the capacity to model mediated relations between explanatory factors of knowledge mobilization.
One potentially relevant tool for this purpose is structural equation modeling (SEM), which allows modeling complex relations by including indirect effect and mediating variables. In other words, it allows the investigation of possible mechanisms operating in a statistical model. Such methods, long known in the social sciences, have been scarcely used in research on knowledge mobilization (see notably, Cummings, Estabrooks, Midodzi, Wallin, & Hayduk, 2007;Wellstead, Stedman, & Howlett, 2011), and where they have been used, issues of interpretation sometimes arise. The problem here is not that the data presented are not instructive or valid, but rather that the communication strategy does not cash out the full implications of the identified relations nor does it allow to qualify the intensity of the relation in a suitable fashion. This is an issue for most results reported from quantitative studies and is perhaps even more salient when performing SEM as the complexity is increased. Strategies need to be devised to provide informative and substantial interpretation of regression coefficients, on an intelligible scale that would be intuitively understood and would reflect some uncertainty by reporting a confidence interval. We here argue that statistical tools can help to simplify the interpretation of regression results and can be put to use in complex models such as SEM. This can be seen as a gain for applied researchers using similar methods, but perhaps even more for non-academic research users interested in consulting and using research results.
Therefore, the main research objectives guiding this article are the following: (a) to identify the correlates of research evidence consultation by policy analysts and the mediated relations between those factors, and (b) to describe, using statistical simulation tools, the magnitude of those effects on the main outcomes. Using cross-sectional data collected among Québec (Canada) government policy analysts, we test a complex model derived from the absorptive capacity model using path analysis and describe the predicted probabilities of explanatory factors of interest.

Knowledge Mobilization Theories and Assumptions
Knowledge utilization has been conceptualized in a number of ways-most notably as various dimensions (e.g., Weiss, 1979) or various stages (e.g., Knott & Wildavsky, 1980;Rich, 1997;Rich & Oh, 2000), then operationalized into scales and indexes (e.g., Cherney, Povey, Head, Boreham, & Ferguson, 2012;Landry, Amara, & Lamari, 2001). However, although these strategies have had some success, issues of precision remain. Following Estabrooks et al. (2011), we argue that the conceptual confusion commonly found in the field can be, at least in part, resulting from the relative absence of reliable measures and indicators of the phenomenon of interest. That is why we will privilege here a more restrictive understanding of the phenomenon, thus lending itself to unambiguous and empirically relevant measurements.
We here focus specifically on the consultation of scientific evidence by policy analysts, as potential users, while recognizing that diffusion and impact of research are logically distinct phenomena (Rich & Oh, 2000). In a similar vein, potential users of research evidence can be identified and defined rather broadly, 1 and we are here focusing on civil servants categorized as policy analysts located in government departments.
Policy analysis, defined as the analytical and advicegiving tasks performed by policy analysts 2 to inform the public policy-making process (Colebatch, 2006;Lomas & Brown, 2009;Mayer, van Daalen, & Bots, 2004), has been recently highlighted for its relevance to evidence-informed policy making (Howlett & Newman, 2010), mainly for three reasons. First, given the professional tasks of these individuals, they can be considered as potential entry points for research evidence and possible channels of communication (Cohn, 2007;Howlett, 2009a;Nutley, Walter, & Davies, 2007;Ouimet et al., 2010;Sundell, Wärngård, & Head, 2013). To a certain extent, these professionals can be considered to act as informal knowledge brokers in that they absorb, interpret, and communicate external knowledge (Howlett, 2011). Second, it has been noted that to this day, relatively few studies focus on policy analysts compared with other public administration positions (such as managers or directors-e.g., Howlett & Walker, 2012;Jennings & Hall, 2011): "empirical data on just about every aspect of policy analysis in government are lacking" (Howlett, 2009a, p. 2). And finally, the quality and nature of their professional tasks are intrinsically related to the overall policy analytical capacity of the public administration, the latter being instrumental for evidence-informed policy making to succeed (Howlett, 2009b).
Knowledge mobilization, as an umbrella concept, is seen here as encompassing multiple related phenomena (such as knowledge transfer, knowledge translation, knowledge use, etc.). 3 As such, it can also be decomposed along specific dimensions (Green, Ottoson, Garcia, & Hiatt, 2009), namely, the source of research evidence, the content, medium of communication, user's attributes, and context. What specific theories of knowledge utilization do is depict relations between these dimensions, while highlighting specific factors of interest. 4 In this respect, three main theoretical approaches, or families of theories, exist (Landry, Lamari, & Amara, 2003;Rich & Oh, 2000): (a) theories dealing with actors' rationality (and its limitations), (b) theories focusing on organizational interests, and (c) theories highlighting communication and social relations among actors. Although they can be considered as alternative accounts, they can in return be considered complementary theories that can, and to some extent need to be, synthesized.
Theories of rational choice, and perhaps even more in the case of limited rationality theories (Bendor, 2010;Rich & Oh, 2000), focus partly on the individual's cognitive capacity when processing information. For instance, the ability to cognitively access and interpret scientific results (provided in part by one's academic training) matters greatly with respect to knowledge mobilization. Also, as limited rationality theories tell us, individuals are bound by structural and organizational factors worth taking into consideration when attempting to provide a specific behavioral account. Therefore, the organizational resources (or lack thereof-physical, technological, human, etc.) in a given policy context are thought to facilitate (or impede) knowledge mobilization (Ellen, Lavis, Ouimet, Grimshaw, & Bédard, 2011;Léon, Ouimet, Lavis, Grimshaw, & Gagnon, 2013;Oliver et al., 2014;Orton et al., 2011). And finally, it is assumed that greater relations with actors within and/or outside the policy context (e.g., academics) can facilitate and stimulate knowledge exchange and mobilization (Innvaer et al., 2002;Lavis et al., 2005;Oliver et al., 2014;Orton et al., 2011). As summarized in Oliver et al.'s systematic review (2014), Organisational factors, including availability and access to research were considered to be important influences on whether evidence was used in policy, and the quality of the relationship and collaboration between researchers and policymakers to be the single most mentioned facilitator. (p. 11) We argue that these kinds of factors need to be assessed more thoroughly in their relation to knowledge mobilization as we have little indication of their empirical importance or of possible mediated relations between them.

Absorptive Capacity Model and Hypotheses
The absorptive capacity model summarizes most of the theoretical components surveyed above. Although this framework 5 was initially developed in the field of management to model features of private entities (i.e., firms), insights from this theoretical perspective can be extended to public organizations (Ouimet, Landry, Ziam, & Bédard, 2009). In this context, the individual absorptive capacity mainly refers to four components that are thought to operate sequentially: the ability to recognize the value of the knowledge to acquire, the acquisition of external knowledge, the utilization of knowledge, and, finally, its application. In other words, if we only focus on the first two components, being able to recognize the value of research and thus have a positive view of its relevance would influence future consultation.
The absorptive capacity as a whole is itself thought to be function of various variables such as prior knowledge, social interactions, and so on. Beyond the individual level, the absorptive capacity is considered to depend on organizational features such as the research knowledge infrastructures available that would allow one to actually obtain research evidence from outside sources such as academic journals. As one can start to appreciate, the absorptive capacity model does not limit itself in describing direct relations between its components and takes on board the assumption that some variables (such as the ability to recognize the value of external knowledge and social interactions) can mediate the relations. Put differently, one could assume under the absorptive capacity model that the likelihood of an individual engaging in a relation or communication with an academic might be mediated by the very fact that this prospective knowledge user has an academic background allowing him to see the policy relevance and usefulness in requesting scientific information from an academic. What we intend by drawing on this model is to reexamine factors that have been highlighted as important in the literature and to assess possible mediated relations using quantitative techniques to estimate the magnitude of effects in that context. From this model, specific hypotheses can be derived and will be put to the test using the data and methods described below. These hypotheses are formulated for both expected direct effects (i.e., the shortest relation between two variables) and indirect effects (i.e., relations between variables that follow a path through a third variable).
Among the factors affecting the components of the absorptive capacity of the individual is prior knowledge. With some filiation with limited rationality theories, we can expect that research utilization will depend on individual cognitive factors such as academic training. Consequently, Hypothesis 1: The level of academic training, as well as the disciplinary field of training, has a direct effect on the probability of using research.
Because it is assumed in the model that prior knowledge has an effect on both social interactions and the recognition of the potential value of research, it is possible to derive two hypotheses concerning indirect effects. In both cases, some form of familiarity with academic research principles and practices can act on the level of trust one puts into academics themselves and the knowledge they produce. Therefore, it is plausible that a training background in research can lead to greater interactions with academics and an overall positive attitude toward academic research: Hypothesis 1.1: The level of academic training, as well as the disciplinary field of training, has an indirect effect on the probability of using research, through an effect on the probability of interactions with academic researchers. Hypothesis 1.2: The level of academic training, as well as the disciplinary field of training, has an indirect effect on the probability of using research, through an effect on the probability of recognizing the potential value of research.
Here, we can derive a third hypothesis by combining the previous two hypotheses:

Hypothesis 1.3:
The level of academic training, as well as the disciplinary field of training, has an indirect effect on the probability of using research, through a successive effect on the probability of interactions with academic researchers and of recognizing the potential value of research.
In this model, the attitude an individual displays toward research (i.e., its perceived relevance, quality, and usefulness) is likely to act as a predictive factor of utilization. As described in the hypotheses above, one's attitude toward research is bound to be shaped by prior knowledge and experience and can in turn influence interactions with researchers and research utilization. Beyond this, it is plausible that one's attitude also has a direct effect on research utilization. We can therefore formulate the following hypothesis: Hypothesis 2: Recognizing the potential value of research has a direct effect on the probability of using research.
Third, the presence (or absence) of organizational resources allowing some form of access to external knowledge is a factor that is generally taken for granted 6 and therefore is rarely put to empirical testing. It nonetheless appears fundamental in facilitating research utilization. What is more, the level of physical access to external knowledge is likely to vary across policy sectors, as demonstrated by Léon et al. (2013) where substantial variations in access to key scientific journals were found among Canadian Health Ministries. We can thus formulate the following theoretical expectations regarding access infrastructures: Hypothesis 3: Access to infrastructures permitting research acquisition has a direct effect on the probability of using research.
Although it may appear rather straightforward that physical access to research would be considered a precondition to actual use, physical access to research can also have potential indirect effects, as it is plausible that accessing research might be a means of researcher identification. Through database searches and the retrieval of published literature, one can identify experts in a given field with whom it may appear relevant to engage with. Therefore, Hypothesis 3.1: Access to infrastructures permitting research acquisition has an indirect effect on the probability of interactions with academic researchers. Hypothesis 3.2: Access to infrastructures permitting research acquisition has an indirect effect on the probability of using research, through an effect on the probability of recognizing the potential value of research.
Then again, we can derive a third indirect hypothesis by the combination of the previous two hypotheses: Hypothesis 3.3: Access to infrastructures permitting research acquisition has an indirect effect on the probability of using research, through a successive effect on the probability of interactions with academic researchers and recognizing the potential value of research.
While capturing in part the variations in knowledge infrastructures across the public administration, as well as hiring and training policies, we can argue that levels of consultation of research evidence might vary across specific policy sectors. And although it was not explicitly described as part of the original absorptive capacity framework, it nonetheless appears that the professional context of an individual is an important factor. Therefore, Hypothesis 4: Working in a given policy sector (such as health and social services), compared with other sectors, has a direct effect on the probability of using research.
Finally, as signaled by Todorova and Durisin (2007), social interactions occupy a central role in the absorptive capacity framework: "Social integration mechanisms, which build connectedness and shared meanings, influence all processes of knowledge absorption" (p. 781, emphasis added). Therefore, we can expect to find the following relation: Hypothesis 5: Interacting with academic researchers has a direct effect on the probability of using research.
As it was signaled that this factor could affect every component of the model, notably the recognition of the potential value of research, it can be expected that interactions will act as a mediating factor. Although we argued above that prior knowledge can affect the perceived relevance of academic research, it is reasonable to expect that engaging with academics can also contribute to one's displaying of a positive attitude toward academic research. Therefore, Hypothesis 5.1: Interacting with academic researchers has an indirect effect on the probability of using research, through an effect on the probability of recognizing the potential value of research.
Combining these hypotheses gives the specific model that has been tested in this article. In the next section, we describe the data and analytical strategy used.

Data and Variables
The data used in this article comes from a telephone survey conducted among Québec (Canada) public administration professional employees. 7 More specifically, the survey targeted 14 professional groups of civil servants without formal decision-making powers but who nonetheless performed analytical and advice-giving functions. A stratified random sample selection procedure was adopted, and at the end of the data collection phase, 1,617 individuals located in 18 of the 22 ministries had completed the survey, with a net response rate of 62.48%. 8 The questions asked covered a 12 months period (as opposed to longer reference periods in other surveys, such as 5 years) so as to assure precision. Given the sampling procedure used and the large number of observations, the data can be considered representative of the sampled population of analysts in Québec public administration.
The model described below is built from three outcome (endogenous) variables taken from the survey: The main outcome variable, research utilization, was measured by asking respondents whether they had consulted academic research over the past 12 months within their work context (yes or no), interactions with academic researchers was measured by asking respondents whether they had contacted at least one academic researcher over the last year (yes/no), and attitude toward research was measured by asking respondents to identify and rank which information/knowledge-producing groups (i.e., public civil servants, journalists, university professors/researchers, private-sector consultants, nonuniversity/think tank researchers) produced the most useful information/knowledge, based on the professional experience of the past 12 months.
The main explanatory factors considered here are prior knowledge, captured by a first variable describing the type of training, measured by asking the respondent to identify the type of degree they earned, and by a second variable describing the disciplinary field of training, measured by asking the respondents to identify in which field(s) their training took place; and physical access to research, captured by a variable describing the respondent's reported access to electronic bibliographic databases from their workstation. As controls, the following variables were included: gender, age, and administrative domain

Analytical Strategy
The analytical strategy applied in this article is inferential and essentially aims at identifying the correlates of knowledge utilization by professional civil servants and at describing the complex mediation relations between those factors and the magnitude of these relations.
Path analysis (as part of the SEM family) was used. Recall that, contrary to conventional regression (linear or not), SEMs allow us to test models accounting for greater complexity by the identification of direct and specific indirect relations. SEMs are mainly promoted due to their ability to model relation between latent variables and multiple indicators, through factorial analysis. Although it may seem preferable, a priori, to build theoretical constructs from multiple latent variables, it appears that it is not a necessary condition to perform valid analyses. 9 For instance, Kline (2011) suggests that solely using observed variables is admissible: "There are also times when there is just a single observed measure of each construct, and [path analysis] is a singleindicator technique" (p. 103). In other words, path analysis in such a context would be based only on manifest, directly observed variables (MacKinnon, 2008). In this respect, the quality of the analysis is intimately related to the quality of the variables mobilized, but depends most of all on the quality of the model: "one indicator is sufficient if it is properly causally specified" (Hayduk et al., 2007, p. 284; see also Hayduk & Littvay, 2012). In this case, as in many others, parsimony should be privileged.

Path Model
The path model, derived from relevant theories and hypotheses described above, can be summed up by the following three equations: where, y n corresponds to endogenous variables, each with its own sub-equation, β n n y represents the estimated parameters between two endogenous variables, γ n n x describes parameters estimated for exogenous variables, and ζ n is an error term included in each equation. This model is a saturated and recursive one. 10 It basically means that every possible relation in the model is estimated, and as a consequence, it is a model with zero degrees of freedom.
The justification for specifying a saturated path model comes from the fact that, contrary to what one might intuitively think, postulating a model with fewer relations is equal to imposing supplementary assumptions about the theoretical model. As Reichardt puts it (2002), Overidentifying restrictions are never correct. The fewer incorrect restrictions that are imposed, the less biased the estimates of the path coefficients are likely to be, ceteris paribus. Because they have no overidentifying restrictions, just-identified models are to be preferred to overidentified models to the extent that minimizing bias is a priority. (p. 313) In our case, we lack any plausible justification to withdraw a theoretical relation in the model. This is a contentious argument as most SEM researchers tend to privilege overidentified models because these allow overall model evaluation through exact fit tests and various adjustment indices. 11 As suggested by Bowen and Guo (2012) This disjunction between model fit and individual relations is due to the nature of the tests performed (i.e., chisquare and fit indices), which, more or less, only reflects the disparity between the data covariance matrix (Sigma) and the theory-implied matrix (Sigma ( ) Θ ). In other words, a proper fit is not a sufficient condition to corroborate the verisimilitude of a model (Hayduk & Pazderka-Robinson, 2007;Kline, 2011;Tomarken & Waller, 2003;Waller & Meehl, 2002). This is where estimating the predicted probabilities and the marginal effects of variables of interest come in handy.

Predicted Probabilities
The estimation of the model described above generates results of interest (direct, indirect, and total effectsstandardized or not), but our main focus lies in the estimation of the magnitude of those effects on an intelligible scale. To this end, we use statistical simulation to convert the estimated parameter of effects into probabilities and express them along with a confidence interval. The estimation of those predicted probabilities follows a counterfactual perspective in that we compare an observed state with a hypothetical one-in a fashion akin to experimental epistemology (Reiss, 2011). This perspective is particularly well-advised when using dichotomous variables (as in our case), as suggested by Valeri and VanderWeele (2013): "The counterfactual approach . . . helps in understanding mediation with binary outcomes and binary mediators" (p. 6).
More specifically, the estimation proceeds by comparing the predicted probabilities of the outcome variable (y) when a given variable (x) shifts in value (i.e., typically, from 0 to 1, for a binary variable) while holding other variables constant. In this case, the exogenous variables remaining constant are left to their observed values (rather than to their mean or any other value-mean, mode, etc.). Therefore, the values used to estimate the effect are the same values each individual takes on each variable. The strategy involves three steps. First, the model described above was estimated 12 using Mplus 6 (Muthén & Muthén, 1998-2010 and the outputs (mainly, the parameters and the variance-covariance matrix) were then adapted to be read on Stata 12 (StataCorp, 2011). 13 Second, a normal distribution (n = 1,000) is simulated for each parameter of the model (i.e., the parameter itself becomes the mean of that new distribution) so as to allow for a confidence interval. Finally, the probit equations for each model are solved: The outcome of the equations where the variable of interest is at its maximum (1) is subtracted to the outcome of the equations where the variable of interest is at its minimum (0). This is computed by simulating sequentially, through a loop and for each individual in the data set ( ). x i n − A normal distribution of parameters is obtained for individual i simulating that every individual has the same values on every variable, and then an average effect is obtained for this specific individual by estimating the subtraction described above. The same goes for x 2 to x n , and in the end, the reported effect is an average of those averaged effects, now described within a lower and upper confidence interval (95%). In all these stages, the exogenous variables remain constant at their observed values, and the results are expressed following a conversion into probabilities. 14 Therefore, the estimated outcome can be interpreted as the effect, in percentage points, of the explanatory factor of interest on the probability of occurrence of the outcome, bound within a confidence interval. This strategy was repeated for every exogenous variable of interest (i.e., every factor found to be statistically significant in the model).

Descriptive Statistics
First of all, it was found that 78.45% of the respondents answered positively when asked whether they had consulted at least one scientific article over the past 12 months. As was expected, there are some variations in answers when compared across policy sectors. For instance, the Ministry of Sustainable Development, Environment, and Parks, the Ministry of Agriculture, Fisheries, and Foods, and the Ministry of Health and Social Services are the administrative domains where consultation rates were the highest with, respectively, 91.43%, 87.22%, and 87% of respondents claiming that they have consulted research evidence over the past 12 months. Comparatively, the Ministry of Justice, the Ministry of Family and Seniors, and Horizontal ministries (i.e., Executive Council and Treasury Board) were the administrative domains where the rates were the lowest: Respondents claimed to have consulted research evidence with proportions of 56.1%, 58.06%, and 63.01%, respectively.
When looking at the other endogenous variables of our model, we found that 53.93% of respondents had not contacted an academic researcher over the past 12 months and most of those who did contacted researchers in (exclusively) natural sciences and engineering (17.5%). With regard to the attitude toward research (i.e., recognizing its potential value), it was found that 53.91% ranked university professors or researchers at the top of the list (first or second) when comparing five information and knowledge producers (i.e., public civil servants, nonacademic think tanks, journalists, private-sector consultants).

Path Model
The estimated paths 15 are represented in Figure 1, as we see the relations of interest between the exogenous variables and the three endogenous variables considered that were described theoretically in the structural equations above. Recall that this is a saturated model, and as such, it does not show any degree of freedom, which means that the global model fit cannot be assessed. Nonetheless, and in line with our research objective, it remains instructive to scrutinize the individual relations of the model and their magnitude.
A certain number of variables significantly affect the principal endogenous variable-research utilization-in a direct fashion. More specifically, seven of them describe either a positive (+) or a negative (−) relation: Interactions with academic researchers (+), attitude toward academic research (+), holding a professional master and/or doctoral degree (+), holding a research master and/or doctoral degree (+), academic training in social sciences (−), academic training in administrative sciences (−), and physical access to bibliographic databases (+). In the same way, three variables directly affect the interactions with academic researchers variable, that is, holding a research master and/or doctoral degree (+), academic training in administrative sciences (−), and physical access to bibliographic databases (+). Also, we denote three variables, which were significantly associated in a direct fashion with the attitude toward academic research variable: Interactions with academic researchers (+), holding a professional master and/or doctoral degree (+), and academic training in administrative sciences (−). 16 In this model, some specific indirect relations (i.e., relations mediated by one or multiple factors) appear as being significant when in relation with research use, through a path via the interaction with academic researchers variable. There are three of those: Holding a research master and/or doctoral degree (+), academic training in administrative sciences (−), and physical access to bibliographic databases (+). Also, it was found that two indirect relations allow some influence on research consultation, via the attitude toward research variable. These two variables are the following: Holding a professional master and/or doctoral degree (+) and physical access to bibliographic databases (+). Finally, and almost by way of deduction (or perhaps extension), we note that some indirect relations are significant when influencing research consultation through paths describing successive influence on the interaction with academic researchers and attitude toward academic research variables. The three variables forming these chains of influence are the following: Holding a research master and/or doctoral degree (+), academic training in administrative sciences (−), and physical access to bibliographic databases (+).

Predicted Probabilities
Applying the method described earlier on, the effects of variables that have been found statistically significant have been estimated. The results of those estimations are presented in Table 1.
First of all, it can be seen that specific relations can be identified as being significant or insignificant by looking at the scope of the confidence interval: An effect covering 0 is potentially nil and is likely to be insignificant. Relations that have been found insignificant earlier describe a potentially nil effect, but nonetheless, some relations describe a potentially nil effect while they had been found statistically significant in the basic model. That is the case for the specific indirect relations sequentially going through both mediators (i.e., interactions and attitude) that starts from both postgraduate training in research and physical access to bibliographic databases.
As described in Table 1, the single most important effect found in this study, interpretable in percentage points, 17 is attributable to training in administrative sciences (−27.6%), Note. The shaded lines represent the relations that are detected as statistically non-significant (at the 0.05 threshold) when estimating the model. They are nonetheless included in the computation as they theoretically contribute to the total effect. CI = confidence interval.
when compared with training in social sciences. However, the intervals are rather wide in this case, thus limiting the precision of the estimate. Let us note that this variable also has a sizable negative effect on research utilization (−4.3%) through an effect on the probability of interactions with academic researchers. The variable describing the physical access to research also has a considerable effect on research utilization as its total effect is of 18.9% (which corresponds to a direct effect of 15.2% and to an indirect specific effect of 2.5% mediated by the interactions with academic researchers variable). With regard to the postgraduate training in research variable, when compared with bachelor's degree, we note an increase of 11.7% on the probability of consulting research. This total effect can be decomposed into a direct effect of 7.3% and a specific indirect effect of 2.6% mediated by the interactions with academic researchers variable. Finally, the postgraduate professional training variable, when compared with a bachelor's degree, describes the least important effect as we note a total effect of 10.6% on the probability of using research. Put differently, this variable generates a direct effect of 7.2% and has a specific indirect effect of 1.3% on utilization, mediated by the attitude toward research variable. 18 In return, notice that the interval bounds comprise values close to 0. We can see through these analyses that the specific indirect effects remain somewhat weak (perhaps with the exception of the training in administrative sciences variable, mediated through the interactions variable). Even in the case of statistically significant relation, the confidence interval bounds tend to include values leaning toward 0, thus increasing the likelihood of a potential nil effect. As a corollary, we note that none of the variables generated specific indirect effect that were all significant: In all cases, only one out of three relations appear significant (as one can tell either by the p value in the model or the confidence intervals describing the distribution of the postestimations).

Theoretical and Policy Implications
Based on the results just described, we can affirm that our hypotheses regarding direct relations have been largely corroborated, with some nuances with regard to the indirect relations expected. First of all, it was found that academic training at the postgraduate level (either professional or research-oriented), when compared with an undergraduate level's training, is a significant correlate of research utilization. In contrast, there is a difference between a postgraduate training in research (i.e., research-oriented master thesis and PhD), associated with a direct and positive relation to the probability of interactions with researchers, and a professional postgraduate training (i.e., master with essays and internships), rather associated with a positive attitude toward research. With regard to the field of training, administrative sciences (compared with health sciences) were the only field where a significant (and negative) relation was found.
These sets of hypotheses thus corroborate the existing assumptions according to which prior experience in research can act as a predisposition to research utilization (Hanney, Gonzalez-Block, Buxton, & Kogan, 2003;Jewell & Bero, 2008;Landry et al., 2003;Lavis et al., 2005;Orton et al., 2011). In this respect, our conclusions tend to support views expressed elsewhere: Research skills are . . . crucial in generating, assessing and analyzing evidence. . . . Such skills can, for instance, help to expose the meaning and limitations of evidence from both formal research and other sources. In addition, a detailed understanding of research methodology is needed for effective relationships with external research contractors, in relation to complex data analysis and other issues. (Nocon, 2009, p. 96) This last remark suggests that the interpretation of results and their import into a given policy context are rarely straightforward (Greener & Greve, 2013;Moat, Lavis, & Abelson, 2013). In a similar fashion, our results support the idea that policy analysis, to be properly conducted, requires a set of specific skills (Howlett & Lindquist, 2004;Kothari, MacLean, & Edwards, 2009;Oliver et al., 2014). What is manifest here is that research abilities are crucial for policy actors to fully adopt and implement the principles of evidence-informed policy making (Bédard & Ouimet, 2012;Howlett, 2009b). Without such skills, multiple consequences are foreseeable, such as the inability to recognize and interpret limitations and potential biases and/or erroneous application of research results (Deeming, 2013;Jewell & Bero, 2008).
Our second hypothesis was also corroborated as it was found that displaying a positive attitude toward the potential value of research was identified as increasing the probability of research utilization. Nonetheless, this variable, when considered a mediator, contributed rather weakly to an explanation of research utilization, as most indirect relations mediated by this variable were found insignificant. Therefore, our results, without completely falsifying this hypothesis, suggest a nuanced interpretation of this otherwise importantly regarded factor in the literature (see notably, Landry et al., 2001;Lavis et al., 2005;Mitton et al., 2007;Orton et al., 2011).
Similar to the relations pertaining to the field of training variable, the estimated relations from the physical access variable corroborate our hypotheses regarding the direct relation and the indirect relations, with the exception of the relation mediated by the attitudinal variable. This organizational factor, which used to be somewhat ignored in previous systematic reviews (see, for instance, Innvaer et al., 2002;Lavis et al., 2005), is considered to be a crucial one in explaining research utilization (Oliver et al., 2014). Our results corroborate this idea, as it was the most important factor in terms of magnitude in our study.
The fourth hypothesis concerned the expected variations in research utilization across policy sectors (Davies, Nutley, & Smith, 2000;Howlett & Lindquist, 2004) and was also corroborated. Without detailing all the estimated comparisons, let us note that when compared with the Ministry of Justice, some ministries (for instance, Health and Social Services, Municipal Affairs, Regions and Land Occupancy, Agriculture,Fisheries,and Food,etc. 19 ) had positive effect on research utilization, whereas others (for instance, the Immigration and Cultural Communities, etc. 20 ) had a positive effect on interactions with researchers. Finally, some ministries (such as Transportation 21 ) instead had a negative effect on displaying a positive attitude toward research. Overall, sectors were a considerable source of variation, often having a rather sizable effect in percentage points.
Finally, the fifth set of hypotheses, which described the direct relation between the interaction variable and the utilization of research, as well as the indirect relation between them mediated by the attitude variable, was found to be significant. Recall that the argument underlying the interaction variable suggests a certain number of virtues to this explanatory factor: Interaction can also promote receptivity of the research by policy makers through early and repeated engagement of them with the research findings. Interactions might help research users to better understand research findings and facilitate information exchange between the two groups about professional norms and expectations. (Kothari, Birch, & Charles, 2005, p. 119) In this respect, our results suggest that interactions perform these tasks, in part through direct effects, but also in part through indirect effect-the latter not being recognized formally in the extant literature. One of the distinctive features of our results is that while the interaction variable was found to be significant, it was also found to be instrumental in obtaining statistically significant indirect effects. Interactions with academic researchers are generally presented as a substantive explanatory factor-although its proper empirical impact is seldom demonstrated (see notably, Almeida & Báscolo, 2006;Contandriopoulos et al., 2010;Green et al., 2009;Innvaer et al., 2002;Lavis et al., 2005;Mitton et al., 2007)-and it is implied that a direct effect is at work. In contrast, our analyses allow us to support this claim, but also demonstrate that its effect is in fact greater than expected once we take into account its potential indirect effects.
At this stage, we can formulate four general observations with regard to the theoretical implications of our results. First of all, the absorptive capacity framework was largely corroborated. Although the global model has not been tested per se, its corroboration is evidenced by the examination of its lower-order components (i.e., the specific relations constituting the model- Bowen & Guo, 2012). What we have shown is consistent with the main approach focusing on interactions but also that more complex relations are at work between these factors.
Second, our analyses show that beyond individual relations, an explanation of research mobilization appears to be built on a conjunction of factors. In this respect, it would have been possible to estimate the probability of utilization as a function of a combination of factors, rather than isolating a single one each time while maintaining the other factors constant. These possible combinations-being quite numerous, and partly arbitrary-have been ignored for parsimony's sake. Nonetheless, we have noted that cumulated specific indirect effects represent a substantial part of the effect emanating from a given factor. It could be expected to observe similar patterns in different contexts.
This combination of factors could be heuristically represented using Mackie's (1965) description of INUS conditions as being "an insufficient but necessary part of a condition which is itself unnecessary but sufficient for the result" (p. 245). In this respect, it would be interesting that actors involved in the implementation of interventions aiming at increasing the mobilization of scientific knowledge in the policy process consider similar combination of factors. As it was shown here, certain variables can be considered as being anterior to a given phenomenon. For instance, implementing an intervention aiming at increasing the level of interaction with researchers is likely to be limited in a context where cognitive factors are not taken into account. In other words, some relations appear to have sequential features.
Third, the mediation analysis performed in this study demonstrates, as could have been expected, that a single factor can have multiple effects. This observation is partly related to our first one regarding the complexity at work between the explanatory factors. There is a need here for a greater recognition of this in the theoretical literature but perhaps more importantly in the analytical and empirical efforts devoted to reflect this complexity. To the extent that "contributions to the effect can come from different sources and via different pathways" (Cartwright & Hardie, 2012, p. 24, emphasis added), it is important to take these into account when setting interventions to increase research mobilization. Recall that it would be possible to dismiss some possible explanatory factors as having no direct effect while some sizable indirect might be at work. In other words, our understanding of the extant literature lead us to observe a gap between theories, the research objectives, and the methodological apparatus deployed accordingly.
Finally, although our results can be confidently generalized to our study population, our research design does not warrant further generalizations. Policy analysts may tend to share similar attributes and tasks, they nonetheless operate in highly varied administrative contexts (e.g., resources, networks, decision-making processes, etc.), thus limiting comparability. Furthermore, little data are available to systematically compare those policy analysts across contexts (for related surveys, see, for instance, Howlett, 2009a;Wellstead & Stedman, 2010;Wellstead, Stedman, & Lindquist, 2009).

Implications for Researchers
Beyond the theoretical and empirical aspect of our work, three methodological features need to be discussed. First of all, our analytical strategy is in line with further attention to more complex models integrating mediated relations allowing the exploration of causal chains and causal mechanisms. Methodological refinement, when applied to substantial research questions, can be highly valuable, especially in a field that can hardly be said to have reached maturity. Moreover, beyond the theoretical motivation to pursue these, it has been argued that these can in fact be relevant from a policy point of view (Pearl, 2001), as policy interventions can plausibly have indirect effects and take place in a set of complex relations. Perhaps more difficult to interpret, complex models nonetheless appear to be more realistic when used to described social and policy contexts.
Second, remaining committed to simple regression analysis, in our case, would have underestimated the empirical importance of some factors as demonstrated above. Although most indirect effects remained relatively modest, they nonetheless appeared rather sizable when considered conjointly (for instance, when considering the difference between the total and the direct effect 22 ). We argue, after many others (Ferguson, 2009;Fritz, Scherndl, & Kühberger, 2013;Hanmer & Kalkan, 2013;Herron, 1999;King, Tomz, & Wittenberg, 2000;Preacher & Kelley, 2011;Wood, 2013;etc.), that there is a real need to go beyond basic statistics and to report information more intelligible and substantial metrics than mere coefficients, p values, and statistical thresholds. When relevant to do so, the benefit from this methodological shift is (at least) twofold: Significant and substantive effects (as opposed to significant but unsubstantive ones) can more easily be detected, and research results can be interpreted and communicated in a more intelligible way to both researchers and policy makers. As the field of research on mobilization has been concerned with formats of presentation of results (Lavis et al., 2005;Lavis, Permanand, Oxman, Lewin, & Fretheim, 2009), here is one aspect that should require greater attention and interest.
Third, although not discussed at length in the present article, the analytical strategy to compute quantities of interest needs to be carefully chosen. As suggested by Hanmer and Kalkan (2013), the outcome of the postestimation is likely to be sensitive to the computational strategy. In our case, when isolating an effect of interest, the remaining variables were set to be constant at the observed values, although they could have been set to any other value. The rationale for doing for was to remain as close as possible to the actual values of the data set to ensure greater realism. However, it is frequent to find in the empirical research literature that these values have been set to be constant to their mean, their mode, an average case or else, and that it is not necessarily relevant to do so (for instance, setting a dichotomous variable to its mean). It has been argued elsewhere that one should be careful when selecting imputed values in the computation of predicted probabilities so as to avoid extreme or unrealistic counterfactual scenarios can bias the estimation (King & Zeng, 2006) and we hereby reiterate these warnings. More generally, the idea here is that results tend to be sensitive to the methodological and analytical choices one makes, and in this respect, greater rigor and transparency would be most welcome.

Author's Note
This research was completed while holding a postdoctoral fellowship from the Social Sciences and Humanities Research Council of Canada (SSHRC).

Acknowledgments
This article has gone through multiple iterations, and the author therefore thanks Mathieu Ouimet and Julian Reiss for useful thoughts and suggestions in the process.

Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding
The author(s) received no financial support for the research and/or authorship of this article. "An individual who is likely to be able to use the knowledge generated through research to make informed decisions about health policies, programs and/or practices. A knowledge user's level of engagement in the research process may vary in intensity and complexity depending on the nature of the research and his/her information needs. A knowledge-user can be, but is not limited to, a practitioner, policy-maker, educator, decision-maker, health care administrator, community leader, or an individual in a health charity, patient group, private sector organization, or media outlet." 2. We follow here a definition of policy analysts, provided by Durning and Osuna (1994): "The people who transform data, information, and other analytic inputs into policy advice (or enlightening information) relevant to a particular decision for a specific client" (p. 630). 3. Although these terms have slightly different meaning, they are all nonetheless related in their focus on various aspects of the general phenomenon of knowledge mobilization. For further conceptual clarifications, see, for instance, Thompson, Estabrooks, and Degner (2006). 4. It is also what is reported, although in a rather atheoretical fashion, by systematic reviews of literature on knowledge mobilization, where factors affecting knowledge mobilization ("barriers and facilitators") most commonly cited are summarized. For the purposes of this article, we make the assumption that these factors can be considered in conjunction and can be linked to existing theories. 5. For the main descriptions of this theoretical framework, see articles from Noblet, Simon, and Parent (2011); Todorova and Durisin (2007); Zahra and George (2002); and Cohen and Levinthal (1990). 6. As suggested above, organizational infrastructure has only recently made its way in systematic reviews of factors affecting research mobilization and use. See, for instance, Oliver, Innvaer, Lorenc, Woodman, and Thomas (2014), compared with previous systematic reviews. 7. The data collection took place between September 6 and November 25, 2008, and was handled by a contracted survey firm (Infra International Inc.), using computer-assisted telephone interviewing (CATI) technology. For more detail on the full survey approach, see Ouimet et al. (2010). 8. Four ministries have been excluded beforehand (i.e., the Ministry of Tourism, the Ministry of International Relations, the Ministry of Revenue, and the Ministry of Governmental Services) as their focus is either operational or in a sector less amendable to research mobilization. Given their nature, these ministries were less suited for our research needs. A total of 1,614 observations were retrieved for analysis as the Ministry of Work (n = 3) was excluded post hoc due to a low number of observations. 9. MacCallum and Austin (2000), in their review of existing structural equation modeling practices, suggest that 25% of their sample of psychology articles did not use latent variables. 10. A recursive model is defined by the unidirectionality of the presumed causal relations. In contrast, a non-recursive model would include feedback loops and/or bidirectional relations. See notably, Kline (2011, pp. 106-ff). 11. It is understood that in a saturated model, the "the implied matrix will be the same as the input matrix (i.e. perfect fit will be obtained)" (Bowen & Guo, 2012, p. 136). Thus, indicators of overall fit in this context are trivial. To the extent that the theoretical model is plausible, one should rather focus on what are called lower-order components, that is, the parameters themselves. 12. Because our endogenous variables are dichotomous, the weighted least squares means and variance adjusted (WLSMV) estimator was used, as it is recommended when at least one endogenous variable is categorical-binary or ordinal (Muthén & Muthén, 1998-2010. 13. The main reason for this crossover is that although Mplus allows for the estimation of non-linear structural equation models (i.e., probit, logit, etc.), it is not aptly suited to simulate and estimate the predicted probabilities as we have defined them. On the contrary, while the estimation of predicted probabilities is fairly simplified using Stata and allows for structural equation models, estimation of models with dichotomous outcomes is not yet permitted. Our strategy thus combines the benefits of both computational capacities. 14. The formula to obtain an estimation of probabilities from an odds ratio is the following: P(Y ) e logit = = + − 1 1 ( ) p , where e is the natural logarithm, and logit(p) is the estimated logit coefficient for a given independent variable. 15. The model was estimated with 1,564 observations, following a listwise deletion procedure. 16. The model described in Figure 1, although being adjusted for gender, age, and administrative domain, does not show these relations, out of parsimony. Nonetheless, the model revealed that age is not a significant factor in any of the three equations. In return, it appears that being a woman, rather than a man, has a statistically significant effect on the probability of interacting with academic researchers (i.e., in the second equation). And finally, some ministries (when compared with the Ministry of Justice, the reference category) have a statistically significant effect on the probability of using research, thus corroborating Hypothesis 4. 17. We retain the use of the percentage symbol (%) here, for simplicity, but what we mean is percentage points defining absolute change, rather than a relative amount of a given quantity. 18. Also, note that this is the only case where a mediated effect through the attitude toward research variable is found. 19. In fact, 12 of the 15 dummy variables are significant and describe positive relations in the first equation modeling research utilization. 20. 4 out of the 15 dummy variables were found significant and described positive relations in the second equation modeling interactions with academic researchers. 21. 3 out of 15 dummy variables were found significant and describe negative relations in the third equation modeling attitude towards research. 22. For instance, recall that the administrative training variable had a total effect of 27.6% and a direct effect of 20.8% when calculated from observed values.