What Do Policymakers Want From Us? Results of a Survey of Current and Former Senior National Security Decision Makers

What do the most senior national security policymakers want from international relations scholars? To answer that question, we administered a unique survey to current and former policymakers to gauge when and how they use academic social science to inform national security decision making. We find that policymakers do regularly follow academic social science research and scholarship on national security affairs, hoping to draw upon its substantive expertise. But our results call into question the direct relevance to policymakers of the most scientific approaches to international relations. And they at best seriously qualify the “trickle down” theory that basic social science research eventually influences policymakers. To be clear, we are not arguing that policymakers never find scholarship based upon the cutting-edge research techniques of social science useful. But policymakers often find contemporary scholarship less-than-helpful when it employs such methods across the board, for their own sake, and without a clear sense of how such scholarship will contribute to policymaking.

In his April 14, 2008, speech to the Association of American Universities, then-Secretary of Defense Robert M. Gates argued that "we must again embrace eggheads and ideas." The key assumptions undergirding what he dubbed the Minerva Initiative were that "throughout the Cold War, universities had been vital centers of new research" and that at one time US national security policymakers were successful in tapping intellectual "resources outside of government" to guide them in formulating policy (Gates 2008). In that same spirit, then-Democratic Presidential hopeful Barack Obama promised while campaigning in Virginia in August 2008 to assemble a policy team consisting of "the best and the brightest" with the objective of tapping universities to bring important expertise on to his Administration's foreign and security policy teams (Bohan 2008).
Obama's and Gates' efforts to bridge the Beltway and the Ivory Tower gap came at a time, however, at which it never seemed wider. Harvard Professor (and former high-level State Department, Defense Department, and Intelligence Community official) Joseph Nye penned a widely discussed article in the Washington Post, in which he opined that "the walls surrounding the ivory tower never seemed so high" (Nye 2009a:A15). There is a broad consensus that this gap has widened in recent years and widespread concern that it was a bad thing for both policymakers and scholars. 2 According to the 2011 Teaching and Research in International Politics (TRIP) survey of international relations scholars, nearly 85 percent of American scholars recognized that a theory/policy gap persisted or was growing in size (Maliniak, Peterson, and Tierney 2012:66). The TRIP surveys also clearly demonstrate that "there is a disjuncture between what American scholars of IR think about the value of producing policy-relevant work and the actual research they generate" (Maliniak, Oakes, Peterson, and Tierney 2011:437).
The TRIP survey and our parallel policymaker survey show that very few scholars (8%) or policymakers (4.7%) believe the former should not contribute to policymaking in some way (Maliniak et al. 2012:67). But many of the former argue, as do Political Scientists Jeffrey Frieden and David Lake, that "only when International Relations brings science to the discussion does it have anything of enduring value to offer, beyond informed opinion" (Frieden and Lake [2005:137-138]; Bueno de Mesquita and Morrow 1999:56-57 give "pride of place" to logical consistency over originality and empirical validity). This view, however, rests on a definition of science that assumes that it can only be expressed mathematically (for example , Martin 1999:78).
Widespread acceptance of such a narrow definition could explain why, in the words of the authors of the TRIP study, American IR has become "a field whose members believe their work is more policy relevant than it actually is" (Maliniak et al. 2011:460). While most IR scholars report their primary research approach is of a qualitative nature, the TRIP survey of publications in "top journals" shows that they are dominated by quantitative articles. The majority of the TRIP respondents recognize that these state-of-the-art methodologies of academic social science constitute precisely those approaches that policymakers find least directly useful to them. In addition, few of these articles provide much direct policy advice.
In this paper, we try to answer one key question: What, precisely, do the most senior national security policymakers want from international relations scholars? An answer to this question matters because there has been recurrent interest among policymakers since the Second World War in drawing upon academic social science expertise in support of more effective national security policymaking. Despite this high-level interest, there has also been enduring frustration on both sides of the "theory/policy gap" with our inability to bridge it. One of the primary obstacles to building this bridge is systemic data about when and how academic social science is useful to policymakers. As early as 1971, a National Academy of Science study concluded that "what are required are assessments of the research needs and resources from the point of view of policy-makers" (Advisory Committee on the Management of Behavioral Science Research in the Department of Defense 1971:28).
Working with the TRIP project at the College of William and Mary, we have taken a first step to get a better sense of when and under what conditions policymakers pay attention to the work of academic social scientists and what specifically they find useful or not by administering a survey to current and former policymakers to gauge how they use academic social science to inform national security decision making. Our unique policymaker survey provides the only systematic evidence to date of what the highest-level national security decision makers want from academic international relations scholars. 3 To obtain it, we asked questions from the TRIP project survey-the largest and most extensive data collection effort to date on the field of international relations-that shed light on when and how, from the senior national security decision maker's perspective, academic social science research has been useful to them. Our policymaker (or demand-side) survey coincided with the fourth TRIP survey of international relations scholars (the supply side). These two surveys share some common policy questions. In addition, we asked policymakers their views on the usefulness of academic ideas and studies, while academic respondents were asked a number of questions about when and how academic social science research has been useful to senior national security decision makers.
In addition to providing guidance to scholars interested in doing policy-relevant work, we believe that a better understanding of what policymakers want will help us adjudicate among three different perspectives on the relationship between science and policy in our discipline. If the scientific purist position is correct that science and policy are two distinct realms with little real overlap, and that scholars should focus exclusively upon the former, there should be no demand from policymakers for academic expertise (for discussion of this position, see Stokes 1997:27-45). Conversely, if the argument that the most scientific approaches to international relations are directly relevant to policymakers is right, we should see evidence that the latter are familiar with the leading advocates of this approach, and consciously employ their methods and findings in the course of their duties (see also King, Keohane, and Verba 1994:14-19). Finally, if proponents of the "trickle down" approach are right, that the progress of science indirectly confers policy benefits via its normal operations, we should at least expect familiarity with the findings of the most widely embraced academic theories and most methodologically sophisticated approaches to the academic study of international relations (Stokes 1997:45-57;Walt 2005:25;Bennett and Ikenberry 2006:651).
Briefly, we find that policymakers do regularly follow academic social science research and scholarship on national security affairs hoping to draw upon its substantive expertise. This calls into question the scientific purist claim that the realms are completely distinct. Our results also call into question the direct relevance to policymakers of the most scientific approaches to international relations. The former evidence little familiarity with the proponents of those approaches that employ the discipline's most sophisticated methodologies. Finally, our results at best seriously qualify the "trickle down" theory that basic social science research eventually influences policymakers because policymakers show little familiarity with, or confidence in, even the findings of most of those methods and theories.
While policymakers do use theory (what they refer to as background and frameworks), they are skeptical of much of academic social science which they see as jargon-ridden and overly focused on technique, at the expense of substantive findings. Not surprisingly, rank in government is often negatively associated with tolerance for sophisticated methods; more striking, in our view, is that level of education also has that same negative correlation, indicating that it is those most familiar with those theories and techniques who are most skeptical of them. Finally, policymakers believe that the most important contributions scholars can make are not as direct policy participants or trainers of aspiring government employees, but rather as informal advisors or creators of new knowledge. However, severe time constraints limit their ability to use such scholarship in any but its very briefest presentation. In sum, the short answer to our question is that what the academy is giving policymakers is not what they say they need from us.
To be clear, we are not arguing that policymakers never find scholarship based upon the cutting-edge research techniques of social science useful. Rather, we are making a more nuanced argument: That policymakers often find contemporary scholarship less-than-helpful when it employs such methods across the board, for their own sake, and without a clear sense of how such scholarship will contribute to policymaking. In addition, policymakers often find the contributions of qualitative social science research, which is increasingly less well represented in the discipline's leading journals, of greater utility. Given the predominance of quantitatively oriented articles and this lack of attention to policy advice, it should not be surprising that a theory/policy gap remains.
In the remainder of this paper, we first describe how we constructed our survey pool and report some basic demographic statistics about our respondents. Next, we highlight the substantive results about what scholarship policymakers find useful and how they use it. We then try to anticipate some of the most important objections to our findings and interpretations. Finally, we conclude with some concrete recommendations as to how to make IR scholarship more useful to policymakers based on policymakers' responses with an eye toward both documenting the gap and providing guidance for those scholars who want to bridge it.

Description of the Survey
To construct our survey pool, we tried to identify all of the senior government officials involved in national security decision making in the George H.W. Bush, Bill Clinton, and George W. Bush Administrations. We sought to identify positions tasked with making, analyzing, and implementing policy and so excluded positions whose primary responsibilities included management, coordination, and/or legal counsel. We focused on seven departments and agencies. These included the Departments of Defense, Homeland Security, and State, the Central Intelligence Agency, the National Security Council, the Office of the Director of National Intelligence, and the United States Arms Control and Disarmament Agency. Obviously, not every department or agency existed for the entire period. We relied primarily on the U.S. Government Manual and the Federal Yellow Book to identify "policyrelevant positions" within each department and agency (Washington Monitor, Inc Various years, 1976; Office of the Federal Register 2008). Table 1 provides an outline of the positions we selected. Of the 915 officials we tried to contact, 234 responded and completed the survey, either through the mail or online. 4 Our reported response rate of 25% is probably a conservatively low estimate because we undoubtedly had inaccurate contact information for some of the policymakers who did not respond.
The respondents are part of a specialized pool of government officials. As such, they shared many traits in common (see Table 2). The youngest respondent was 32, while the average age was 59. The vast majority were also white (90%) and male (85%). Fully 85% had some form of post-graduate training. We weighted the survey pool toward high-level officials and those with direct policymaking responsibilities. The respondent demographics reflected this fact; 59% reported their primary job responsibility as policy making/policy advice, and a plurality (44%) described their highest rank in the US government as Senate confirmable policy or department/ agency leader. The average length of government service was 24 years. The greatest diversity came from the respondent's primary disciplinary background, though nearly a third of respondents (30%) received their primary training in international affairs.

Analysis of Results
Our primary objective with the policymaker survey was to understand what policymakers want from academic social science research. To that end, we sought to determine their views on the influence of individual scholars, prominent theories in the academy, and increasingly popular methodological approaches. We also asked them to rate how useful specific academic disciplines and sources of information were to their work in government. Finally, we assessed how they thought social science research contributed to their work as well as how social scientists can best contribute to national security decision making. The full text of the survey is available online at www.nd. edu/~carnrank.
In this section, we focus first on what policymakers find useful in terms of the contributions of various social science disciplines, the helpfulness of different methods or analytical tools, the utility of various theories, and the application of all of these to a number of substantive issues. We then report how policymakers use the work of scholars by considering their views of the roles academics should play in policymaking, the mechanisms by which they access the findings of scholars, and how often they make use of scholarly work in their government jobs.
Policymaker Assessments of Disciplines, Theories, and Methods Figure 1 tracks policymaker views of six academic disciplines. Aside from economics, the scholarly disciplines that policymakers found of greatest use were area studies and history.
Respondents were more tolerant of "highly theoretical writings [and] complex statistical analysis of social science topics" in the realm of economics, but also expressed preference for that part of the discipline's work which was comprised of "analysis of economic and demographic trends broadly." Interestingly, controlling for other factors, the higher the policymaker's level of education, the less useful he or she found economics to be (see Table 3). 5 Conversely, officials at higher ranks in government and those with substantial policy-making responsibilities were more likely to find the arguments and evidence in area studies useful to them. Area studies has historically been interdisciplinary and more influenced by humanistic approaches to scholarship. Political Scientist Richard Betts attributes their demise within the social science disciplines to changes which have "driven area studies out of fashion" in the name of becoming more scientific (Betts 2002:59).
Finally, compared to the other disciplines, political science did rather poorly (see Figure 1). This lower ranking may reflect the fact that in recent years the discipline has become dominated by more complex methodologies such as formal modeling and statistics. Policymakers tend to eschew, in the words of one respondent, "all formulaic academic, as opposed to historically based temperamental, realist projects," preferring, in the words of another, "historical analysis, case studies, theoretical writings that illustrate theory with case studies and concrete examples." The higher the respondent's government rank, the less Assistant secretaries for regional and policy/issue offices (for example, African affairs, refugee programs, political-military affairs) National Security Council (1989Council ( -2008 Assistant to the President for National Security Affairs (National Security Advisor) Deputy Assistant(s) to the President for National Security Affairs Regional offices ( likely an individual was to rank political science positively. The same was true for respondents with greater education. Only policymakers with a political science background were more likely to rank the discipline highly than others. We also asked policymakers about the usefulness of various approaches or methodologies for conducting social science research. The approaches that the most policymakers identified as "very useful" included area studies, contemporary case studies, historical case studies, and policy analysis (see Figure 2). As one respondent put it in the open-ended responses, "most of the useful writing is done by practitioners or journalists. Some area studies work is useful as background material/context." Another cited "any analysis (for example, in area studies) that gets at the UNDERLYING causes, rather than current symptoms, of problems has deep policy value." A third listed "case studies-Kennedy School, Maxwell School, Georgetown-Pew" as an example of social science research that has been, is, or will be useful to policymakers in the formulation and/or implementation of foreign policy.
Conversely, the more sophisticated social science methods such as formal models, operations research, theoretical analysis, and quantitative analysis tended to be categorized more often as "not very useful" or "not useful at all," calling into question the direct influence of these approaches to international relations. Indeed, the only methodology that more than half the respondents characterized as "not very useful" or "not useful at all" was formal models. As Table 4 shows, the higher the rank of the government official, the less likely he or she was to think that formal models were useful for policymaking.
To be sure, one respondent observed that "the work of scholars such as Howard Raiffa and Thomas Schelling in the area of game theory and systems analysis has been of great utility." But more typical were the negative responses to our open-ended invitation for policymakers to "list an example of social science research that you believe has NOT been, is NOT, or will NOT be useful to policymakers in the formulation and/or implementation of foreign policy." Among these, respondents placed "Most formal modeling," "Large-N studies." "The time spent on computer modeling of international systems or conflict resolution is a complete waste. Much of the theory work is as well." "Highly theoretical and quantitative analysis that seems to be more concerned about the elegance of the model than the policy utility." "Many micro-economic models and fitting of history into larger theories is not very useful. Many professors do not want to influence contemporary policy." "Highly theoretical writings [;] complex statistical analysis of social science topics (except economics). Writings that use arcane academic jargon." "Most any quantitative study; virtually every article in APSR" "Formal/game theoretical work and quant in political science-most of what passes as 'methodologically sophisticated' international relations work." One exception to policymakers' aversion to quantitative social science was in the area of public opinion analysis. Respondents included among "useful" approaches "public opinion research/analysis of foreign audiences by whomever." Another argued that "polling data and its analysis is perhaps the most basic and certainly among the most useful such products." A third specifically noted that "PEW Global Attitudes Project has been very important to framing America's position in the world and changing dynamics of globalization and modernity." Indeed, multiple policymakers cited Pew as doing useful survey research. Finally, a fourth agreed that "opinion polling can be very useful in trying to determine what populations think, especially in countries where freedom of expression is limited." In addition to public opinion surveys, two respondents also pointed to "Mansfield and Snyder, Electing to Fight: Why Emerging Democracies Go to War" and Peter Feaver's and Christopher Gelpi's "research on how public support for military operations is affected by casualties and other costs of war has had a direct impact on policymakers" as among the few examples of useful quantitative social science scholarship. 6 The latter relied primarily upon public opinion survey data.
Finally, Table 5 compares the utility ranking for the various approaches between the scholar and policymaker surveys and finds that the assessments of the two are actu-  ally highly correlated. Apparently, the theory/policy gap is not the result of ignorance on the part of academics about what is useful to policymakers, but rather must be driven by something else, perhaps professional incentives within the academy.
We also wanted to gauge policymakers' views of important academic theories of international security, a good indicator of social science's indirect influence. First, we asked policymakers whether they were familiar with a particular theory. If so, we then asked them a series of questions about how they learned about the theory, how confident they were in the accuracy of that theory, how useful they thought it was to policymakers, and if they themselves used it in their work for the US government.
As Figure 3 illustrates, the theories that policymakers have the greatest familiarity with are Mutual Assured Destruction (MAD; for example, Jervis 1989); the "Clash of Civilizations" (for example, Huntington 1993:22-49); Population Centric Counter-insurgency (PCOIN; United States Department of the Army 2007); and Structural Realism (SR; for example, Waltz 1979). Policymakers were slightly more likely than not to be familiar with the Democratic Peace Theory (DPT; for example, Russett 1994), but only 20 percent were acquainted with Expected Utility (EU) theory (for example, Bueno de Mesquita and Lalman 1994), which weakens claims about even the indirect influence of the most scientific approach to international relations.
Interestingly, the longer a policymaker served in government, the less likely he or she was to be familiar with the Democratic Peace Theory (see Table 6). This could either be a function of the relatively recent consensus in political science on the DPT's core propositions or reflective of its perceived lack of utility. We are inclined to think that it is the latter because respondent age is not a significant predictor of familiarity with DPT.
Conversely, government officials whose positions include substantial policy-making responsibilities were more likely to be familiar with Structural Realism, a theory roughly contemporary with DPT. 7 Indeed, having substantial policy-making responsibilities doubles the odds of being familiar with Structural Realism, all other things being equal. 8 When we tried to gauge policymakers' confidence in the accuracy of various theories, we found that respondents were most confident in MAD and PCOIN (see Figure 4). Conversely, they were most skeptical of Samuel Huntington's "Clash of Civilizations," with one respondent dismissing it on the grounds that "the work has a very ethnocentric based approach" and three others criticizing it in the open-ended comments. The DPT, EU, and SR were in the middle in terms of respondents' assessment of their accuracy.
Similarly, the theories that the greatest numbers of policymakers find of use are PCOIN and MAD. Surprisingly, despite doubts about its accuracy and utility to the particular policymakers, the "Clash of Civilizations" scored reasonably well on this global assessment of utility (see Figure 5). Structural Realism and DPT also seemed reasonably useful. Conversely, the lowest number of policymakers found Expected Utility theory of utility. Finally, we asked a slightly different question of policymakers to try to gauge which theories were of the most direct use to them. As Figure 6 shows, those theories were PCOIN, MAD, and despite their other reservations about it, Structural Realism, with more than half of respondents saying that these theories directly influenced the work they did. Conversely, fewer policymakers found the "Clash of Civilizations," DPT, and Expected Utility to be of use to them personally. Government officials whose primary duties were policymaking were more likely to find the theory of Mutual Assured Destruction of use to them in their daily work, as Table 7 suggests. The higher their government rank and education level, the less likely were policymakers to find Expected Utility to be of direct use to them in their work for the US government, as Table 7 also shows. Table 8 offers a comparison of scholars' and policymakers' views of the top ten most influential international relations scholars. Respondents could write in four names. Each individual in the policymakers' top ten received at least five mentions. Columns 1 and 2 report the results from the 2011 TRIP scholar survey asking for respondents' assessments of the scholars who did the best and most influential work in international relations, respectively. Column 3 reports the results of our question to policymakers asking them to list which scholars were the most influential in the policy realm.
For the scholars, the most highly ranked individuals tend to be those who pioneered theoretical paradigms or developed sophisticated methodological approaches to the study of international relations. For example, Alexander Wendt and Peter Katzenstein are widely regarded as the leading proponents of the Social Constructivist approach to international relations, Robert Keohane is a prominent scholar of international political economy within the Neoliberal Institutional framework, James Fearon is among the most sophisticated quantitative analysts of international security affairs, and Kenneth Waltz and John Mearsheimer are the most visible figures with the Neorealist paradigm.
On other hand, scholars who served in public office do very well on the policymaker list, particularly Harvard's Joseph Nye, Henry Kissinger, Samuel Huntington, and Graham Allison. A notable fact about the top ranked scholars on the policymaker's list is that they were not, for the most part, known for their methodological sophistication. Even Thomas Schelling, who pioneered the use of game theory in the analysis of deterrence, was reportedly not regarded among some academic Economists as being particularly rigorous methodologically (Mearsheimer 2004:393-394). While there are limits to how seriously we should take policymakers' assessment of scholars given the fact that many of the leading names were either dead at the time of the survey (Albert Wohlstetter, George Kennan, Samuel Huntington, and Hans Morgenthau) or have only tenuous claim to scholarly standing (Fareed Zakaraia), it is important to keep in mind that no senior policymaker credited leading international relations scholars such as James Fearon or Alexander Wendt as having much influence upon the policy world. 9 Finally, we asked policymakers about their views of the most important regions of the world. Figure 7 compares the percentage of policymakers who rank a region as the "most important" with scholars' primary or secondary research focus in the 2011 TRIP survey. What this comparison reveals is that there is a substantial difference between policymakers and scholars in terms of which regions of the world the former regard as critical and the latter actually study. 10 This is particularly apparent with East Asia (which scholars neglect; see also Hundley, Kenzer, and Peterson, forthcoming) and the former Soviet Union (to which scholars continue to give disproportionate attention despite the decline in policymakers' interest in it). We surmise that this reflects, in part, a generational lag due to the continuing presence in the Ivory Tower of scholars who developed expertise in the Former Soviet Union at the beginning of their careers. It may also reflect the difficulty of their retraining themselves or the academy producing large numbers of new scholars with the language and culture expertise in East Asia. Finally, it highlights the trade-offs between developing the sort of deep expertise that academics specialize in and being relevant to the rapidly changing needs of policymakers. In any case, this is yet another example of the disconnect between the Ivory Tower and the Beltway.

How Policymakers Use Scholarship
Several respondents were highly skeptical about social science's potential to contribute to policymaking. As one respondent put it, "policy implementation is too complex for outside analysis to be relevant. The challenges are real time, in the moment, and situational." Another argued that "there is no time for academic theories in getting policy decided and implemented. It is nearly irrelevant except as a general influence over time." A third con-  Anne-Marie Slaughter 4 10 Bernard Lewis 4 (Notes: Scholars' rankings from Maliniak et al. 2012:48-49.) ceded that "I take the occasional idea or fact from social science research, but find most of it divorced from reality and so lagging events as to be unhelpful." A fourth responded, "I do not recall any recent example of social science research that I thought was particularly useful to policymakers." A fifth dismissed "Most of it" and added that he or she could not "think of an exceptionally useless example at this moment (which is itself perhaps a useful insight)." A sixth explained that "most social science research from academia is of little value to policymakers, as it is more focused on theory rather than practice." Finally, one critical respondent complained that while it was "not an option in the survey, but much of the influence has been negative, creating misleading generalizations to be offset or countered." Despite these pointed critiques, our results show that it is not the case that most policymakers simply ignore social science or believe that academics should have no role whatsoever in the policy-making process, which calls into question the scientific purist position. When asked how scholars should contribute to policymaking, fewer than 5% said that scholars "should not be involved in pol-icymaking." Rather, majorities of respondents said that scholars should contribute as "informal advisors" (87%), "creators of new information/knowledge" (72%), and "trainers of policymakers" (54%; see Figure 8). Slightly more than a third thought scholars should be formal participants in the policy-making process.
While more than half thought that they should also serve as trainers of policymakers, we were surprised that support for this role was so limited. This seems to challenge the view that the most important contribution scholars can make is to train the next generation of policymakers (Nye 2009b:117). Indeed, Figure 9 shows that less than 30 percent of respondents received the "most important intellectual skills" that they use from their education, while more than 60 percent instead selected "field or work experience" or "professional education/job training." Taken together, these findings raise important questions about the curriculum and content of much graduate professional education in international affairs. Figure 10 shows the results of a cross-tabulation of this question with level of education, which highlights the striking fact that in contrast to other education levels, Ph.D.s in government claim to have acquired the most important skills from their formal educations. Figure 11 breaks this down by discipline, revealing that this finding is most likely driven by Ph.D.s in political science and international affairs. Given that the average age of political scientists in our sample is 59 years (meaning that they likely earned their degrees over 30 years ago), and the fact that those with a Ph.D. in economics did not report the same usefulness for their formal training, we are hesitant to interpret this as an endorsement of the current trends in these disciplines.
Finally, Table 9 compares scholar and policymaker rankings of various universities. The 2011 TRIP survey asked for scholars' views of the top Ph.D. and professional master's programs. Our policymaker survey solicited policymakers' views on which universities have faculty who produce the most policy-relevant research in the social sciences and which train the best candidates for jobs with the US government. While these categories are not fully comparable, there are nevertheless suggestive differences between them.  What is notable about almost all of these programs is that they are linked with a major public policy school or program. Conversely, there is more consensus among scholars and policymakers on the top M.A. programs and those schools that send graduates into government service. In other words, the big divergence is on research and Ph.D. training.
While policymakers say they want to engage scholars on policy issues, few of the latter spend much of their time reciprocating. American scholars estimated that they only devoted 6 percent of their work time to nonacademic consulting activities (Maliniak et al. 2012:25). That amount of time fell below the world average and far below countries such as France and Israel. Furthermore, 55 percent of American scholars said that they had not done any paid consulting at all in the past 2 years and 56 percent said they had not done any unpaid consulting in the past 2 years (see Figures 12 and 13). The half full part of this glass is that 45% do spend some time engaging the real world; the more than half empty aspect of it is that those who do spend relatively little time at it.
Turning to the question of how scholars' work influences policymakers, Figures 14 and 15 suggest that the majority of policymakers find the arguments and evidence of scholars to be of most use in providing "intellectual background" to them, rather than "directly applying" to their work, providing more evidence against the direct effect argument. One respondent observed that social science was "useful as a backdrop to the daily policy decision making," but qualified that by saying that "it helps us frame our thinking, but does not have direct influence." Another echoed this view, noting that "There is no time for academic theories in getting policy decided and implemented. It is nearly irrelevant except as a general influence over time." "When I headed [a component of the U.S. intelligence Community], awareness of social science research was important for our analysts, for their work and to understand others," one respondent explained. Another noted that "I need a broader context for my work, that incorporates military, regional, political, economic factors, to help me devise policy solutions," as a role for social science. These comments suggest that there are real limits to the sort of social science work that trickles down to indirectly affect policymakers. Figures 16 and 17 show that the most important sources of information for policymakers are classified information and newspapers. This makes sense in terms of the unique resources inside government and also the limited time policymakers have to read outside materials. It is striking, however, that policymakers find newspapers as useful as classified information, lending more credence to the widely recognized-if seldom acknowledged-fact that most policy is made based upon open sources (Langer 1948:43;Winks 1996:62-63, 475;Steele 2008:138). Conversely, and also not surprisingly, books (both scholarly and trade) and television and radio do not rank as highly as sources of information. In between these extremes are scholarly articles and the Internet as important sources of information.
Apropos of the Internet, we were surprised to learn that the more educated the policymaker, the less likely he or she is to rely upon the Internet or other online sources for information (Table 10). It is, of course, possible that this finding is an artifact of the relatively high average age of our respondent pool, though if it were, "age" ought to be a statistically significant variable. Given that, this finding suggests that more thought needs to be given to how the Internet might facilitate scholarly input into policymaking, especially given the growing enthusiasm among scholars for blogging as a means to reach a wider audience. 11 Interestingly, the same is true of television and the radio. Because most scholars are unlikely to be able to contribute to the classified sources policymakers use, their most prominent avenue of influence is the print media.  Aside from brevity, print media-especially the elite newspapers-probably retains its influence as a result of its reputation and role as the definitive source of "all the news that's fit to print." As New York Times columnist Nate Silver suggests, the Internet and the blogosphere at present increase the "noise" to "signal" ratio and so the traditional  Maliniak et al. 2012:60.) print media may still have an important role to play in providing authoritative news and opinion for policymakers for some time to come (Silver 2012:13). When asked how frequently policymakers make use of social science arguments, only 7 percent selected "never" (see Figure 18). Nearly 45 percent said "daily" or "a few times a week." Respondent characteristics made no major difference in how often they used social science arguments. Policymakers made use of the evidence, as opposed to the arguments, presented by social scientists less frequently (see Figure 19).
This seems counterintuitive in that policymakers seemed to use social science arguments (what we would call theories) somewhat more often than social science evidence (findings or data). Given policymakers' reservations about abstract theory, we would have thought the reverse would have been true. Our hunch is that this supports the view, advanced most notably by the late Alexander George, that policymakers are looking for what he called "middle-range theory" (George 1993:139-142).

Response to Likely Objections
There are a number of reasonable objections to the results that we report which we want to try to anticipate. First, one might argue that because we surveyed only very high-level policymakers, our findings about what they find useful might skew our assessment of what sort of social science research is useful throughout the US government, particularly at lower levels. If we had been able to survey a broader pool of national security officials, so this argument might go, we would have gotten very different results about what they find useful from social science. For example, more sophisticated research methods may be of greater use to individuals at the analytical and policy-support levels. These individuals may, in turn, employ those methods, or findings derived from them, in advising senior policymakers without necessarily alerting the latter to their role. This is, of course, possible, but needs to be demonstrated rather than asserted.
A related objection to our approach would be that instead of asking what senior policymakers want, we should "follow the money" and see what sort of research the US government spends money on. If we did so, according to this line of thinking, we would no doubt find that among these contracts were projects employing advanced methodological techniques.
For example, the US government has contracted for a variety of "big data"-type projects in connection with the prosecution of the Global War on Terror and the counterinsurgency campaigns in Iraq and Afghanistan, using sophisticated social network analyses to try to anticipate future such attacks and draw a map of our adversaries (for discussion of this, see Weinberger 2012). While these efforts are no doubt worth pursuing, the fact that the United States has undertaken them by no means ensures their success nor indicates that their results will directly or indirectly shape policy (for discussion of the limitations of big data, see Silver 2012:9-12). Skeptics might also point to some high-profile examples of this sort of scholarship which appear to have garnered press and even Congressional attention. 12 This is no doubt true and is worth examining in future research. But the fact that such work receives some media or even Congressional attention does not establish that it actually informs policy.
Others point to the recent influence of the "Democratic Peace Theory" upon the Clinton, George W. Bush, and Obama Administrations as evidence that the most  (Levy 1988:88). Political Scientists Jeffrey Frieden and David Lake argue that the democratic peace became relevant outside of the academy precisely "because of the law-like status of a particular empirical finding" (Frieden and Lake 2005:142). Bruce Bueno de Mesquita similarly maintains that "So strong are the statistical patterns that policymakers as well as scholars have embraced the idea that a democratic peace exists. Indeed, Bill Clinton in 1994 specifically cited his belief in the democratic peace findings as a basis for American action in restoring Bertrand Aristide to power in Haiti" (Bueno de Mesquita 2004:232; see also Bennett and Ikenberry 2006:655). However, it is not clear that the influence of the Democratic Peace Theory on recent US foreign policy was due to its unassailable scientific standing. As we have reported above, only slightly more than half the policymaker respondents were even aware of DPT. Moreover, while former Defense Department official and Ohio State Political Scientist Joseph Kruzel conceded that the democratic peace "had substantial impact on public policy," he attributed its attractiveness to its conceptual simplicity rather than its scientific rigor (Kruzel 1994:180). It clearly identifies our enemies (nondemocratic states) and prescribes a simple response to them (make them democratic). Kruzel was certainly in a good position to judge why DPT was as influential as it was as he was both a scholar and a senior policymaker. Finally, one could argue that US policymakers have embraced the democratic peace because of its compatibility with our political culture rather than due to its scientific standing (Oren 1995:147-184;Desch 2007Desch /2008. A related objection to our findings is that while they may be valid for today's senior policymakers, they may not provide an accurate roadmap as to how future policymakers will use academic social science or garner the information upon which they make policy decisions. The notion that tomorrow's senior policymakers may be more methodologically and technologically savvy than today's is also plausible but should not be overstated given that today's senior national security policymakers were often exposed to cutting-edge research techniques such as formal models, game theory, and econometrics well before they took hold throughout the Ivory Tower. Under the auspices of RAND, the subfield of security studies tried out-and in many cases found wantingmany of the techniques that the rest of the discipline of political science would enthusiastically embrace years later (Hounshell 1998:254-255). Also, keep in mind that nearly three-quarters of our respondents had post-secondary degrees, a quarter of them were disciplinary Ph.D.s.
Moreover, there is little doubt that tomorrow's senior policymakers will be very different from today's in many respects, including getting more of their open-source information from the Internet and blogs. But it is also worth thinking hard about the differences between print media and the blogosphere-precisely in terms of volume of information available and its often questionable reliability-before assuming that this shift is inevitable.
Finally, one might concede that our findings and interpretations of them are valid but nonetheless make a normative argument that policymakers ought to want the results of cutting-edge social science international rela-tions scholarship because they would lead to better policy decisions. To be sure, a compelling case can be made that greater scholarly input into national security decision making would have averted significant missteps over the years, particularly in Vietnam and Iraq, although it is not clear that one needed to employ cutting-edge social science techniques to foresee these problems. (On the former, see "5,000 Scholars" 1964:1-2. On the latter, "War with Iraq" 2002:29.)

Conclusions, Recommendations, and Suggested Avenues for Future Research
In conclusion, we first highlight the main findings of the policymaker survey. Next, we offer what we think are the most important policy recommendations that flow from these findings. Finally, we highlight some continuing puzzles and future research questions.
Our results clearly show that policymakers do want scholarly expertise, challenging the scientific purists' strict separation of science and policy. On both sides of the theory/policy divide, the majority of voices clamor for a bridge. But they also call into question when and how often the techniques of the modern science of international relations are directly useful to policymakers. Policymakers seem to prefer mid-range theories to help them make sense of the world and their presentation in brief and jargon-free formats. Finally, these results also speak to the question of the indirect influence of social science on policymakers: While some basic research clearly trickles down (more accurately, percolates up) to the policy world, it is generally not that which is based upon the most scientific approaches to the academic study of international relations. Indeed, our findings about the utility of social science for policymakers in many ways parallel those of the 1960s era "Project Hindsight," which found that little natural science basic research actually translated into useful defense technology (see Sherwin and Isenson 1967:1571-1577. While it will be no surprise to scholars and policymakers that the gap between the Beltway and the Ivory Tower persists, and is probably even growing, we believe that this policymaker survey, in conjunction with the 2011 TRIP scholar survey, has documented that this gap is both substantive (what issues and areas matter) and also methodological (how we should study international relations and national security issues). The gap between the scientific aspirations of contemporary international relations scholarship and the needs of policymakers is greatest the higher one reaches in the policy world. More surprisingly, this gap tends to be greater the more educated the policymaker. This is consistent with the argument that familiarity with advanced techniques instills greater appreciation for both their promise and limits. (For a related argument, see Eriksson 2012:746-749.) It is also worth reiterating that we are not advocating scholars abandon sophisticated research methods or suggesting that these methods are never useful for policy-relevant scholarship. Rather, we believe that policymakers operate under some logic of appropriateness in terms of their tolerance for sophisticated social science (especially statistics and formal models) based upon the particular issue in question. For example, respondents to our policymaker survey seemed amenable to the use of the latest social science tools and techniques in certain realms (economics and public opinion surveys), just not for their own sake. Another conclusion we draw from this survey is that a scholar's broader visibility-both in government and among the public, whether through previous government service or publication in broader venues-enhances influence among policymakers more than his or her academic standing.
Of course, we have long known that the primary constraint policymakers face in digesting scholarly, or any other writings, is lack of time. As one respondent put it, "any research papers that exceed 10-15 pages" are not useful to policymakers. Another noted that "I do not have the time to read much so cannot cite" many examples of useful social science scholarship.
We were surprised by two other findings from our survey about how policymakers get their information: First, unclassified newspaper articles were as important to policymakers as the classified information generated inside the government. This fact opens up an important avenue for scholarly influence upon policy if scholars can condense and convey their findings via this route.
Second, the Internet has not yet become an important source of information for policymakers, despite its ease of accessibility and the generally succinct nature of the presentation of its content. It could be a just a matter of time until a more web-oriented generation reaches the pinnacle of national security decision-making authority, but we also ought to consider whether the Internet suffers from weaknesses vis-a-vis traditional print media that dilute its influence. The plethora of Internet news and opinion outlets, many of questionable reliability, combined with the lack of an authoritative source among them, may mean that the Internet will continue to lag behind the elite print media because it exacerbates the signals to noise problem for policymakers.
But our most important findings concern what role policymakers think scholars ought to play in the policy process. Most recommended that scholars serve as "informal advisers" and as "creators of new knowledge." There were two surprises for us here: First, policymakers ranked the educational and training role of scholars for future policymakers third behind these other two roles. They also confessed that they derived relatively little of their professional skills from their formal educations. The main contribution of scholars, in their view, was research. Second, and again somewhat surprisingly, they expressed a preference for scholars to produce "arguments" (what we would call theories) over the generation of specific "evidence" (what we think of as facts). In other words, despite their jaundiced view of cutting-edge tools and rarefied theory, the thing policymakers most want from scholars are frameworks for making sense of the world they have to operate in.
Given these findings, we offer the following recommendations for scholars who aspire to influence policymakers. While scholars may want to participate in policymaking, they should do so not because of the superior contribution they can make to policymaking directly but rather because doing so will enrich their scholarship. Indeed, the most important roles for scholars to play are as both teachers and researchers, but our results suggest that both areas need careful rethinking. On the former, the findings of our survey should lead to some introspection about how we train students for careers in government service. We suspect that the focus on social science techniques and methods that dominates so much graduate, and increasingly undergraduate, training in political science is not useful across the board to policymakers. On the other hand, a purely descriptive, fact-based approach is not what policymakers seem to want from scholars either.
Since policymakers think that the most important contributions scholars can make is in the area of research, it is worth thinking further about exactly what that should look like. Three aspects of scholarship appear to be most important: First, policymakers appear to want mid-range theory. Policymakers do not reject methodologically sophisticated scholarship across the board but do seem to find much of it not useful. They prefer that scholars generate simple and straightforward frameworks that help them make sense of a complex world. (For related arguments, see Silver 2012:96-97; Mearsheimer and Walt 2013.) They seem not so much to be looking for direct policy advice as for background knowledge to help them put particular events into a more general context. We interpret policymakers' preference for theories over facts to the fact that like most busy people, they are cognitive economizers who need ways to make good decisions quickly and under great uncertainty. Along these lines, Henry Kissinger reportedly demanded of his subordinates: "Don't tell me facts, tell me what they mean." 13 Second, brevity is key for policymakers. We suspect that the reason that Op/Eds are so influential among policymakers is only partly due to where they are published; another important aspect of their influence is their short length. We are by no means suggesting that scholars only write in that format, but we strongly believe that research findings that cannot be presented in that format are unlikely to shape policy. Therefore, our recommended model is one in which a scholar publishes his or her findings in traditional scholar outlets such as books or journals but also writes shorter and more accessible pieces reporting the same findings and telegraphing their policy implications in policy journals, opinion pieces, or even on blogs. 14 Finally, a related issue is accessibility: Policymakers find much current scholarly work-from across the methodological spectrum-inaccessible. Policymakers don't want scholars to write in Greek or French, but rather just plain English.
The results of this survey are by no means the last word on this large and very complex question of what policymakers want from scholars. At least, five particular puzzles or research challenges remain: First, why is it that policymakers are relatively tolerant of complex modeling and statistical work in economics and survey research but not in other areas of political science and international relations? One possibility is the logic of appropriateness argument we suggested above. But another possibility is that policymakers in international and national security affairs operate under a misconception about how influential some of those toolsparticularly in economics-really are in policymaking (Krugman 1994: chapter 1).
Second, why do scholars continue to do business in a way that they know is not useful to policymakers despite their clear desire to influence policy? While the "new scholasticism," "the cult of the irrelevant," and the "flight 13 Personal conversation with New York University Political Scientist and former Kissinger Speech Writer Lawrence Mead, March 27, 2013. 14 For example, Snyder and Mansfield published their influential argument about the higher propensity of democratizing states to go to war not only in a book and a scholarly article, but also in an accessible and high-visibility policy article (see Mansfield and Snyder 1995a:79-97).
from reality" away from policy relevance among many scholars are well documented, we lack a comprehensive explanation for this phenomenon (Shapiro 2005;Mead 2010:453-464;Menand 2010;Van Evera 2010:4-9). Third, more work clearly needs to be done exploring why undergraduate and graduate training seem not to be contributing as much as they could to the preparation of aspiring policymakers. Fourth, since it is possible that lower-ranking analysts and policy-support officials may be more amenable to work based upon the most sophisticated social science approaches and tools, it would be worthwhile to survey them as well to get a complete sense of the various indirect routes by which social science can influence policymaking.
Finally, it could be that the relative unimportance of the Internet to current policymakers is just an artifact of the high average age of our sample and the relative newness of the technology. But there may be elements of the Internet information architecture that make policymakers shy away from it that need to be considered.
In sum, we hope that these results will be of use to scholars who are interested in contributing to security policymaking by helping them better understand when and how academic social science is of use to policymakers and suggesting pathways for them to make their work useful to national security policymakers.