Public Deliberation and Co-Production in the Political and Electoral Arena : A Citizens ’ Jury Approach

This study presents an empirical evaluation of the co-production of a “Statement to the Candidates” and a “Voters Guide” for a key U.S. Congressional race. Citizens produced these materials during an intensive process called “Reclaim November Ohio,” which used the Citizen Jury method of public deliberation. We use a series of preand post-test surveys to evaluate this unique application of coproduction. Specifically, we assess whether this deliberative approach to co-production in the political and electoral arena improved participants’ perceptions of politics and government and made citizens more interested in and knowledgeable about policy issues. We also assess whether participants believed the Reclaim November Ohio process would help the candidates better understand citizens’ concerns and whether participants were satisfied with the event. The results suggest that co-production in the political and electoral arena can have positive effects on citizens’ agency, voice, and perceptions of politics and government. The results are mixed for both the perceived influence of the event and issue interest and knowledge. Nevertheless, participants were extremely satisfied with the event and its various components.

American political parties are increasingly polarized (Poole, 2012), a trend that has coincided with a continued deterioration of the public's trust in government and the media, both of which are at all-time lows (Pew Research Center, 2013;Gallup, 2013).Despite the public's frustration, citizens cannot duck all the blame.U.S. voter turnout remains the lowest in the industrialized world (Friedman, 2012), and there is emerging evidence that citizens often rely on party identification in making political decisions, substituting ideology for facts and personal experience (NORC, 2012).When citizen disillusionment does boil over into action, it may be less than productive, as was the case in a series of particularly boisterous town hall meetings on health care reform in 2009 (Shea & Sproveri, 2012).
Against this backdrop, a small Minnesota-based nonprofit decided to engage in the task of co-producing a "Statement to the Candidates" and a "Voters Guide" with citizens.These materials were created by two Citizen Juries that participated in a three-weekend event called "Reclaim November Ohio."The goal was to help voters make informed, reasoned decisions in a tight race that could determine the balance of power in the U.S. Congress.The nonprofit organization Jefferson Action started this endeavor by searching for a suitably competitive congressional race, and selected Ohio's 16 th Congressional District, where the organization had no established contacts.In a few short months, Jefferson Action recruited and trained a local staff, conducted a random-sample telephone survey of the District's voters, selected from that sample a diverse and representative group of citizens, empanelled those citizens on two Citizen Juries, educated them on key issues in the race, and led them in deliberation until broad and general agreement was reached on the materials.All the while, the organization sought, and received, substantial press coverage as a means of extending the influence of the process to the broader population.This article evaluates this unique process of public deliberation and coproduction.First, we examine briefly the concepts of public deliberation and coproduction and discuss several deficits in the American political system.Second, we explore the Citizens Jury as a potential model for public deliberation and coproduction in the political and electoral arena.Specifically, we assert that public deliberation and co-production can (1) give agency and voice to citizens, thereby improving their interactions with candidates and elected officials and improving perceptions of politics and government; (2) make citizens more knowledgeable about the content, trade-offs, and challenges of public policy; and, (3) help candidates and elected officials better understand citizens' needs, interests, concerns, and values, thus shaping the way they approach public policy decision making.Third, we discuss the methods we used to evaluate these claims.Fourth, we present several sets of results, focusing on the demographics of participants, and the effects of participation on trust in government, perceived influence of the U.S. Congress, political efficacy, issue interest and knowledge, perceived influence of the process on congressional candidates, and participant satisfaction with the process.We conclude by summarizing our findings.

Public Deliberation and Co-Production
Public deliberation is an umbrella term for participation processes that enable lay citizens, often in concert with policymakers and stakeholders, to come together in public spaces to engage in constructive, informed, and decisive dialogue about important public issues (Gastil, 2008;Nabatchi, 2012;Nabatchi, Becker, & Leighninger, 2014;Nabatchi, Gastil, Weiksner, & Leighninger, 2012).Public deliberation processes vary by several dimensions, including but not limited to who participates in deliberation, how participants exchange information and make decisions, and the link between the deliberations and policy or public action (Nabatchi, 2012).
Regardless of these and other variations, all of these processes involve deliberation -a form of discussion that centers on problem analysis, often in hopes of finding agreeable high-quality solutions (Gastil & Levine, 2005;Nabatchi, 2012).Specifically, deliberation is a rigorous approach to communication about public problems wherein people "carefully examine a problem and arrive at a well-reasoned solution after a period of inclusive, respectful consideration of diverse points of view" (Gastil, 2008, p. 8).In general, public deliberation processes proceed through five (often iterative) steps: 1.The creation of a solid information base about the nature of the problem at hand; 2. The identification, weighing, and prioritization of the key values at stake in an issue; 3. The identification of a broad range of potential solutions to the problem; 4. The weighing of the pros, cons, and trade-offs of the solutions through the systematic application of relevant knowledge and values to each alternative; 5.The arrival at the best decision(s) possible in light of what was learned through deliberation (if in a decision-making body), or the arrival at independent judgment(s) (if not in a decision-making body) (for a discussion, see Nabatchi, 2012).
Advocates of public deliberation assert that it not only has normative and intrinsic value, but also has instrumental benefits for individuals, communities, policy, government institutions, and the broader processes of governance (for discussions, see Nabatchi, 2010aNabatchi, , 2012)).In general, empirical research supports claims about these and other benefits of public deliberation (see Delli Carpini, Cook, & Jacobs, 2004;Nabatchi, Gastil, Weiksner, & Leighninger, 2012).In terms of individuals, research suggests that deliberation has educational effects on the individuals who take part in it; for example, by improving tolerance and respect for diverse views, generating issue understanding and knowledge, and increasing political efficacy, sophistication, and trust, among other civic dispositions (see Pincock, 2012).Research also suggests that public deliberation can help build community capacity by cultivating leadership, increasing organizational development, promoting community organizing, and fostering collaboration (see Kinney, 2012).Finally, research suggests that public deliberation can improve policy decisions, government institutions, and the quality of governance (see Barrett, Wyman, & Coelho, 2012).
Co-production (also known as co-creation) is gaining popularity as a means of increasing user involvement in service production and consumption.While there is disagreement about how to define the term co-production, most definitions share two common features: (1) a view of people as assets who ought to have a role in public services, and (2) a focus on action, wherein people (professionals and service users) come together to design, create, and/or deliver public services (Fledderus, Brandsen, & Honingh, 2014;Pestoff, 2014).Moreover, scholars tend to emphasize the transformative potential of co-production; that is, its ability to increase the agency of citizens and providers and improve public services.Here, we adopt Alford's (1998, p. 128) broad definition of co-production: "the involvement of citizens, clients, consumers, volunteers, and/or community organizations in producing public services as well as consuming or otherwise benefitting from them." In the United States, the practice of co-production started in the 1970s, where it was established as a means of reducing urban fiscal expenses, while accounting for rising public expectations of services (Brudney, 1984).Since then, coproduction has gained attention as a potential solution for the issues plaguing public service provision (Needham, 2006).It is believed that this approach can make services and service delivery more efficient and effective, while also enhancing the morale of both providers and users.The idea that co-production can bolster state capacity has also led to inquiries about the role of the third sector (i.e., nonprofits, non-governmental organizations, and civil society organizations) in supporting co-production activities (Kendall & Knapp, 1995).Specifically, research suggests that co-production is less effective when it is used as a top-down function and more effective when third sector organizations help mediate the relationships between producers and clients (Bovaird, 2007;Pestoff, 2006).
Over the past several decades, scholars have applied the concept of co-production in all phases of the public service management chain, from planning, designing, and commissioning, to managing, delivering, monitoring, and evaluating (Bovaird, 2007).While co-production has been used across a range of management activities, it has been applied almost exclusively within the realm of public service delivery.In this article, we apply the concept in a different realmthat of politics, elections, and voting.Specifically, we examine the co-production of a "Statement to Candidates" and a "Voters Guide" for a contested Congressional election in the United States.While this is certainly a unique application of the co-production concept, we believe it is a good fit, as the endeavor involves citizens working with professionals to produce materials for the benefit of the public at large.
In the context of public services, Needham (2007) asserts that the co-production approach has at least three distinct advantages over traditional models: (1) it gives agency and voice to frontline providers, which can improve their 'street level' interactions with citizens, thus changing citizens' perceptions about both public policy and bureaucracy; (2) it can make citizens "more knowledgeable of the content, costs and limitations of municipal services and their joint responsibility with service agents for their delivery" (Brudney & England, 1983, p. 62); and, (3) it can better match public needs, preferences, and services, thus improving the efficiency, effectiveness, and overall quality of public services.
We assert that these advantages can also be found when co-production is used in the political and electoral arena.Specifically, we believe that when coupled with public deliberation, co-production in the political and electoral arena can: (1) give agency and voice to citizens, thereby improving their interactions with candidates and elected officials and their perceptions of politics and government; (2) make citizens more knowledgeable about the content, trade-offs, and challenges of public policy; and, (3) help candidates and elected officials better understand citizens' needs, interests, concerns, and values.Such outcomes are particularly important in the context of U.S. politics and elections, which have been plagued by a variety of deficits.

Deficits in American Politics
Recent years have seen numerous problems in American politics, including citizenship, democratic, and financial deficits, all of which may threaten the legitimacy and stability of the political system.The term citizenship deficit broadly refers to an erosion of civil society and civic engagement and more specifically to an erosion of civic skills and dispositions among the general public.Evidence of a citizenship deficit in the United States is seen in the numerous statistics that show a decline in the political engagement, voter turnout, civic dispositions (e.g., internal and external political efficacy, and trust in government), and social capital of the public (for discussions, see Macedo et al., 2005;Mathews, 1994;Nabatchi, 2010a;Rosenstone & Hansen, 1993;Wattenberg, 2002).A collective view of these and other indicators suggests that Americans are experiencing a decay of their ties to each other and to the political system (e.g., Putnam, 2000).In turn, this has led to a "growing sentiment among contemporary political scientists and political analysts that the foundations of citizenship and democracy in America are crumbling" (Dalton, 2006, p. 1).
Related to the citizenship deficit is the democratic deficit, which refers to a perception that organizations, institutions, and governments are neglecting democratic principles in their practices and operations.In the United States, the term democratic deficit is used to signal the disconnect between citizen opinions and preferences and political decisions and policy outcomes (for a discussion, see Nabatchi, 2010a).This disconnect is directly connected to the American system of popular representation in government, where "the link between ordinary citizens and their representatives is stretched so thin that it has almost disappeared" (Stivers, 2008, p. 85).
Although debate about the existence of the citizenship and democratic deficits continues (e.g., Stolle & Hooghe, 2004, p. 164), the financial deficits in the United States are very real and have had serious negative impacts on popular support for government.The Great Recession's shock to budgets, both public and personal, has not only soured moods and faith in government, but has also ushered in an age of austerity.Americans, struggling to hold onto or regain their pre-recession standards of living, are being asked to pay more taxes (although federal tax rates are still near historic lows), while simultaneously seeing cuts in services and benefits as a result of the sequester (Marr & Frentz, 2013;Weisman, 2013).
Together, these deficits reinforce and exacerbate one another in a circular causal process.Financial and economic issues erode already low levels of public trust and faith in government, which increases the public's inclination to withdraw from political activities.As more citizens withdraw, public preferences are less known and, therefore, less well reflected in public policy decisions.As more policy decisions poorly manifest public preferences, citizens further distrust government.This cycle is aggravated by the media and politicians.Specifically, the media has shifted the frequency, substance, and tenor of content on government and governance, particularly in relation to economic issues.Stories tend to be more negative and emotionally charged, and issues are covered in such a way as to block and distort intellectual and rational thinking.As a result, "factoids get confused with facts, 'affect' (or emotion) drives out 'intellect' in evaluating news, and a confused public paradoxically presses further demands for redress upon a federal government it perceives as ineffectual" (Durant, 1995, p. 28).Likewise, "risk-averse, resource-dependent, and media-conscious politicians" see "scant rewards in a trusteeship model of public service" and tend to communicate in sound bites rather than in ways that increase public understanding, policy discourse, and civic debate (Durant, 1995, p. 29).
Together, these trends and deficits may threaten the overall stability and health of the American political system.They prevent meaningful public discourse about political, economic, and other public issues and foster a governance structure that is too hollow in capacity to nurture (let alone achieve) policy goals.It is within this context that Jefferson Action, a small non-profit in Minnesota, decided to take action by empanelling a representative group of voters in a Citizens Jury process called "Reclaim November Ohio" to co-produce materials for a contested Congressional election.

Reclaim November Ohio
The mission of Jefferson Action "is making the informed voice of everyday citizens heard -and making sure elected officials and politicians address the issues" (http://jeffersonaction.org/who-we-are/).To achieve its mission, Jefferson Action uses the Citizens Jury, a public deliberation process developed by its founder, Ned Crosby (Gastil & Richards, 2013).In a Citizens Jury project, a randomly selected and demographically representative "microcosm" of citizens is empanelled to carefully examine an issue of public significance.This Jury, usually consisting of 18 to 24 individuals who are compensated for their time, meets for a period of two to five days.They hear from a variety of expert witnesses and deliberate together on the issue, with the assistance of one or more trained facilitators.At the conclusion of the deliberative process, members of the Citizens Jury present their recommendations to decision-makers and the public.
In 2012, Jefferson Action decided to employ the Citizens Jury approach to address economic issues in a divisive congressional election in Ohio's 16 th District, a sprawling, newly created district that roughly centers on the city of Akron.According to Jefferson Action, the location was appealing for several reasons.First, it has diverse demographics and includes both urban and rural populations.Second, Ohio was hit hard by the U.S. economic crisis and played a unique role as the key swing state in the 2012 presidential election, in part because neither the Democratic nor Republican political party had managed to dominate the state.Finally, and most importantly, two incumbents were pitted against each other in this district.Republican Jim Renacci and Democrat Betty Sutton had both been elected two-years earlier, only to see their districts merged into one.Thus, the race featured two successful candidates, one from each party, who now needed to cement the advantages of their incumbency in a new district.
Once the location was set, Jefferson Action designed a process called "Reclaim November Ohio" (RNO), which consisted of two Citizens Juries, dubbed "Citizen Election Forums" (CEFs) for the event.Jefferson Action then commissioned a random-sample telephone survey to create benchmarks on the district population's race, socio-economic status, and political orientation.To recruit participants, Jefferson Action staff mailed an informational letter and brief questionnaire to a random sample of 30,000 voters registered in Ohio's 16th Congressional District.Voters who were interested and potentially available to participate in the RNO project responded with their contact information and completed questionnaire, which contained both demographic and social/political information.These responses were then made anonymous, and used to randomly populate the RNO participant panels to ensure that a microcosm of the district was represented in each CEF phase.Jefferson Action took extra steps to recruit additional participants from the Cleveland area who were underrepresented in the initial voter registration list provided by the Ohio Secretary of State.
This complex recruitment strategy was meant to balance the desire to give everyone in the population an equal chance to participate with the need to have demographic representativeness.Potential jurists were contacted, given an overview of the process, including details about their daily stipend ($150) and travel reimbursement.Through this procedure, Jefferson Action identified 54 participants, enough to empanel two Juries, each consisting of 27 people (24 jurists and 3 alternates). 1In an interview with the research team, Jefferson Action founder Ned Crosby said that the purpose of having two Citizens Juries was to (1) decrease the workload and time commitment for jurists, (2) expand the total number of jurists exposed to the process, and (3) increase outside perceptions of process legitimacy.
The first Citizens Jury participated in CEF 1, a weekend-long (3-day) event, where members were tasked with setting the agenda for the subsequent Jury.Specifically, prior to the event, Jefferson Action had identified 10 issues broadly related to the economy. 2The Jury was presented with expert testimony (from a politically balanced group of experts drawn mostly from Ohio colleges, universities, policy organizations, and think tanks) on each issue.The Jury was then led through a deliberative process until its members came to broad agreement on the three areas they felt should be addressed in the subsequent Jury: (1) the federal deficit and debt, (2) unemployment, and (3) weak economic growth.Finally, they collaboratively wrote a "Statement to the Candidates," which was released to the public and distributed through various media (for more on this project and links to the Statement, see: http://jeffersonaction.org/what-wedo/current-projects/).
The second Citizens Jury, which consisted of an entirely new panel, participated in CEF 2 (a 4-day process) and CEF 3 (a 3-day process).During CEF 2, jurists heard expert testimony on the three policy areas identified by the first Jury during CEF 1.In addition, they were taught about and given basic policy analysis tools to help them evaluate candidate positions.During CEF 3, the same jurists (minus 1, who did not return) started by reviewing what they had learned the previous weekend.They then formulated questions for the congressional candidates and interviewed both the candidates and their campaign staff about their positions on, and plans for, addressing the three economic issues.They finished their work by evaluating the candidates' positions on the issues and collaboratively writing a "Voters Guide," which was released to the public and distributed through various media (see: http://jeffersonaction.org/wp-content/uploads/Combined-Voter-and-Issue-Guide.pdf).A general overview of the RNO process is provided in Table 1.

Methods
As noted earlier, we assert that public deliberation and co-production in the political and electoral arena can have meaningful and positive impacts.Specifically, we propose that such processes can (1) give agency and voice to citizens, thereby improving their perceptions of politics and government; (2) help inform citizens about policy issues; and (3) help candidates and elected officials better understand citizens' needs, interests, concerns, and values.
We use data from pre-and post-test surveys to determine whether these outcomes were present in the Reclaim November Ohio process.Specifically, researchers administered surveys to all participants before and after each of the three CEFs, for total of six surveys.The pre-and post-surveys were linked so that we could examine both individual-level and group-level changes.The surveys asked questions about numerous issues, not all of which are covered here.Rather, here we focus only on the survey data that pertains to our general propositions about the usefulness of public deliberation and co-production in the political and electoral arena.Specifically, to determine whether deliberation and co-production gives agency and voice to citizens and improves their perceptions of politics and government, we examine trust in government, perceptions about the influence of the U.S. Congress, and levels of political efficacy.To determine whether deliberation and co-production informs citizens about policy issues, we examine participants' levels of interest and perceived and actual knowledge about the issues under discussion in the CEFs.Finally, to determine whether participants believe that deliberation and co-production can help officials better understand their needs, interests, concerns, and values, we examine participants' perceptions about the influence of the RNO event on the candidates.These general propositions and measures are shown in Table 2. Before presenting our results for these analyses, we briefly discuss the demographics of the two Citizens Juries.We conclude the results section with a discussion of participant satisfaction.

Results
Before examining the results, it is useful to briefly remark on how we present findings.Some reported results use data pertaining to a specific Citizens Jury and/or CEF, whereas other results aggregate data from across both Citizens Juries and/or all three CEFs.As noted above, the first Citizens Jury (Jury 1) participated in CEF 1.The second Citizens Jury (Jury 2) participated in CEF 2 and CEF 3. Since the same group of people participated in CEF 2 and CEF 3, we report aggregate results from these forums under the heading Jury 2. With that in mind, we present several sets of results, including: (1) demographics, (2) perceptions of politics and government, (3) informing citizens, (4) perceptions of effects on public officials, and (5) participant satisfaction.

Demographics
Table 3 presents the demographic characteristics of Jury 1 and Jury 2. The participants in the two Juries closely mirrored the district, and were similarly diverse in terms of gender, race, age, and educational attainment.While the employment status within each Jury was diverse, there were some differences.For example, Jury 1 had nine retirees and three students, whereas Jury 2 had seven retirees and zero students.Moreover, Jury 2 had nine participants who worked full-time, whereas Jury 1 had only six participants who worked full-time.There was also variation in income levels within each Jury.Jury 1 had four participants who earned less than $10,000 a year, whereas Jury 2 had zero participants at this income level.
The participants in Jury 1 and Jury 2 were also diverse, but similar, in terms of political ideology.Each jury had eleven conservatives and five liberals.Jury 1 had seven moderates and Jury 2 had eight.The political ideology question was based on a 5-point Likert scale, where 1 = very conservative and 5 = very liberal.The mean for Jury 1 was 2.74, and the mean for Jury 2 was 2.71, suggesting that both juries leaned toward conservative views overall.It is important to note that the political views of the participants mirrored the political views of the average American at the time.A 2011 USA Today/Gallup poll asked Americans to rate their own political ideology on a 5-point scale with 1 being very liberal and 5 being very conservative (note the inversion of the scale).Americans' mean score on this scale is 3.3, indicating that the average American is slightly to the right of center ideologically, as were the participants in the two juries.

Agency, Voice, and Perceptions of Politics and Government
To determine whether public deliberation and co-production give agency and voice to citizens and improve their perceptions of politics and government, we examine: (1) trust in government, (2) perceptions of congressional influence, and (3) political efficacy.
Trust in Government.Using the standard question and response range (1 = just about never to 4 = just about always), we asked jurists, "How much of the time can you trust government to do what is right?"Our analyses showed statistically significant increases in trust in government in Jury 1 and Jury 2. In Jury 1, the mean response increased from 1.73 in the pre-survey to 1.91 in the post survey (p = .05),and in Jury 2 the mean response increased from 2.05 in the CEF 2 pre-survey to 2.41 in the CEF 3 post-survey (p = .05).Despite the statistically significant increases, it is important to note that these mean responses still indicate a low level of trust in government overall.
Congressional Influence.We asked participants about the ability of the U.S. Congress to affect what happens in the nation and in one's personal life.Responses to these questions were based on a 5-point Likert scale, where 1 = not at all to 5 = a great deal.Results are displayed in Table 4.For Jury 1, we used data from the pre-and post-surveys of CEF 1.The results indicate that after participation, Jury 1's perceptions about the ability of the U.S. Congress to affect both the nation and one's personal life increased in a statistically significant way.For Jury 2, we used data from the pre-survey of CEF 2 and the post-survey of CEF 3. The results indicate that that after participation, Jury 2's perceptions about the ability of the U.S. Congress to affect the nation increased in a statistically significant way; however, perceptions about its ability to affect their personal lives decreased slightly, but not significantly.To us, these results suggest that the extended participation of jurists in CEF 2/3, and/or the particular format of those forums, allowed for a more thorough and complete understanding of the powers and responsibilities of legislative branches of government, as well how actions by those branches can affect personal lives and national agendas.Political Efficacy.We asked several questions to measure internal and external political efficacy.We drew from the standard questions and included several additional questions.This is not unusual; the scales used to operationalize internal and external political efficacy vary considerably in the literature.However, analyses showed that the summative indices for these questions had alphas below standard acceptable levels.Again, this is not surprising, as research shows that scales for internal and external political efficacy are subject to measurement error and thus lack validity and reliability (for discussions of political efficacy scales, see Craig, Niemi, & Silver, 1990;Morrell, 1999Morrell, , 2003Morrell, , 2005;;Beaumont, 2011).Given the weaknesses of the summative indices, we simply summarize the results for the various items measuring perceptions of internal and political efficacy.
Internal political efficacy (IPE) refers to one's feelings of personal competence to understand and participate effectively in politics; it represents beliefs about the impact one can have on politics and the political process (Nabatchi, 2010b).On all of the pre-and post-surveys, we asked a series of five questions to measure IPE.Each used a 5-point Likert scale, where 1 = strongly disagree and 5 = strongly agree.The results are presented in Table 5.Specifically, the Table reports changes for Jury 1 (using data from the pre-and post-surveys for CEF 1), along with changes for Jury 2 (using data from the pre-CEF 2 and post-CEF 3 surveys).Several specific results are worth noting.First, no indicators were significant for both Jury 1 and Jury 2. Second, one indicator (item a) was statistically significant for Jury 1, but not for Jury 2, suggesting that participation in CEF 1 increased jurists' perceptions about their ability to understand politics and government.This result may be a function of the design of the first CEF, which was dedicated to learning about and understanding economic issues.Third, one indicator (item d) was statistically significant for Jury 2, and approached significance for Jury 1, suggesting that participation tended to increase jurists' confidence in their ability to discuss politics and government.Finally, four indicators (items b, c, d, and e) were significant for Jury 2. Together, these results suggest that participation in Jury 2 had a greater overall effect on perceptions of IPE than did participation in Jury 1.This is not surprising.Research indicates that internal political efficacy is the product of numerous forces, including political socialization, familial, social, and educational forces, personal experiences, and other factors (Almond & Verba, 1963;Bandura, 1994;Nabatchi, 2010b;Pincock, 2012).Thus, it is makes sense that the shorter experience in Jury 1 had comparatively less effect on participants' perceptions of IPE, than the longer experience of Jury 2. External political efficacy (EPE) refers to one's perceptions about the responsiveness of the political system (both governmental authorities and institutions) to citizen demands; it is the belief that the political system is receptive to the interests and actions of citizens (Nabatchi, 2010b).We asked a series of five standard questions to measure EPE.All questions were on a 5-point Likert scale, where 1 = strongly disagree and 5 = strongly agree.The results are presented in Table 6, which shows the changes for Jury 1 (using data from the pre-and post-surveys for CEF 1), changes for Jury 2 (using data from the pre-CEF 2 and post-CEF 3 surveys).Several results are interesting.First, there were changes in the means of all five indicators, and these changes all indicated increases in EPE, but not all were statistically significant.Second, increases for two indicators (items a and b) were statistically significant.This suggests that after participation, jurists were more likely to believe that had a say in government (item a) and that elected officials would care what they thought (item b).Third, one indicator (item e) was statistically significant for Jury 2, and approached significance for Jury 1, suggesting that after participation jurists were more inclined to believe that their participation made a difference in the outcomes of the RNO events.Finally, two indicators (items c and d) were not statistically significant for either of the Juries.Together, these results suggest that the forums had an effect (albeit limited) on participants' perceptions of external political efficacy.
Together, these results for trust, congressional influence, and political efficacy suggest that public deliberation and co-production in the political and electoral arena can have positive impacts on citizens' agency, voice, and perceptions of politics and government.Specifically, the results show that participation in these CEFs increased perceptions of trust in government in a statistically significant way.Also, for CEF 2/3, participants in Jury 2 seem to have generated more realistic understanding about the influence of the Congress on the nation.Finally, the analyses show that participation in the CEFs changed perceptions of political efficacy.Specifically, participation resulted in statistically significant increases for some, but not all, of the indicators of IPE.In general, these effects were greater for Jury 2 than for Jury 1, most likely because of the longer duration of participation.The results also show that the indicators of EPE all changed in the hoped for direction; however, only two indicators changed in a statistically significant way.In general, these results buttress the research on public deliberation showing that it can have educative effects on the individuals who participate (for reviews of this literature see Delli Carpini, Cook, & Jacobs, 2004;Nabatchi, 2010b;Pincock, 2012).

Informing Citizens
To determine whether public deliberation and co-production informs citizens about policy issues, we examined participants' levels of interest and perceived and actual knowledge about the issues under discussion in the CEFs.For CEF 1, we solicited participants' level of interest and perceived knowledge on broad economic issues, as well as on the ten specific economic issues under discussion: (1) the cost of a college education; (2) the cost of healthcare; (3) the federal budget deficit and debt; (4) the financial performance of Americans' savings and retirement investments; (5) government regulations on private enterprise; (6) home values, foreclosures, and mortgages; (7) living standards for the poorest Americans; (8) too much wealth controlled by too few Americans; (9) unemployment; and (10) weak economic growth.The results from CEF 1 show that participants' interest in broad economic issues and the costs of a college education increased slightly.The mean level of interest in the cost of health care, the federal budget deficit and debt, retirement savings and investments, housing issues, income disparities, and unemployment all fell slightly.Interest in government regulations, the living standards of the poorest Americans, and in economic growth remained unchanged.The only statistically significant changes were the declines in interest for retirement savings, housing, and unemployment.Participants perceived that they gained knowledge on all issues discussed during CEF 1; however, only the increase in perceived knowledge about income inequality was statistically significant.
For all three CEFs, we examined participants' interest and perceived knowledge on the three specific issues under discussion, including: (1) the federal budget deficit and debt, (2) weak economic growth, and (3) unemployment. 3Finally, we examined actual knowledge before and after CEF 2 with a series of multiple choice questions about the three economic issues.
To examine changes in participant interest and perceived knowledge, we used a summative index for the three specific issues.The questions for issue interest used a 5-point Likert scale with the anchors of 1 = not at all interested and 5 = very interested.The questions for perceived issue knowledge used a 5-point Likert scale with the anchors of 1 = know nothing and 5 = know a lot.To create the indices, we grouped items into the relevant index based on face validity.We then calculated the average for each participant's responses to the questions, and calculated the overall average of all participants.Finally, we tested the psychometric adequacy of each index with Cronbach's alpha (α).The issue interest index has an alpha of .694,which is just below the acceptable level of reliability, while the perceived knowledge index has an alpha of .863,indicating a good level of reliability.Table 8 reports the number of items used in each index, the index alpha, and the pre-and post-survey means for each CEF.
In terms of issue interest, there are two important results.First, participants had a very high level of interest in the issues going into each forum, and after each forum, their interest fell slightly but not in a statistically significant way.For CEF 1, mean interest fell from 4.67 to 4.55; for CEF 2, it fell from 4.68 to 4.64; and for CEF 3, it fell from 4.84 to 4.79.It is important to note that even though issue interest dropped in all the forums, the means for issue interest were still very high -all were above 4.5 on a 5-point scale.Second, and perhaps more importantly, the mean issue interest among members of Jury 2 increased from the CEF 2 presurvey (4.68) to the CEF 3 post-survey (4.79).Although this increase is not statistically significant, it does suggest that Jury 2 developed a deeper interest in the issues, perhaps as a result of their extended participation.The results for perceived knowledge are quite different.First, at the start of CEF 1 and CEF 2, participants reported having a knowledge base that was only slightly above average (3.84 and 3.72, respectively).Second, for all three CEFs, participants reported gaining knowledge.For CEF 1, mean perceived knowledge increased from 3.84 to 3.97; however, this increase is not statistically significant.This result is not surprising given that the overall results from CEF 1 showed statistically significant gains in only one issue: income inequality.That said, the reported knowledge gains for CEF 2 and CEF 3 were statistically significant.For CEF 2, mean perceived knowledge increased from 3.72 to 4.39 (p = .001),and for CEF 3, mean perceived knowledge increased from 4.33 to 4.61 (p = .043).These results suggest that Jury 2 believed that they learned about these issues as a result of their participation.Finally, the results show that mean perceived knowledge also increased in a statistically significant way, from 3.72 in the CEF 2 pre-survey to 4.61 in the CEF 3 post-survey (p = .001).This suggests that Jury 2 believed they gained important and significant knowledge about the issues over the course of the two CEFs.
In addition to perceived knowledge, we also attempted to measure actual knowledge gains among Jury 2 participants.Specifically, we worked with Jefferson Action to develop six multiple-choice questions that measured participants' knowledge on the three economic issues under discussion.Appendix One lists the questions to measure participant knowledge.The results are worth mentioning briefly here.First, the results show both increases and decreases to the overall number of correct answers to the questions, with no changes being statistically significant.Second, when looking at the individual results, some people went from having incorrect answers on the pre-survey to having correct answers on the post-survey, while others went from correct answers on the presurvey and to incorrect answers on the post-survey.These odd findings may be the result of people guessing on both the pre-and post-surveys, or of people becoming confused on the specific knowledge points presented to them during the weekend event.Regardless, because of the mixed results, we do not believe the questions provided suitable measures of actual knowledge.
In all, these results about informing citizens suggest that public deliberation and co-production in the political and electoral arena may not affect citizens' interest in policy issues, but that it may affect their perceived, if not actual, knowledge about issues.Specifically, levels of interest in the issues tended to decline after each CEF, though no declines were statistically significant.Moreover, at the start of CEF 1 and CEF 2, participants reported having a knowledge base that was only slightly above average (3.84 and 3.72, respectively).Participants also reported statistically significant knowledge gains for all three forums.Unfortunately, we believe the questions measuring actual knowledge gains were inadequate, and do not have faith in the validity of those results.Once again, these results buttress empirical research on the effects of deliberation on individuals (see or reviews of this literature see Delli Carpini, Cook, & Jacobs, 2004;Pincock, 2012).

Perceptions of Public Officials' Understanding
We examine participants' perceptions about the influence of the RNO event on the candidates to determine whether they believe that such processes can help officials better understand their needs, interests, concerns, and values.Specifically, we asked three 5-point Likert scale questions in both the pre-and post-surveys: 1.How much impact do you think this weekend's Reclaim November Ohio event will have/had on how your congressional candidates think about and deal with economic issues?(No impact, A little impact, Some impact, A moderate impact, A strong impact) 2. To what extend to you agree that congressional candidates will be responsive to the recommendations of participants in this weekend's Reclaim November Ohio event?(Strongly disagree, Disagree, Neither agree nor disagree, Agree, Strongly agree).3. To what extend do you agree that this weekend's Reclaim November Ohio event will help/helped the congressional candidates better understand the economic concerns of average citizens?(Strongly disagree, Disagree, Neither agree nor disagree, Agree, Strongly agree).
Here, we analyze participant perceptions of event influence using a summative index of these three questions. 4With an alpha of .806,this index has good reliability.Specifically, we examine the pre-and post-survey indices for each of the CEFs, as well as for the overall RNO event.Table 7 shows the results from these analyses.
Three sets of findings from this analysis are important to note.First, for all CEFs, as well as for the RNO event as a whole, the mean scores for the event influence index range from a low of 3.28 (in the post-survey for CEF 2) to a high of 3.88 (in the post-survey for CEF 1).These findings suggest that participants were slightly, but not highly, optimistic about the potential of the forums to influence their congressional candidates.
Second, while there was an increase in perceived influence for CEF 1, the change was not significant.Likewise, there was no change in perceived influence for CEF 3 (the pre-and post-test means are identical).However, there was a statistically significant change in perceived event influence for CEF 2, with an increase from 3.28 in the pre-survey to 3.89 in the post-survey (p = .012).This increase suggests that CEF 2 participants believed the forum would influence the way candidates thought about, approached, and responded to citizen concerns about economic issues.It is possible that the inconsistent results for perceived influence are a function of the goals and tasks presented to participants in each forum.
Finally, an analysis of the cumulative average of the pre-and post-test means for CEF 1, 2, and 3 show a statistically significant change in the perceived influence of the event on congressional candidates.Specifically, the overall average mean of the index increased from 3.59 in the pre-tests to 3.86 in the post-tests (p = .026).This suggests that while participants may have had less faith in the potential influence of individual forums, the overall RNO event was seen as better enabling candidates to gain a sense of the issues that matter to constituents.While the results vary among the three CEFs, the cumulative average of the preand post-test means suggest that participants believed the RNO event as a whole would influence the congressional candidates.While not conclusive, the results do suggest that public deliberation and co-production can, at the very least, influence citizens perceptions about whether public officials have a sense of what matters most to constituents.More research is needed to determine whether public officials actually better understand the needs, interests, concerns, and values of constituents after a co-production exercise.

Participant Satisfaction
Finally, we believe it is important to look at one last set of measures -participant satisfaction -even though these are not directly related to the theorized benefits of public deliberation and co-production in the political and electoral arena.Here, we examine six summative indices of participant satisfaction. 1Appendix Two lists the questions used for each index.Table 9 reports the number of questions used in each index, the index alpha, the index means for each CEF and the overall RNO event.
The results show a high and consistent level of satisfaction with the process, outcomes, moderators, background information, discussions, and staff for each individual CEF, and consequently for the RNO event as a whole.For all indices, the difference between the highest and lowest satisfaction scores among the CEFs is less than four-tenths of a point.Moreover, while participants reported high levels of satisfaction on every index, the overall results for RNO show that they were most satisfied with the moderators (4.85), discussions (4.76), and staff (4.74).This suggests that participants found particular value in the facilitated deliberations, and were very pleased with the overall management and coordination of the forums.

Conclusion
In sum, our analyses about the Reclaim November Ohio event reveal several important findings.First, the forums were demographically diverse, and seem to mirror the general makeup of Ohio's 16 th district.Second, we found significant increases in trust in government, although overall levels of trust were still low.Third, we found that the CEFs, in general, increased perceptions that Congress had a significant impact on the country, but decreased perceptions of Congress's impact on participants' own lives.Fourth, although the results for political efficacy were mixed, they generally show improvements in such perceptions as a result of participation.Fifth, although the results for actual knowledge were unclear, participants reported gaining knowledge about the federal deficit and debt, economic growth, and unemployment.Sixth, the participants were slightly optimistic that Reclaim November Ohio would positively influence their congressional candidates.Finally, participants were extremely satisfied with each CEF and the RNO event as a whole.Together these results provide evidence that public deliberation and co-production in the political and electoral arena can (1) improve participants' sense of voice and agency, as well as their perceptions of politics and government; (2) increase participants' levels of perceived knowledge about policy issues, and (3) shape participants' perceptions about how well officials understand their interests and concerns.
These results are in line with the general results of empirical research showing that public deliberation can have educative and civic effects on citizens (Delli Carpini, Cook, & Jacobs, 2004;Pincock, 2012).Thus, this article adds to the growing evidence about the benefits of public deliberation and lays the groundwork for examining co-production techniques in the political and electoral arena.Although this is a rather novel setting for the exploration of co-production and more research is needed, our findings lend further support to the mounting evidence that thoughtfully designed deliberation processes can increase participants' perceptions of political efficacy while contributing meaningfully in the co-production of an important service, in this case election and voting materials.

Table 1 : Overview of the Reclaim November Ohio Process
Jefferson Action also contracted researchers from the Program for the Advancement of Research on Conflict and Collaboration (PARCC) at the Syracuse University Maxwell School of Citizenship and Public Affairs to conduct an evaluation of the RNO process.While the research team evaluated several aspects of RNO, this article focuses on assessing the Citizens Jury within the context of co-production in the political and electoral arena.