A Survey Experiment on Citizens’ Preferences for ‘Vote–Centric’ vs. ‘Talk–Centric’ Democratic Innovations with Advisory vs. Binding Outcomes

Previous research on public support for participatory decision-making fails to distinguish between vote-centric (referendums and initiatives) and talk-centric (deliberative-style meetings) instruments, despite a deliberative turn in democratic theory suggesting that political discussion among ordinary citizens improves decision-making. In an online factorial survey experiment conducted among a sample of 960 Americans recruited on Amazon’s Mechanical Turk, we compared support for the use of referendums and public meetings, arguing that attitudes towards these instruments depend on whether they are used to inform legislators or take binding decisions. Public meetings were rated considerably lower than referendums and initiatives, especially when the outcomes were binding. Contrary to expectations, we did not find a preference for binding (over advisory) referendums and individuals from referendum and initiative states, where these instruments are legally binding, expressed less support for binding participatory reforms than individuals from non-direct democratic states. Despite the many critiques of direct democracy, public debate in the US has not considered whether advisory outcomes might appease some of these concerns. The results also demonstrated that individuals expressing concerns about the inability of ordinary citizens to understand politics and about the welfare of minority groups were not as negative about participatory decision-making when legislators had the final say.


Introduction
Several studies in established democracies document broad popular support for allowing citizens a greater role in political decisions (Bengtsson & Mattila, 2009;Bowler, Donovan, & Karp, 2007;Craig, Kreppel, & Kane, 2001;Dalton, Burklin, & Drummond, 2001;Donovan & Karp, 2006). Data from cross-national surveys such as the European Social Survey (Round 6, 2012) and the International Social Survey Program (ISSP Research Group, 2016) demonstrate that 70-80% of respondents agree with the use of referendums for making political decisions. How-ever, previous research on public support for participatory decision-making fails to distinguish between 'votecentric' and 'talk-centric' instruments-despite a 'deliberative turn' in democratic theory-and between consultative and legally binding outcomes. These are important distinctions because they tell us more about how citizens want to be involved in political decision-making and whether they want to be in control of policymaking, or are satisfied simply being heard.
Participatory reforms or democratic innovations are 'instruments designed to increase citizen participation in the political decision-making process' (Smith, 2009, p. 1). Whereas some of these instruments are more 'vote-centric' e.g., referendums and initiatives, others are more 'talk-centric' e.g., mini-publics and deliberative polls (LeDuc, 2015). Normative political theorists have urged more deliberation to improve decision-making processes (Chambers, 2012;Fishkin, 1991;Gutmann & Thompson, 1996;Habermas, 1996), however previous research on citizens' perceptions measures only support for referendums or for the general idea of 'giving citizens more opportunities to participate in political decisionmaking' (Bengtsson & Mattila, 2009;Bowler et al., 2007;Craig et al., 2001;Dalton et al., 2001;Donovan & Karp, 2006). By means of a factorial survey experiment where respondents are randomly assigned vignettes about referendums and initiatives or about 'public meetings where citizens collectively discuss political issues', we investigate whether people are more favorable towards votecentric or talk-centric decision-making processes.
Public support for these instruments may depend on how much control citizens have over policy outcomes. In recent years, politicians in New Zealand (Karp & Aimer, 2002), the Netherlands (Qvortrup, 2018), and the UK (Merrick, 2016) have debated whether the outcomes of referendums should be respected or subject to parliamentary approval. Talk-centric instruments generally do not constitute a formal part of the decision-making process, although some normative theorists have argued for 'empowered' deliberative forums (Fung, 2007;Pateman, 2012;Setälä, 2011). Therefore, with our factorial survey experiment, we also test whether support for votecentric and talk-centric instruments depends on whether they are used to inform policymakers or to take binding decisions.
Finally, we develop several hypotheses about the skeptics of participatory reforms, testing whether their attitudes towards these instruments are 'less negative' when the outcomes are advisory as opposed to binding. Critics of referendums and initiatives express concerns about the inability of ordinary citizens to understand the issues on the ballot and about the harmful consequences of majoritarian democracy for minorities (Bowler & Donovan, 2000;Carpini & Keeter, 1996;Gamble, 1997;Gerber & Hug, 2001;Haider-Markel, Querze, & Lindaman, 2007;Kang, 2002;Lewis, 2013;Magleby, 1984). Surprisingly, public debate in the US, where binding referendums are commonly used at the state level, has not more carefully considered whether advisory outcomes might alleviate some of these concerns.
The experiment was conducted among a sample of 960 US respondents recruited on Amazon's Mechanical Turk (MTurk). Due to the variation in the use of direct democratic instruments across states, the US provides a particularly interesting case study. 26 out of 51 states provide for referendums and/or initiatives that are in most cases legally binding (Cronin, 1999, p. 176). Nonetheless, Americans remain deeply divided over the use of direct democratic instruments and their consequences for democracy (Cronin, 1999;Haskell, 2018;Lewis, 2013). Some institutions such as the National Conference of State Legislatures (NCSL, 2002) recommend switching to advisory procedures and several states have flirted with the idea of advisory referendums in the past (Schaffner, 1907).

Vote-Centric Instruments
In the 26 American states providing for direct democracy, referendums and initiatives are generally binding, although some states only allow indirect initiatives where successful petitions go to the legislature before going on the ballot (NCSL, 2018). This stands in contrast to the European experience where referendums are usually government-initiated and advisory (except Switzerland and Italy) and where citizens can vote on national issues (Setälä, 2006). Despite the many criticisms of referendums and initiatives, public debate around the possibility of advisory measures is lacking in American states. In the early 20th century, advisory referendums and initiatives were adopted in some cities but the idea never took off nationally. Several Congressmen promoted an advisory referendum on national issues, which received considerable public support in a Gallup Survey (Cronin, 1999).
Political scientist David Magleby has advocated greater use of advisory instruments in the US mainly because current referendum and initiative (R&I) procedures are too complicated. The process would run more smoothly if, instead of requiring citizens to propose actual legislation, legislatures polled citizens about their general policy preferences (Magleby, 1984, p. 195). Advisory referendums can signal a lack of consensus on an issue whereas a binding initiative is enacted by as slim a majority as 50%. Advisory referendums are also more flexible because they allow for proposed legislation to be improved or amended after the vote (Cronin, 1999, p. 178). Studies on the indirect effects of referendums and initiatives suggest that proposals can have an impact on politics even if they do not make it to a popular vote or are not approved by a majority of voters (Gerber & Hug, 2001).
Although there is somewhat of a scholarly debate around the amount of control referendums and initiatives have over policy outcomes, the question of whether citizens want to play a decisive role in policymaking seems to be overlooked. A preference for advisory outcomes implies that citizens trust legislators to make the right decisions, provided they have been sufficiently informed, whereas a preference for binding outcomes suggests that citizens desire more opportunities to keep legislators in check (Bowler et al., 2007).

Talk-Centric Instruments
Vote-centric instruments are more commonly used than talk-centric instruments, however deliberative democrats are critical of the unreflective, aggregative, and majoritarian qualities of referendums (Chambers, 2001;Setälä, 2011). Deliberative democracy identifies political discussion among citizens, who must be prepared to defend their views with reasoned argument, as the key component of political decision-making (Dryzek, 2007). Examples of participatory instruments emphasizing discussion are citizens' juries, mini-publics and deliberative polls (Pateman, 2012).
Advocates of a discussion-based approach emphasize a range of positive outcomes: among others, participants learn important civic skills, consolidate their political views, empathize with opposing viewpoints, and set aside personal interests in the pursuit of common goals (Gutmann & Thompson, 1996;Habermas, 1996;Roberts, 2004). Critics argue that talk-centric instruments make unrealistic demand on ordinary citizens who lack interest and knowledge of politics (Hibbing & Theiss-Morse, 2002;Mutz, 2006;Warren, 1996). Others express concerns about inequalities in deliberation due to diverging communication styles and the status effects of sex, race, culture, and ethnicity (Karpowitz & Mendelberg, 2014;Sanders, 1997). Nonetheless, there seems to be a growing consensus among democratic theorists that deliberation enhances democracy (Pateman, 2012).
Talk-centric instruments are very loosely connected to policymaking and seldom have more than advisory force (Setälä, 2011). Some deliberative theorists promote the idea of 'empowered deliberation', arguing that when citizens see a real connection between participation and outcomes, they are more willing to participate and take their role seriously (Fung, 2007;Johnson & Gastil, 2015;Pateman, 2012;Smith, 2009). Others contend that talk-centric instruments cannot perform a decision-making function: these instruments are primarily 'schools of democracy' where citizens acquire important civic skills (Fishkin, 1991); higher stakes privilege emotions over rational thought and inhibit the potential for compromise (Mansbridge, 2007); and decisions require that participants reach a consensus which, according to Warren (2017), is neither feasible nor desirable in deliberative settings.

Citizens' Support for Advisory vs. Binding Talk-Centric and Vote-Centric Instruments
We argue that ordinary citizens do not follow the normative debate about deliberative democracy and are therefore not familiar with the reasons why this approach to political decision-making might produce better solutions. Talk-centric instruments are not commonly used for informing or making political decisions and are likely to be perceived as chaotic, inefficient, and less inclusive, especially in comparison to referendums and initiatives. Therefore, we hypothesize that citizens express greater support for vote-centric than talk-centric decision-making instruments (H1).
Public meetings are less inclusive than referendums because only a small subsection of the public can par-ticipate, hence the decisions made in these meetings are likely to be perceived as less representative of what the general public wants than the decisions made by referendums (Parkinson, 2003). Deliberations produce outcomes that are considered legitimate by the body that deliberated, but are not necessarily legitimate as binding decisions over a wider democratic polity (Johnson & Gastil, 2015). Therefore, we hypothesize that advisory outcomes are preferred for talk-centric instruments (H2a).
Voting is the most common deciding rule in contemporary democracies and the decisions are perceived as legitimate by citizens because they correspond to the views of the majority. The counting of votes reveals more about the strength of an opinion (Warren, 2017). Votecentric instruments are more inclusive than talk-centric instruments because every citizen can participate and they may be perceived as more democratic in the sense that every vote weighs equally, whereas equality of voice is difficult to achieve in deliberative settings where louder or 'privileged' voices tend to dominate (Sanders, 1997). Finally, considering claims of widespread dissatisfaction with representative institutions, especially in the US context (Cooper, 2018;Hibbing & Theiss-Morse, 2002), it seems logical to expect that citizens would want preferences upheld by the majority of the polity to be implemented and not just accounted for. Therefore, we hypothesize that binding outcomes are preferred for votecentric instruments (H2b).

Direct vs. Non-Direct Democratic States
The US offers a particularly interesting case study for this research because half of the states provide for referendums and initiatives. Surveys conducted in R&I states demonstrate that citizens tend to regard these instruments in a positive light (Bowler & Donovan, 2002;Craig et al., 2001;Cronin, 1999). Since South Dakota first adopted the initiative and referendum in 1898, no state has ever chosen to do away with them and states without are gradually adopting them at a rate of about one state per decade since the end of World War II (Matsusaka, 2005, p. 186). Furthermore, studies in the US (Bowler & Donovan, 2002) and Switzerland (Stadelmann-Steffen & Vatter, 2012) have demonstrated that exposure to referendums and initiatives is associated with more positive attitudes towards government. If these instruments contribute to greater satisfaction with government, then citizens who have access to referendums and initiatives are probably more positive about them than those who do not have access. Therefore, the previously hypothesized preference for vote-centric (over talk-centric) decisionmaking instruments is expected to be stronger for individuals living in R&I states than for individuals in non-direct democratic states (H3).
Finally, given that in the US the decisions made in R&I procedures are usually binding, switching to advisory decisions may be perceived by individuals in R&I states as a step back for democracy. By contrast, individuals resid-ing in non-direct democratic states might consider advisory instruments as a step forwards and therefore not differentiate as much between advisory and binding outcomes. We hypothesize that the positive effect of residing in a R&I state on support for participatory reforms is weaker when the decisions are advisory as opposed to binding (H4).

The Skeptics of Participatory Reforms
Several political scientists have criticized direct democracy on the grounds that citizens lack sufficient knowledge to make political decisions (Bowler & Donovan, 2000;Carpini & Keeter, 1996;Kang, 2002;Magleby, 1984) and that majority rule via referendums and initiatives contributes to the suppression of minority rights (Gamble, 1997;Gerber & Hug, 2001;Haider-Markel et al., 2007;Lewis, 2013). However, with the exception of Anderson and Goodyear-Grant (2010), who argue that these are the reasons why highly informed voters are skeptical of referendums, no studies have investigated whether these concerns influence ordinary citizens' attitudes towards participatory reforms.
Pollsters consistently demonstrate that citizens are not just ignorant about policy issues, but also about the basic structure of government and how it operates, for example a recent poll by the Annenberg Policy Centre found that only 36% of Americans could name the three branches of federal government (Somin, 2014). Majorities of voters in Arizona, California, Colorado, Oregon, and Washington agree that initiative and referendum measures are too complicated to understand (Cronin, 1999;Kang, 2002). Magleby's (1994) examination of voting materials from four states found that they required the reading level of a third-year college student. Studies demonstrate that citizens often cast votes supporting the opposite outcome from what they actually intended (Dubois & Feeney, 1998).
A second critique of direct democratic instruments is that they can provide a tool for majority action against unpopular minorities, as demonstrated by the Swiss and American experience with referendums and initiatives (Dalton et al., 2001, p. 151). Examples of anti-minority measures in the US include initiatives banning same-sex marriage, revoking affirmative action programs, and establishing the preferential treatment of persons from specific racial or ethnic backgrounds for employment or housing (Gamble, 1997). In recent years, initiatives repealing anti-discrimination laws for LGBT persons were submitted in several states, including California's infamous Proposition 8 attempting to reverse the highcourt's ruling in favor of same-sex marriage. In a Gallup Survey Poll, 32% of Americans agreed that 'if people were allowed to vote directly on important issues at the state and local levels, minority groups in the population would not get a fair say' (Cronin, 1999, p. 99).
These concerns about participatory decision-making could be mitigated if the instruments were restricted to informing decisions and legislators were allowed more discretion. For example, legislators would be able to critically examine proposals that received only limited media coverage, as this would imply limited public debate and public knowledge of the issue at hand, or proposals that infringe on civil rights and liberties. Therefore, we hypothesize that the negative effect of concerns about public incompetence on support for participatory decisionmaking is weaker when the outcomes are advisory as opposed to binding (H5) and the negative effect of concerns about minority rights on support for participatory decision-making is weaker when the outcomes are advisory as opposed to binding (H6). However, concerns about minority rights may apply more to vote-centric instruments than talk-centric instruments. Referendums contribute to fears of a 'tyrannical majority' because they promote an aggregative form of democracy where outcomes are determined by a contest of numbers (Setälä, 2011). Deliberations, on the other hand, encourage participants to consider a range of alternatives and perspectives (Warren, 2017). Therefore, we hypothesize that the negative effect of concern for minority rights on support for participatory decision-making is weaker (or absent) when the instrument is talk-centric as opposed to votecentric (H7).

Sample
The survey was programmed and administered on Qualtrics between December 2016 and February 2017. Respondents were recruited on MTurk, an online crowdsourcing website where people complete tasks in exchange for a small compensation (usually around 1-2 USD for surveys). In order to prevent ballot-box stuffing, the survey was programmed so that each respondent received a unique completion code that could only be entered once. 985 people participated in the survey, 25 of which did not complete the survey and were excluded from the final sample (N = 960). On average, respondents took 7.10 minutes to complete the survey and only 34 respondents took less than 3 minutes. Analyses with and without these 34 respondents yielded the same results and only very small differences in effect sizes, therefore we decided to keep them in the final sample.
Mturk is an increasingly popular recruitment tool among social scientists, however respondents from this platform are often younger, higher educated, and more liberal than the general population (Berinsky, Huber, & Lenz, 2012). In order to obtain a sample that was more similar to the general population in terms of age, education, and political ideology we made use of MTurk's 'premium qualifications'. This function enabled us to target and over-recruit respondents who were older, lower educated, and conservative. In the final sample, some groups are slightly under or overrepresented, but the deviations from the general population are not large (for descriptives comparing the sample to the general population see Table 1). As will be explained in the following section, representative samples are not crucial for factorial survey experiments where the emphasis is on interactions between respondent characteristics and the experimental manipulations.
Despite these concerns about sampling bias, several studies have found that respondents recruited on MTurk are relatively similar to those recruited for nationally representative surveys. For example, Huff and Tingley (2015) report similar voting patterns among respondents from an MTurk sample and the Cooperative Congressional Election Survey, concluding that 'MTurk could be an excellent means for exploring how experimental manipulations influence voting tendencies' (p. 4). Weinberg, Freese and McElhattan (2014) compare the results of an online factorial survey for a sample recruited from a population-based Internet panel and a sample recruited on MTurk, demonstrating that the results are similar and the data quality is higher for the MTurk sample.

Factorial Survey Experiment
Factorial survey experiments are commonly used in sociology, but less frequently used in research on political attitudes and behaviors (Wallander, 2009). Notable exceptions are Jasso and Opp (1997) and Neblo, Esterling, Kennedy, Lazer and Sokhey (2010). In a factorial survey experiment respondents rate hypothetical descriptions of a person or situation, commonly referred to as 'vignettes'. The vignettes include several theoretically relevant factors (or attributes) consisting of two or more values or 'levels', which are randomly allocated to respondents.
Factorial survey experiments combine the advantages of a survey with the advantages of an experiment. Whereas most standard survey research is correlational, these experiments allow for causal inference about the effects of the vignette factors (Auspurg & Hinz, 2015, p. 11). We can investigate with greater certainty how the different attributes of a situation influence respondents' evaluations of the situation (Auspurg & Hinz, 2015;Wallander, 2009). Incorporating the experiment in a survey with a large sample makes it possible to investigate whether the effects of the vignette factors differ for different respondents. Therefore, when testing interactions between the experimental factors and respondent characteristics, having sufficient variation in the respondent characteristics within the sample is more important than having a representative sample (Auspurg & Hinz, 2015, p. 61).
In our factorial survey experiment, respondents evaluated vignettes describing a situation where citizens are given more opportunities to influence political decisions. The vignettes were composed of five factors, each of which has two levels: 1) administrative scale; 2) instrument of decision-making (vote-centric vs. talk-centric); 3) topic of discussion; 4) source of information; and 5) outcome (advisory vs. binding). For this study, only the effects of the instrument and outcome factors were analyzed. The factors in the story are not correlated because the levels were randomly distributed, therefore it is not necessary to control for the other three factors in the story. Indeed, we found that when the other factors were included in the models our results were not affected. Below is the vignette text with all possible factor levels (the factors that are analyzed in this study are italicized): Imagine that citizens in your [town or city/state] were given more opportunities to influence political decisions by [voting directly on issues in referenda and initiatives/participating in public meetings where randomly selected citizens are invited to discuss issues and collectively make decisions]. The citizens will be able to make decisions on all kinds of topics [including sensitive topics/except for sensitive topics], such as civil rights. To ensure that the citizens make well informed decisions [an independent commission of experts will provide information on each issue/each political party is responsible for communicating its position on the issue to the public]. Any decision the citizens make will be [advisory, meaning the local/state government can choose whether to carry it out/binding, meaning the local/state government must carry it out].
The variation of these levels produces a total of 2 5 = 32 possible combinations, or experimental conditions. In order to increase the number of ratings per vignette, but also because we wanted respondents to compare vignettes, each respondent was randomly assigned two vignettes. Each of the 32 experimental conditions was rated by an average of 60 respondents (range: 54-66). The two vignettes were presented in random order on the same screen to facilitate comparison.
In order to account for the nested structure of the data, i.e. vignettes clustered in respondents, the data were analyzed with multilevel modeling techniques, as is recommended for factorial survey designs (Auspurg & Hinz, 2015;Hox, Kreft, & Hermkens, 1991). The state was not included as a third level because an empty threelevel model revealed that approximately 0% of the variation in support for participatory reforms was at the state level (including the state level also did not affect the results). To test our hypotheses about the difference in the effect of advisory vs. binding decisions on support for vote-centric and talk-centric instruments (H2a and H2b), we included an interaction between the outcome and instrument factors. To test our hypotheses about how the vignette factors and respondent characteristics interact (H3-H7) we included cross-level interactions between the respondent-level variables and the vignette-level variables (the factors). All analyses were done in Stata13.

Dependent Variable
Support for participatory reforms is measured by asking respondents, after each vignette, 'do you think this is a good or bad way of making political decisions?' (0 = very bad/10 = very good).

Vignette Factors
The 'instrument' factor represents whether the mode of decision-making is talk-centric i.e. a public meeting (= 1) or vote-centric i.e. referendums and initiatives (= 0). The 'outcome' factor represents whether the decisions made by citizens are advisory (= 1) or binding (= 0).

Respondent Characteristics
Residing in a R&I State. Respondents indicated which state they currently reside in. All 51 states (as well as Puerto Rico) are represented in the sample with an average of 13 respondents per state, although California, Florida, Pennsylvania, and Texas are overrepresented. The states were recoded into a dummy variable where '0' means referendums and initiatives are not provided for by the state constitution and '1' means referendums and/or initiatives are provided for by the state constitution. Approximately half of the sample resides in a referendum or initiative state. Information about the availability of direct democratic instruments across states was obtained from the website of the NCSL (2018).
Skeptics. Concern for public incompetence is measured with a reverse code of the statement: 'most people have enough sense to tell whether the government is doing a good job' (0 = strongly disagree/4 = strongly agree). Concern for minority rights is measured with a reverse code of the statement: 'in a democratic society, letting the majority decide is more important than protecting the rights of minorities' (0 = strongly disagree/ 4 = strongly agree).
Control variables. Age, sex (0 = male, 1 = female), political interest, and education were included as control variables as they might confound the relationship between the respondent characteristics described above and support for participatory reforms. For political interest respondents were asked 'how interested are you personally in politics?' (0 = not at all interested/ 4 = very interested). Education is a categorical variable representing the respondent's highest level of education (0 = middle or high school; 1 = vocational degree or some college experience; 3 = college graduate). Several studies have found that education is negatively correlated with referendum support (Bengtsson & Mattila, 2009;Collingwood, 2012;Donovan & Karp, 2006) and Anderson and Goodyear-Grant (2010) argue that highly informed voters express concern for public incompetence and minority rights.
For descriptives of the variables see Table 1.

Results
The overall mean support for participatory reforms is 5.89 on a scale from 0 to 10 (see Table 1), which is not overwhelming but still above the midpoint (5). The descriptive results in Table 2 demonstrate that respondents evaluate vote-centric instruments more favorably than talk-centric instruments. Advisory outcomes seem to be preferred but especially for talk-centric instruments; the difference in support for binding vs. advisory referendums is quite small. The main effects of the factors and respondent characteristics are presented in Table 3, Model 1. The interactions for H2-H7 are included separately in Models 2-7. All Models are based on the total sample of vignettes (N = 1,920 or 2 experimental conditions per re- spondent). Starting with the main effects presented in Model 1, talk-centric instruments receive considerably less support than vote-centric instruments (almost a full point on an 11-point scale), confirming H1. Advisory decisions lead to more support for participatory reforms than binding decisions, although the effect is not very large (0.31 on an 11-point scale). Given that each respondent rated two vignettes, we checked whether these two vignette factors had similar effects within and between respondents, which was indeed the case. Taking the instrument factor as an example this means, simply said, that respondents who were assigned two public meeting vignettes gave a lower average rating than respondents who were assigned two referendum vignettes (betweenperson effect), but also that respondents who got one public meeting vignette and one referendum vignette, on average rated the public meeting vignette lower than the referendum one (within-person effect). This strengthens our findings on the effects of the vignette factors. Model 1 also demonstrates that individuals residing in R&I states do not differ in support for participatory reforms from those residing in non-direct democratic states. An empty model in which the state-level was included as a third level showed no variation in the dependent variable at the state level, making it unlikely that we would find a main effect of state characteristics. However, it could still be that living in a R&I state interacts with the vignette factors, which we test in Models 3 and 4. Confirming expectations, individuals with less faith in the political competence of the general public and individuals expressing concern for minority rights are less favorable towards participatory reforms, which was the case even when controlling for education. The three education groups hardly differ in their mean scores on public incompetence and minority rights, which suggests that these concerns are not unique to higher educated individuals. Turning to the controls: women express greater support than men; age and political interest do not have an effect; college graduates express less support for participatory reforms than non-graduates.
In Model 2 we added the interaction between the outcome factor and the instrument factor. The main effect of advisory vs. binding (which in this model is the effect in the condition public meeting = 0, or the effect on support for vote-centric instruments) is not significant demonstrating, contrary to H2b, that people do not have a preference for advisory vs. binding decisions when the instrument of decision-making is vote-centric. The main effect of public meeting (which is the effect in the condition advisory = 0) shows that binding meetings are evaluated less favorably than binding referendums. The in-teraction coefficient is positive and borderline significant (p = 0.88), suggesting that the negative effect of public meetings on support is weaker when the outcomes are advisory. This provides some support for our expectation that talk-centric instruments are evaluated more favorably when they play an advisory role (H2a). More straightforward evidence for this hypothesis is provided by an additional regression analysis of the public meeting vignettes only, i.e. on half of the sample of vignettes: in this model the advisory factor has a positive effect (B = 0.599, p = 0.001) on support for talk-centric instruments (see Appendix Table 2).
The interaction between the instrument factor and residing in a R&I state was included in Model 3. This interaction is not significant, demonstrating that the preference for vote-centric instruments over talk-centric instruments does not depend on whether one lives in a R&I state or not. Hence, we did not find support for our expectation that the preference for vote-centric instruments would be stronger among individuals in R&I states (H3).
In Model 4 we added the interaction between residing in a R&I state and the outcome factor to test our hypothesis that the positive effect of residing in a R&I state on support for participatory reforms is weaker when the decisions are advisory as opposed to binding (H4). We did not find an overall positive effect of residing in a R&I state (see Model 1) and the main effect of R&I states in Model 4 (now representing the effect of R&I states in the condition advisory = 0) shows that individuals from R&I states are less positive about participatory reforms with binding outcomes than individuals in non-direct democratic states. The interaction coefficient is positive and borderline significant (p = 0.061), suggesting that the negative effect of living in a R&I state disappears when the outcomes are advisory.
The interaction between concerns about public incompetence and the outcome factor was included in Model 5. The significant interaction effect demonstrates, consistent with H5, that individuals expressing concerns about the average citizens' inability to understand politics are not as negative about participatory reforms when the outcomes are advisory as opposed to binding. In fact, the negative effect of public incompetence concerns disappears when the decisions are advisory, suggesting that this is no longer a concern when citizens play a consultative role.
In Model 6 we added the interaction between concern for minorities and the outcome factor. The significant interaction effect demonstrates, consistent with H6, that individuals expressing concern for minority rights Table 3. Multilevel regression estimates of support for participatory reforms (N = 1920 vignettes nested in 960 respondents).

Variables
Model 1 Model 2 Model 3 Model 4 Model 5 Model 6 Model 7  are not as negative about participatory reforms when the outcomes are advisory as opposed to binding. In conclusion, our expectation that the critics of participatory reforms would be less skeptical if the outcomes were advisory is confirmed for both public incompetence and minority rights. Finally, the interaction between the instrument factor and concern for minority rights is included in Model 7. The interaction effect is not significant demonstrating that concern for minorities is associated with lower levels of support for participatory reforms, regardless of whether the decisions are made in referendums and initiatives or public meetings. Therefore H7, which predicted that the negative effect of concern for minorities is weaker (or absent) for talk-centric instruments, is rejected.

Conclusion
Deliberative democrats claim that political discussion among ordinary citizens improves decision-making processes and enhances democracy (Dryzek, 2007;Gutmann & Thompson, 1996;Mansbridge, 2007;Pateman, 2012). Political discussion has an educative effect, by stimulating participants to develop their argumentation skills, and a reconciling effect, by encouraging participants to consider opposing viewpoints (Fishkin, 1991;Habermas, 1996). By contrast, decision-making processes centered on voting mechanisms are perceived as unreflective and polarizing (Chambers, 2012). On these grounds, some scholars have suggested incorporating political discussions into participatory decisionmaking and empowering these forums to take binding decisions (Fung, 2007;Pateman, 2012;Setälä, 2011). However, the main finding of this study suggests that such an approach to political decision-making would not go unquestioned by ordinary citizens. Respondents are considerably less enthusiastic about talk-centric decision-making instruments than vote-centric decisionmaking instruments: vignettes about public meetings are rated on average one point lower on a 0-10 scale than those about referendums and initiatives. Public meetings are evaluated more favorably when they are restricted to an advisory role, but even then support is far from overwhelming (advisory meetings score just above the midpoint of the scale).
This experimental finding contrasts with standard survey research demonstrating that citizens score similarly on single items about referendums and public assemblies (Font, Wojcieszak, & Navarro, 2015). The vignettes encouraged respondents to consider these decision-making instruments more carefully and therefore provide a more fruitful way of disentangling citizens' attitudes towards participatory decision-making. This finding also contrasts with a survey experiment by Neblo et al. (2010, p. 573) demonstrating that a large majority of Americans (83%) are interested in participating in a deliberative session with a member of Congress.
Empirical evidence from mini-publics shows that citizens both welcome and enjoy the opportunity to take part and to deliberate, and that they take their duties seriously (Pateman, 2012). However, the low level of support for the use of public meetings in our study suggests that respondents are not enthusiastic about more active engagement in political decision-making, closer to Hibbing and Theiss-Morse's more pessimistic claim that what most Americans want is a 'Stealth Democracy ' (2002).
On the other hand, support and participation are not necessarily the same: citizens may dislike the idea of using public meetings to inform political decisions, but still happily participate in a deliberative forum. Future research should investigate why citizens support talkcentric decision-making instruments less. Is it because they do not want to participate in more demanding activities or is it because the decisions made by a small selection of citizens are not perceived as legitimate for the wider democratic polity? The common complaint of political elites not representing the interests of ordinary citizens makes it difficult to imagine that citizens would accept the decisions made by an unelected body that cannot be held accountable.
Ordinary citizens do not follow the normative debate about deliberative democracy, which might explain why they are less enthusiastic about talk-centric instruments. By informing citizens about the benefits of political discussion for political decision-making and by advertising these forums among the general public, deliberative democrats can build support for talk-centric instruments. Based on evidence from 41 deliberative forums, Michels (2011) demonstrates that participants in deliberative forums are considerably more positive about the process and the outcomes than non-participants. Therefore, future research could investigate whether participation improves attitudes towards talk-centric decisionmaking, or whether positive attitudes are caused by a self-selection bias.
Although referendums and initiatives were rated more favorably than public meetings, respondents did not have a clear preference for binding referendums over advisory ones. This is surprising because concerns about growing dissatisfaction with representative institutions suggest that citizens desire more mechanisms to control their representatives (Bowler et al., 2007;Cooper, 2018), but also because binding referendums are a common feature in many American states. Comparing the attitudes of citizens with and without access to referendums and initiatives leads to findings that might concern the proponents of direct democracy. First, individuals in R&I states are not more enthusiastic about the use of referendums and initiatives than those in non-direct democratic states, which suggests that access to these instruments does not contribute to more participatory attitudes about political decision-making. Second, contrary to expectations, individuals in R&I states (where these instruments are usually binding) are less positive about binding outcomes than individuals in non-direct democratic states, which may imply some dissatisfaction with the role these instruments currently play in political decision-making.
Finally, the results demonstrate that concerns about public incompetence and minority rights are two potential reasons why some citizens are skeptical of participatory decision-making. However, individuals who express these concerns are found to be more favorable towards participatory reforms when outcomes are advisory, giving politicians the final say. In fact, concerns about public incompetence no longer influence citizens' attitudes towards participatory reforms when the decisions are advisory. Legislatures could address concerns about public incompetence and minority rights by using advisory referendums for issues that are either too technical or highly divisive. The finding that concerns about minority rights apply to talk-centric instruments as well is potentially troubling for theorists who claim that deliberation provides a platform for minority perspectives to be heard (Mendelberg, 2002). Respondents may fear that public meetings are overshadowed by the loudest voices.
Turning to the limitations of our study, the sample is not representative which means that the absolute levels of support for participatory reforms and the main effects of respondent characteristics should be interpreted with caution. However, our hypotheses are largely about effects of the experimental factors and how these differ between respondent groups, for which sufficient variation in the respondent characteristics matters more than a representative sample. Even so, the results raise important questions about the extent to which direct and deliberative decision-making instruments are welcomed by ordinary citizens and warrant further investigation on a representative sample.
On one hand, the vignette text could be expanded to provide more details about a deliberative meeting. On the other hand, the task of reading and comparing the two vignettes is cognitively demanding, especially in comparison to standard survey items. Therefore, a more detailed description would make the experiment even more challenging for respondents, especially those with lower levels of schooling. A qualitative pilot among a convenience sample of students revealed that vignettes with more factors were difficult to rate. The complexity of the vignettes might explain some of the small effect sizes. Perhaps a larger sample size would have resulted in more significant interactions, as some relatively large interaction effects had p-values between 0.05 and 0.10. Despite the complexity of the vignettes, the factors did produce effects, suggesting that citizens can have opinions about more complicated political questions and that factorial survey experiments can be a useful method for studying political norms and attitudes.  Notes: * p < 0.05, ** p < 0.01, *** p < 0.001.