Evaluating the Short-and Long-term Effects of a Modified Deliberative Poll on Idahoans ' Attitudes and Civic Engagement Related to Energy Options

Deliberative polling engages citizens in a process of dialog, exposure to information, and interaction with experts, with the goal of developing more informed, stable attitudes and promoting civic engagement. There have been relatively few assessments of deliberative polling’s short-term effects and even fewer on its long-term effects. Moreover, most assessments examine aggregate changes in attitudes, although theory would suggest that changes might be idiosyncratic, especially in situations where balanced information is provided. This study examines the effects of a one-day modified deliberative poll regarding energy options for the state of Idaho. Using quantitative data from pre-tests, an immediate post-test (n=61), and a follow-up post-test eight months later (n=44), as well as qualitative data from interviews, this study explored preferences for five energy options and how participation affected civic engagement. We also report on participants’ assessments of the deliberative poll itself. Results show few significant changes in aggregate attitudes, with the exception that preference for nuclear power declined after the deliberative poll event. However, examining within-subjects data revealed that up to approximately 30% of participants exhibited substantive changes in their preferences, both immediately after the event and several months later. The extent of changes between the post-test and delayed posttest reinforces that attitudes are affected by individuals’ life circumstances, and suggests participation did not lead to highly enduring attitude changes. There was little evidence that participation led to increased civic engagement, either through communicating what was learned with others or through becoming more politically active


Introduction
The deliberative democracy movement advocates meaningful citizen engagement in policy issues (Thompson 2008).Proponents argue citizens' off-the-cuff opinions are often uninformed, do not reflect what might expected if they thought carefully about issue-relevant information, and therefore are a poor guide for policy decisions (Andersen & Hansen 2007;Fishkin & Luskin 1999, 2005).Participatory processes that expose people to balanced, thorough, and accurate information, along with the opportunity for reasoned and open discussion, are thought to lead to more considered opinions and broader engagement in civil society (Luskin et al. 2002;Parkins & Mitchell 2005) and higher-quality decisions (Walmsley 2010).Various experimental studies have confirmed that-in a typical opinion poll-a disturbingly large percentage of people have "non-attitudes"; that is, they express an opinion on a fictitious issue (Bishop et al. 1980;Schuman & Presser 1980;Smith 1984).Where this is the case, policy makers are understandably reluctant to base decisions on such input.Thus, such findings provide support for processes such as deliberative polling, which seek to generate more informed, stable, and "real" opinions to help inform policy-making.Participation in deliberative events may help people develop well-informed attitudes and increase capacity and willingness for policy engagement.
Various forms of involvement are promoted within the deliberative democracy movement, all of which include processes for learning, engagement with fellow citizens and scientists, and expressing support for policy options (Fishkin & Rosell 2004).Currently, one of the more widely used processes is the deliberative poll.Deliberative polling involves assembling a group of citizens, who engage in a series of structured educational and interactive exercises.The participants ideally should represent the citizenry, which is generally achieved by drawing a random sample (Fishkin & Luskin 1999, 2005).In such exercises, participants are provided accurate, comprehensive scientific evidence and policy information through "briefing documents" that enumerate the pros and cons of alternative policy options.Opportunities are provided for dialog among event participants, and they collectively generate questions for experts, which are addressed during a panel discussion (or "conference").At the end of deliberative polls, the citizen participants make policy recommendations, either privately through a questionnaire or through a consensus-building process.Deliberative polls are often broadcast, and being filmed creates social pressure on participants to be civil, thoughtful, and cooperative.
The goals of deliberative democracy are laudable; however, as yet there have been relatively few systematic evaluations to determine how well deliberative activities achieve their objectives of helping citizens develop informed attitudes or promoting civic engagement.Through a mixed-methods investigation over a period of nearly one year, this study evaluated the effects of a modified deliberative poll on participants' attitudes toward energy alternatives and their levels of civic engagement.In the next sections, we discuss the mechanisms by which deliberative processes might affect attitudes and engagement, as well as the current state of knowledge based on recent evaluations of deliberative polls and related forms of engagement.Then we describe the methods used in our modified deliberative poll, followed by a discussion of its short-term and long-term effects.We conclude with implications for deliberative democracy and engagement processes.

Theorized Effects of Deliberative Processes
Proponents of deliberative engagement processes argue that opinions on issues will change as a result of participation, because citizens are presumed to be largely ignorant of political and policy issues, so that exposure to information and others' opinions will naturally lead to a more comprehensive understanding of the issues and alternatives (Fishkin & Luskin 2005;Luskin et al. 2002).However, the mechanisms of such change and the conditions under which change will occur are poorly described in the deliberative polling literature.Indeed, deliberative polling seems more concerned with citizens developing substantive and well-informed opinions of any sort than with the specific nature of changes and mechanisms of influence.Social psychological theories of attitudes and persuasion are rarely discussed in deliberative democracy literature, but offer great promise to help predict when and what types of attitude changes may occur.Specifically, the Elaboration Likelihood Model of Persuasion (Petty & Wegener 1999) and the Heuristic-Systematic Model (Chaiken et al. 1996) posit that attitudes are composed of integrated networks of beliefs.If people are motivated to process arguments carefully, as they presumably are during deliberative events, attitude change will be determined by the aggregate number of positive and negative thoughts elicited in relation to different potential outcomes (Petty et al. 1995).
To the extent that people's initial opinions about complex technical issues are poorly formed or based on limited, potentially incorrect information, it is reasonable to expect that individuals' attitudes might change during a deliberative process, as they become aware of and evaluate the implications of different policy options for themselves and others.On the basis of persuasion theory, one might expect a process that provides balanced information to lead people to form opinions (if they had not thought about issues previously) or to become ambivalent (if they came to realize that there are many legitimate arguments for and against each option).There is, however, no a priori reason to expect an aggregate attitude shift in one direction or another when information and debate are balanced, because generally there are legitimate arguments for multiple perspectives and options, and people's different value priorities will lead to divergent evaluations of the same options (Steg et al. 2005;Hansla et al. 2008).
For instance, one participant who is concerned about climate change might become convinced that nuclear power is the best option, while another who is concerned about cost might become convinced that it is not.Even though both learned the same information, they reached different conclusions about what policy to adopt.
Although changes may occur as described above, contrary to deliberative polling literature, persuasion theory suggests that it can be very difficult to change attitudes, because people with a personal interest in a topic or who have already formed a strong attitude are likely to engage in "defensive" elaboration, in which they attend selectively to facts that support their view and counterargue those that contradict it.Although deliberative events are, by design, intended to elicit openminded contemplation of issues, humans unconsciously and automatically engage in selective processing (Knowles & Linn 2004).People with high levels of interest in a topic have strong attitudes that are linked to values and other attitudes in tightly integrated cognitive structures (Eagly & Kulesza 1997).Public involvement processes tend to engage people with interest in the issues, and these people are the least likely to exhibit changes in attitudes (Lien 2001;Petty & Wegener 1999).
In addition to generating more informed attitudes, proponents of deliberative processes argue this type of involvement can lead to higher levels of subsequent civic engagement both for the issue at hand and other issues.It is often claimed that, in modern democracies, citizens feel alienated from the policy process (Fishkin & Luskin 1999), which seems more and more to devolve into ideological battles and extremism.Furthermore, citizens are aware of the growing gap between scientific expertise and their own limited knowledge of science and technology.Thus, people may feel incapable of contributing meaningfully to policy decisions on such complex social issues.In deliberative processes, people learn they are capable of understanding and reasoning about issues constructively and civilly with fellow citizens (Fishkin & Luskin 2005).Furthermore, they learn that the role of science is to inform -not make -policy decisions.When they observe policy makers taking them seriously, they should experience empowerment to influence policy (Eggins et al. 2007;Mutz 2008;Rowe et al. 2005).These assumptions are consistent with Social Cognitive Theory (Bandura 2001), which posits that observing and practicing model behavior can lead to increases in self-efficacy.

Empirical Findings Regarding the Effects of Deliberative Processes
Although rigorous assessments are far fewer than the number of deliberative polls and similar participatory processes that have been conducted, a few studies have investigated their effects on participants' attitudes toward policy issues.These paint a complex and not altogether coherent picture (Mutz 2008).Some studies report systematic shifts in policy preferences among participants.For example, Eggins et al. (2007) found a modest (7%) shift in the mean attitude rating among participants in an Australian deliberative poll about a bill of rights, and French and Laver (2009) documented a strong, consistent shift against a waste incineration proposal, which they attributed to the differential persuasiveness of expert speakers.Abelson et al. (2003) also found consistent shifts, which they believed were the result of particularly persuasive members of small group discussions.The primary developers of deliberative polling, Fishkin and Luskin, report that "usually statistically significant net changes" in attitudes occur (Fishkin & Luskin 2005, p. 291).
On the other hand, several recent studies find no, or very weak, aggregate changes among participants as a whole.In their study of the first British deliberative poll, Luskin et al. (2002) found mean attitudes toward various social policies to shift between 3 and 6%.More recently, Farrar et al. (2009) reported that, in deliberative polls about national security and free trade, "mean opinions changed trivially and in no consistent ideological direction" (p.626).Likewise, Sturgis et al. (2010) found that exposing people to either short or long versions of a balanced film about genomic science led to no changes in attitudes compared to a control group, either immediately or 4-9 months later.
As argued above, to the extent a deliberative poll succeeds in presenting balanced information, it may not be surprising that there is no overall change in mean attitudes, even if individuals do experience substantial idiosyncratic shifts.It is therefore somewhat puzzling that most studies examine aggregate attitudes and few have looked at changes within subjects by using different analytic techniques.Those who have done so present intriguing findings.For instance, in Eggins et al.'s (2007) study, the variance in responses increased after the deliberative poll, suggesting the information and engagement led to divergent changes among participants.Similarly, Andersen and Hansen (2007) found that between 7 and 28% of respondents reversed their opinions about issues related to the Euro, even though individuals did not change in a consistent direction.Additionally, they reported that many people who were "undecided" at the pretest formed an opinion as a result of participation.Luskin et al. (2002) reported that, depending on the specific issue, 30-40% of participants "changed sides" after participating in a deliberative poll.Additional research is needed to investigate how attitudes change as the result of a deliberative process, and this study focuses primarily on such within-subjects changes.
Given the democratic goals of deliberative polling, it is also surprising that little attention has been devoted to examining the effects of participation on participants' subsequent political engagement (Abelson et al. 2003).In one of the only studies on the subject, Eggins et al. (2007) examined how deliberative polling affected participants' intended political engagement, specifically their intention to discuss the issue with others and desire to play an active role in development of others' views.They found a fairly strong relationship, with much of the variation in political engagement explained by people's feelings of pride in participating and their sense of being a community representative.Unfortunately, the study did not include a follow-up component to assess whether such intentions were later carried out.Another study of actual behavior (Andersen & Hansen 2007) found that participants became only slightly more politically active after a deliberative poll.However, Fishkin and Rosell (2004) report that follow-up studies create lasting changes in political participation.Thus it remains unclear how much such processes actually increase civic engagement.
Most evaluations have taken place directly following a participation event.In the immediate aftermath of one or more days of intense discussion, people may feel excited, interested, and proud of their involvement.Proponents of deliberative polling argue that such outcomes persist over time, having profound impacts on both attitudes and civic engagement.While this is certainly possible, it is also possible, for instance, that information which seemed compelling at the time is forgotten, leading people to revert to their original attitudes.Or, as people return to their usual social groups, information sources, and daily lives, their "new" attitudes may fade over time.Intentions to become more active may give way to more habitual forms of action.Thus, in this paper we report on both the immediate and long-term effects of a modified type of deliberative poll held in April, 2009, in Boise, Idaho, that explored energy options for the state.

Methods
This was a mixed methods study.Structured questions investigated attitude changes for five energy options (fossil fuels, renewables, nuclear, hydroelectric, and energy conservation and efficiency).Open-ended interviews 8-10 months afterwards explored how the event affected people's attitudes and its effects on their subsequent civic engagement.The interviews also elicited evaluations of the deliberative process itself.Before describing the measures we used, we explain the deliberative process and how it varied from a true deliberative poll.

The Modified Deliberative Poll
The modified deliberative poll that underpins this study was designed as a quasiexperiment, in which randomly selected citizens were exposed to different treatments, based on the recommendations of deliberative polling proponents (Luskin et al. 2002).As Fishkin and Luskin (1999, p. 6) note, a deliberative poll involves "one grand manipulation," and therefore it is not possible to disentangle the effects of specific components.Indeed, Fishkin and Luskin (2005) later noted that research would be welcome that investigates how much each element matters to achieving desired outcomes.Thus, our modified process sought to isolate the effects of briefing documents, group discussion, and interaction with the expert panel, though (as discussed later) this proved challenging.
Two thousand randomly selected citizens from seven Idaho counties received a pre-test survey in the mail in January, 2009, which included attitude measures (see below) and a postcard invitation to the deliberative event.Among the 504 people who returned completed surveys, those who did not decline further contact were invited to the event.After repeated phone calls, 95 people agreed to attend the one-day session.They were randomly assigned to seven different treatments, three control groups that did not attend the event and four event groups.(Only the event groups are included in this study.)Two of the event groups then received a 35-page briefing document in the mail, which provided a balanced description of how the electricity system works, an overview of the energy situation in Idaho, and the benefits and problems with the five potential energy options.Each option was discussed in terms of safety and security, reliability and predictability, public trust, impact to the environment, cost, responsiveness and adaptiveness, aesthetic considerations, and additional benefits beyond energy supply.
Sixty-one people attended the event and were compensated $75 for their time.All of them listened to an hour-long presentation that reviewed the highlights of the briefing document.Some participants then engaged in facilitated small group discussions about energy options, in which they generated questions for an expert panel; those who did not engage in deliberations listened to a presentation about deliberative democracy.All of the participants then attended a luncheon conference in which a panel of experts answered questions that had been developed by the small groups.Afterward, the participants who were part of small group deliberations reconvened to discuss what they had learned, while the others were dismissed.All participants completed a post-test survey containing the same attitudinal items before they departed.All event participants for whom we had telephone information were contacted by a trained university interviewer between November, 2009, and January, 2010, to participate in a brief interview.Several attempts were made to reach each participant; ultimately 44 completed interviews were completed.

Measures and Analysis
For the purposes of this paper, we refer to the initial survey as the pre-test, the survey completed immediately after the April event as the post-test, and the telephone interview as the delayed post-test.The pre-test included sociodemographic questions (age, income, gender, and education) as well as several measures not reported here.The key attitude questions of interest were preference ratings regarding the five energy options, which were posed identically at all three times; participants rated their preference on 11-point scales from -5 (strongly opposed) to +5 (strongly support).Consistent with conventional aggregate analysis, we examined changes in mean preference between the pre-test and posttest, and between the post-test and delayed post-test, using paired t-tests.
We also examined the data for attitude changes within subjects."No change" indicates that a person gave identical ratings two times."Shift within position" indicates different ratings, but within the same side of the scale, either positive or negative (for example, from +1 to +4, or from -3 to -4)."Reversed" indicates participants who changed from a negative to a positive rating, or vice versa."Formed opinion" indicates people who changed from a neutral rating (0) to either a positive or negative rating, while "became neutral" indicates a shift from a valenced score to a neutral rating.A small difference (e.g., from +1 to +2) may not indicate an actual change in attitude, as there is likely to be some unknown degree of measurement error.Moreover, the policy implications of an individual changing from, say, slightly to moderately supportive seem limited.However, reversal, forming an opinion, or becoming neutral may indicate substantive changes, and we focus on those.
In the follow-up interviews, participants were asked to self-report the effect of the event on their knowledge and attitudes about energy options ("How do you think the deliberative poll affected your knowledge and attitudes towards the energy options for Idaho if at all?" "Did anything in the small group discussion influence your opinions about any of the energy options?" "Did anything a panelist said affect your opinions on any of the energy options?").Probing questions were used to elicit deeper insights or clarify responses.To assess civic engagement effects, subsequent questions asked participants about the degree to which they had sought out additional information after the event ("Since the conference have you looked further into any of the energy options?"), shared information with others, or encouraged others to look into energy policy issues ("Have you encouraged anyone else to look into these issues?").Another question asked, "Have you gotten involved in energy policy issues in any way since the conference?"Finally, to evaluate the quality of the deliberative event, participants were asked about the small group interactions and the expert panel, as well as their overall evaluation of the process.
Interviews were transcribed verbatim.Two coders independently read through the interviews to identify common themes.After consultation, a codebook was developed, and each coder applied it to samples of interviews to establish inter-coder reliability (Krippendorff 1980;2004).When an adequate κ value was achieved (>.80 for each top-level code), both coders independently applied the codebook to all remaining interviews.Discrepancies were resolved through consultation.Codes used in this study pertained to the impact of the event on knowledge, attitudes, or behavior related to energy; evaluation of the event (and specific elements); and personal insights related to energy (any influences on attitudes apart from the deliberative event, such as media, local politics, or home energy issues).

Results
Though derived from a random sample of citizens, the 504 people who returned the pre-test questionnaire were different socio-demographically from the citizenry of their counties.Compared to the four counties that make up 95% of responses, pre-test respondents were older (39% over age 65, vs. 10% for the population), more likely to be male (79% vs. 51%), better educated (51% had at least a Bachelor's degree, compared to 28% of the population), and wealthier (median income $67,000 vs. $53,000).However, the people who participated in the deliberative event were generally similar to the pre-test sample, both demographically and attitudinally.For instance, the largest differences across all the attitudinal variables were for general environmental concern (96% of event participants were concerned about impacts to the environment, compared to 88% of the pre-test group) and belief that fossil fuels are harmful to the environment (76% vs. 63%).As with other deliberative polls (e.g., Luskin et al. 2002), we do not have attitudinal measures for those who did not respond to the pre-test, so we cannot say definitively whether our sample represents the general public.However, of the deliberative event participants, 21 had either schooling or employment in an energy or related industry, 10 were active in some way with energy issues, and 8 mentioned being aware of or involved in local issues such as power plant siting in their region or neighborhood.Thus, it seems reasonable to assume that many participants had high interest and pre-existing attitudes about energy issues.
The results that follow first present aggregate changes, consistent with other studies, using the 61 event participants in comparisons of the pre-test and post-test, and the 44 interview participants in analyses comparing the post-test to the delayed post-test.This is followed by analysis of changes within subjects, with particular emphasis on substantive changes (as defined earlier).Excerpts from interviews are included as appropriate to illustrate changes.The results conclude with the qualitative findings on long-term civic engagement.
Table 1 shows the number of participants by treatment group."Conference" indicates attendance at the luncheon panel discussion, while "Deliberation" refers to participation in moderated small group discussions.Cooperation with the interview was higher among people who were involved in more aspects of the deliberative poll.During the delayed post-test interviews some respondents were unwilling to give numeric ratings for certain options.Additionally, some interviewees differentiated within one of the categories; for instance 29% of people gave different ratings for coal, oil, and natural gas, although the questionnaires had combined these into one "fossil fuel" category.In these cases, no overall rating could be computed for the delayed post-test.Therefore, for some energy options, the number of delayed post-test responses is less than the total number of participants.First, we explored the aggregate (mean) changes (Table 2).The sample sizes for each treatment are quite small, so the results should be regarded as suggestive, not definitive.Overall, there were few statistically significant aggregate changes.For fossil fuels and hydropower there were no statistically significant changes (at α = .05)within each treatment or across all treatments.The large standard deviations for fossil fuels, hydropower, and nuclear power indicate a high degree of polarization among the groups, especially for fossil fuels, where the mean ratings were near zero.Across all treatments combined, support for nuclear power declined immediately after the deliberative poll and stayed lower eight months later.On the other hand, support for renewable energy sources increased slightly (but not significantly) after the event, but returned to pre-test levels by the time we conducted our interviews, so that the difference between the post-test and delayed post-test was significant.It does not appear from these limited results that any specific treatments evoked more change than another.For example, the group that had the full set of activities (briefing documents, small group deliberation, and the conference) exhibited essentially as little aggregate change as the group that only attended the conference.While these data show few statistically significant changes, theory suggested that there could be idiosyncratic changes within subjects, so we next examined such changes.Table 3 presents the percentage of participants in each treatment that exhibited substantive changes (forming opinions, reversing positions, or becoming neutral).For each option, although there was considerable variation, these data show that people in the treatments with deliberation did not change any more than the people who did not deliberate, nor did people who received briefing documents change more than people who did not.Additionally, the people who only attended the conference did not change less than other people.Indeed, differences among the energy options were larger than the differences across treatments.Thus, these results, though limited by small samples, do not reveal any strong influences of specific deliberative polling elements.Given this finding, we aggregated the participants in looking at specific types of changes within subjects (Table 4).This result reinforced the different patterns of changes for the different energy options.Attitudes toward renewable energy sources were especially stable, while attitudes toward fossil fuels were most likely to change.For respondents who gave a different response, but on the same side of the scale, the mean change was between 1.3 and 1.9 points on the 11-point scale.There were 19 ratings of "neutral" at the pre-test, across all five options together, and 12 of these changed to a substantive value immediately after the deliberative poll.In contrast, few participants shifted from a substantive preference to a neutral opinion.Respondents' self-reports during the interviews provide considerable insight into the effect of the deliberative event on their attitudes.Eleven people said that they had not learned any new information and did not shift their opinions about any of the options.These responses reinforce the view that event participants had strong prior attitudes.

I think my attitudes have not really changed a lot.… Having a background in science and engineering, I already knew quite a bit about the options.
(5-1161SC) I've always been interested in this.I've probably read a great deal more about it than some of the participants had.…It was a good format, but there wasn't anything that was brought up that I didn't know something about.(5-02472MC).
Twenty individuals (approximately half of the participants we interviewed) said that they had learned new information, but nevertheless did not shift their preferences for energy options.This result is consistent with psychological theories of persuasion, which posit that people selectively attend to information and have extensive pre-existing cognitive structures, so that learning new facts may have little impact on attitudes.
I think it expanded my knowledge to some degree.There were certain aspects of our energy policy, for example, the quantity of energy that we import into this state.The fact that we're opposed to coal fired plants in Idaho, yet we'll buy energy from coal fired plants across our border.I wasn't aware of that prior to that conference.In terms of changing my opinion, I don't think so.(1-00195CR) I'd have to say I learned some little things that you don't tend to think about…the finer points that come up that you don't really think about that…yeah, we learned some things….Maybe it reinforced my feelings a little bit concerning wind power and hydroelectric and the renewable end of it.I think there are more options…more availability than we're taking advantage of.It didn't change it as much as it reinforced it a little bit.(4-02578PR) Many of these people recognized that their opinions were well established and unshakeable prior to the event.For example, one participant who acknowledged that the event made him "aware that there's a lot of things going on in Idaho…like the fact that we share all of our power with all the different states" said that even though he "learned more stuff, it didn't affect [his] outlook" because he is "pretty stubborn" (1-00361SM).Another echoed this sentiment, saying that though he "got some good knowledge out of it," he "went in there with one track mind and left with a one track mind" (5-02371SM).
On the other hand, twelve people reported that they had learned new information and as a result changed their attitudes toward one or more of the energy options.As evident in the following excerpts, people picked up on different pieces of information, which led to different attitudes.
I think the conference was excellent as far as getting…to increase my knowledge of the pros and cons of all the different options available to us.…Maybe just safety as a nuclear power industry.…Theexposure to expenses on some of the nonrenewables and conservation alternatives….Apart from the effects on their attitudes about specific energy options, eleven people reported the deliberative poll event had changed how they think about energy, generally making them more aware of the complexities of the issues.As one stated, "it very much opened up my mind to thinking about the need for options and of the need for information about all the options.It let me know that the issues were more complicated than I thought" (5-00036PO).

I think maybe it just allowed me instead of just kind of thinking one dimensionally for myself, you know I can see what other people are thinking…that definitely changed my attitude because like I said, instead of just thinking about myself, I can think about other people. (4-04984NA)
Changes within subjects in ratings from the post-test to the delayed posttest, as well as from the pre-test to the delayed post-test, showed interesting and complex results (Tables 5 to 9).Attitudes appeared almost as volatile across this several-month time period as between the pre-and post-tests.The percentage of people exhibiting "substantial" changes from post-test to delayed post-test ranged from 5% (renewable energy) to 39% (fossil fuels).As occurred between the preand post-tests, fossil fuels and hydropower showed the largest shifts, while attitudes toward renewable energy and conservation were the most stable.It is difficult to determine how the deliberative event itself contributed to these long-term changes.Given people's high level of interest in energy and their divergent opinions, it is likely that people were exposed to different types of information after the deliberative poll event, and evidently this exposure affected some of them.Many participants reported having a prior interest in energy, involvement with local issues, or making home improvements to save energy.Others had friends, family, or coworkers with whom they discussed energy issues.The type of experience ranged from installing solar panels on the home, to training as a nuclear engineer, to employment by a hydropower company.In the interviews, people explained how these factors affected their views on policy options, illustrating how for some people the one-day event was only one small influence.For instance, one participant who had made home improvements and "cut his energy costs in half" argued that, despite what was said at the event, conservation "doesn't pay," because he was "not saving any money" (1-00361SM).On the other hand, another participant felt quite differently: I feel very strongly about solar, to the point that my wife and I have invested roughly $15,000 into our own solar system.I purchased all the materials and I'm in the process of installing about a 3.1 kilowatt system, which will…in the wintertime I'm hoping to get roughly 40% of our total energy bill offset through what they call a net metering program.In the summertime I expect to see 60%.(2-03799VA) Direct experience also came through travel and the media.For example, on participant had made several trips to Germany and had observed that there are "photoelectric cells or whatever you want to call them…all over the place.They've lined the freeways…the autobahns.Farmers have them…the air isn't much brighter than what I've got in Boise, which is crud.Yet, they still manage to produce electricity" (5-03552BE).Thus, he was convinced that renewable energy is feasible and effective.Others mentioned how stories in the news affected their outlook:

You look at Chernobyl and the problems associated with that. I know that France is often times held up as an example [regarding nuclear power], but I've had conversations with people who say that they are not really doing a good job with the waste as well. Storing it outside is not satisfactory. (2-03161GO)
The concept of a modular [nuclear] plant, similar to France…I think France is doing it, if not in modular, but apparently all their plants are more or less the same, which enables them to be even that much more safe in that if they find something wrong with one they can retrofit others before trouble develops.I think there's a lot to be gained from nuclear power, to tell you the truth.And frankly, the Navy's been using it for a long time.More or less without any grief.(2-01057OR) Some participants also had firsthand experience through community issues and local activism.For instance, one person noted that he had "been following the process of the nuclear applications that have been ongoing here in the area.There's 3-4 counties here in southern Idaho that have been approached and I've been following those in the newspaper" (4-01897WI).These examples illustrate how important personal experience and social influences are on people's attitudes, beyond the presumably more factual information presented during the deliberative event.

Effects on Civic Engagement: Sharing information & encouraging involvement
A large majority of the deliberative poll participants we interviewed said they had shared something about the process or what they had learned with other people after the event (Table 10).Generally they talked to immediate family, and sometimes close friends, although it appears that most of these discussions were either not highly substantive or that people had forgotten most of the details.A typical response to being asked about sharing was, "I had a friend…we got into a discussion about energy.I related some of it back to him when I could still remember what I had heard.So it was a good discussion" (1-02910DI).Another said he had "talked a little bit about it with my coworkers and just mentioned to them that I attended" (2-03799VA).One participant mentioned sharing with a "group of guys" with whom he has breakfast once a week, telling them, "something needs to be done quickly.We're running out of energy options.The prices are going to start [going up]…cheap energy will no longer be for Idahoans to have" (5-00009NE).A few people who had family members in architectural or energy jobs said they had more extended discussions with those individuals.For example, one participant whose daughter worked in green architecture said that she had "definitely influenced my thinking.She found it very interesting what the conference was all about" (4-04466VA).I've played off what I remember with some of the people that I associate with.I just bounced off some of the things that I learned.Frankly, favorable reactions I had to it.Some of these people are engineering types, so we get to take an idea that I may have walked away with and beat it to death.(2-01057OR) Some people shared their favorable impressions of the process itself, for instance the participant who told others "that it was a good experience.If they ever had the opportunity…I suggested to them that they really think about going" (4-04736OR).One of the more enthusiastic participants commented, "I was very impressed with the situation.Mostly impressed with how many want to do something about what we need to do" (2-02000WA).
While respondents talked about the deliberative poll and what they had learned about energy with others, the majority responded "no" when asked whether they had encouraged other people to look into energy issues for themselves.One person summed it up: "Not as just a citizen I haven't.I haven't really encouraged anyone to personally look into the issues" (4-03528LO).Another said, "I guess I'd have to say no, because I don't recall doing that.…Yeah, I don't think so.I guess that's not good" (1-03025NY).
In the few cases where people said they had tried to influence others' behavior, they usually described encouraging others to adopt energy conservation measures in their own homes, as in this example: "I tell them about conservation and that they need to think about it.I try not to press my opinions on people too much" (2-04392CA).None described encouraging others to contact policy makers or take other such civic action.Several of the people who said that they had tried to persuade others to act mentioned that they had always been outspoken (e.g., I bend people's ear all the time.…If people are willing to listen I like talking to them about this stuff"), and it appears that the deliberative poll did not change this tendency.Another, who was interested in "smart meters," had investigated the local power company's practices and said, "Every opportunity, I go ahead and have that conversation with people.So it's kind of directly and indirectly related to the conference" (2-03161GO).

Seeking information
People generally reported they did not actively seek out more information about energy issues after the deliberative poll: "I have not done as you suggest, and that's done any original research on this stuff" (2-01057OR).Another remarked, "I have over the years, but since the conference I haven't really done a lot of thinking about it.And I have not done any research on it" (5-01161SC).In some cases, people said they paid more attention when they came across energy information, although they didn't seek it out.As one person said, "I really haven't looked into anything more.When I hear about something on the news, you know, that regards to energy in Idaho, I just pay attention more now, I guess" (4-04984NA).
Some people said that they had looked more into energy issues after the deliberative poll.They tended to use the Internet to look into specific options, especially conservation and renewables, although some used their personal networks because they trusted them.

Activism in energy policy
Most participants said they had not become active in any energy policy issues as a result of the deliberative poll.One person admitted, "I know I should take the time, but I really haven't" (5-04736OR).Age may have played a role for some people, as for the participant who said, "I'm retired and been retired for 20 years.I guess I'm getting too old and lazy to get too active" (2-02000WA).One person expressed his ambivalence about getting involved when he said, "No, I just sit around and gripe like most of the others.Don't do anything" (4-02578PR).
A small number of individuals who had already been active as citizens continued to be so, and a few discussed how they are involved with energy in their jobs.Their activism is unlikely to be due to the deliberative poll.
I consider myself to be semi-active.I write a lot of letters to the local newspapers…the Tribune, Statesman, and Idaho World…and address just about everything that needs to be addressed.I may have, on purpose or not on purpose, mentioned something about power.(5-03552BE) I am not any more or less involved.Like I said, I do deal with the geothermal resources in my job, so I have some involvement with that, regardless of the conference.(1-00312HE) Only two people described political actions they had taken as individual citizens as a result of the deliberative poll: I think I may have sent a message to a representative among some of the general messages I send every once in a while just urging them to look at a broad-based approach.Not lock into some sort of specific technology or approach, which tends to be typical.…Put a bug in their ear that there are a lot of options out there.…The message I throw out every once in a while to representatives just saying we need an energy policy.We need to put that together, and then we need to follow it and update it every 5 years or whatever.(1-04952ME) I think I made a response online at the Public Utilities Commission about one or two of their proposed conservation programs.Not theirs but Idaho Power or another utility's proposed conservation programs.The PUC was seeking comments.(5-0036PO)

Evaluations of the Deliberative Poll
Nearly all participants who were part of the small group discussions had positive things to say about those interactions.They recognized the high level of interest and knowledge of their group members.As one said, "I was pretty impressed with the people that were in my discussion group.That was an interesting, worthwhile experience" (2-01232BE).Several appreciated the opportunity to be exposed to others' views.
I did find quite a few intelligent people that were up on what their philosophy was.I had a wonderful time listening to them.So, I enjoyed hearing as much as I had the opportunity to present my own opinion (2-02000WA) I thought the work groups were very interesting and informative.…I do recall that some information new to me came out of those work group discussions.It was very worthwhile to listen to other people's ideas and experiences.(2-03242ZE) The few people who mentioned disagreements in the group generally appreciated the value of constructive debate.
It was interesting that we didn't all agree and that we all had different views on different things, and I liked that.If everybody agreed, what was the point?(1-02910DI) It was a great opportunity to have an exchange and debate…people in the small group had views different than mine, which I respect.It really gave an opportunity to expand…in a civil format and really deal with the specific facts instead of just media-driven assessment that we tend to have.I'm always suspect of that.I found it very informative and very useful.(2-03161GO).
Nine of the 23 people who participated in small group discussions voiced criticisms of the small groups.Generally the issues were that there were too many "white-haired old men," the discussion was too short, or the discussion was "not all that lively" or strayed from its task.A few people felt that strong personalities dominated their group, with "opposing opinions being debated amongst a couple of people" (5-02458SA).
In their overall assessments of the deliberative poll, 30 participants had positive views and 19 offered criticisms.Positive comments focused on the value of the event for sharing broad information, raising individuals' personal knowledge, and increasing careful thought about issues.Only a few mentioned that having the chance to interact with other citizens was particularly positive.
I would give it…on a scale of 1-10…about a 7. I thought it was better than what we're getting through the media and it was more balanced than what we get through the media.It's probably more truthful information too.In that case it rated pretty high.I think for the general public, it would be great.(4-03568BR) I think it focuses individuals who otherwise would not pay attention.It gives people a chance to respond or not to an area of particular individual interest.It honors people's opinions which seems to, at least for me, stimulate a desire to be involved.It exposes me, as well as others, to things that they otherwise don't think of on their own.(5-00036PO) I'd love to participate in something like that again.Not only was it educational for me, but I felt like it could be potentially educational for the policy makers as well.…I came away from that experience thinking that I'd love to do that again.And now that I know what it was like, I think maybe I would be more apt to force a conversation or get myself involved in the conversation.…Interviewer: How useful do you think deliberative polling is as a way to inform citizens on current issues?Participant: You know, if you would have asked me that before the session, I would have said not so much, because I think that people here in the West…we're pretty set in our ways.We're either one side of the fence or the other.But, having gone through the experience I know that I changed.I learned something.I maybe have changed my opinion a little bit on, like I said, natural gas.And I wondered how many other folks came away with that same thought that yeah, it changed you or I learned something.With that I thought, boy how effective that could be with other issues that face us.Not just energy policy but whatever.(1-03025NY) Negative comments focused generally on the limited participation (e.g., "we need a bigger group of people") and skepticism about whether participants represented the types of people who need to be involved in such discussions.A few thought that the high cost of the event, participation by those who already knew about the issues, and lack of policy impact seriously limited the value of the event.
The main thing on this issue or any issue is to get a better cross section of the public.I think it's good, but it's too bad more citizens don't take part in it.That's the problem.I don't know how you get more people involved though.(2-04990SU) I would say it's not useful and that's only because…you had the most knowledgeable segment of the Boise population there I think.Not maybe the most, but the people that were there have a lot of interest and are above average knowledge.There is also a segment of the population that gets extremely mad about certain discussions and about the ideas.…So what I'm saying is I think the greatest opportunity is with people who wouldn't necessarily attend something like that.I think you're getting a very biased sampling of the population, in other words.(2-01232BE)

Evaluations of the Deliberative Event
As in other studies (Abelson et al. 2007;French & Laver 2009), most participants enjoyed taking part in the deliberative event.They appreciated the effort to create a fair and balanced process and some mentioned that initial fears about a "biased" process were not realized.It is important to remember, of course, that these sentiments reflect the views of the small, self-selected group of people who chose to attend the event, but for these types of people, deliberative polling appears to be a useful form of involvement.
The small group discussions were especially well-liked and were, for some, an opportunity to share their knowledge with others, while others preferred to listen and learn.Further, some of those in treatments without small grop discussion remarked that they would have liked the opportunity for discussion rather than a lecture on deliberative democracy.This response suggests that, when creating and planning deliberative events, organizers should recognize that participants may value the opportunity to garner new insights and share with others, apart from the acquisition of new technical knowledge.

Attitude Changes
The aggregate data analysis and examination of idiosyncratic changes within subjects revealed different insights about attitude changes.Like other studies that strove to use balanced information (e.g., Farrar et al. 2009;Sturgis et al. 2010), we found no aggregate changes in mean preferences for conservation, fossil fuels, and hydropower.(It is, of course, possible that these findings are affected by the small samples we obtained.)Conservation and hydropower were generally favored at all times, but fossil fuels were rated near the mid-point overall, with large standard deviations, indicating polarization among participants.Renewable resources, also highly favored, showed a decrease between the post-test and delayed post-test, so that there was no net change from pre-test to delayed posttest.Only nuclear power exhibited significant mean changes overall, decreasing in preference immediately after the event and remaining lower than at the pre-test during interviews eight months after the event.However, the difference, less than 10% on the 11-point scale, could be considered minor.Overall, then, these data show few important changes in aggregate opinions, either immediately or after several months.
The analyses of changes within subjects also confirmed that many people did not change their views, although the interviews revealed that many participants felt they had learned important new information about Idaho's energy options.Thus, many people said new knowledge had not changed their minds, though it may have reinforced attitudes or added additional nuance.Such a finding is in line with persuasion theories like the Elaboration Likelihood Model, which posit that people who have high levels of interest in a topic and strong prior attitudes are unlikely to be swayed by persuasive appeals.This is especially true when balanced information is provided, because people can elaborate upon the elements that support their opinions and use that information to counter-argue opposing positions (Eagly & Kulesa 1997;Petty et al. 1999).Thus, there is a high probability that deliberative polling events focused on specific, narrow topics, because they will draw the "knowledgeable" participants, may not lead to sizeable attitude changes.
On the other hand, both the within-subjects analysis of the preference ratings and the interviews revealed that sizable percentages of people did undergo changes in attitudes, even reversals, especially for certain energy options.Interestingly, these changes were in line with those reported by Andersen and Hansen (2007), who reported that between 7 and 28% of Danish participants in a deliberative poll reversed their opinions on issues related to the Euro.These findings suggest that examining only aggregate data can provide, depending on circumstances, a sense of attitudinal change or stability that can be somewhat misleading.
Even among participants who did not change their attitudes, the interviews revealed how participation in the event led to more complex consideration and, arguably, to better informed attitudes.Though a handful of people noted that their initial positions were intractable, even well-informed people admitted that the process led them to consider different aspects and to recognize that the issues were not nearly so black-and-white as they had thought.These findings suggest structured engagement processes can enhance informed judgment, even among participants who are knowledgeable and have prior attitudes.
With regards to insights into short-versus long-term changes based on the within-subjects analysis, we found nearly equal degrees of change between the post-test and delayed post-test as between the pre-test and post-test.In other words, it does not appear that participating in the deliberative poll resulted in more stable attitudes.The literature is inconsistent about what types and magnitude of long-term changes might be expected.For instance, Andersen and Hansen (2007) found larger changes three months after a deliberative poll than immediately following the two days of deliberation.However, French and Laver (2009) found very substantial changes in attitudes immediately after a participatory event, while nine months later attitudes had regressed toward their initial levels.In our case, it does not appear that the deliberative poll itself had a large, lasting effect on attitudes.The interviews revealed how the complexity of the social and informational environment in which participants live interacted with, and sometimes countered, the effects of the information and deliberation they gained in the one-day event.Walmsley (2010) makes a similar point, that people's views about different policy options emerge largely from their particular life experiences and situations.Thus, it is important not to overstate the utility of deliberative processes in terms of either their short-or long-term effects on attitudes, although clearly such effects occur for some participants.
Through out experimental design, we had hoped to isolate the effects of the different components of a deliberative poll.If certain aspects (e.g., briefing documents or the expert panel) are more influential, knowing this could help streamline such processes and potentially save resources and time.Unfortunately, the small samples we had for each treatment prevent us from making firm conclusions about the efficacy of specific elements.According to persuasion theory, simply presenting facts may have limited effects on attitudes, and therefore one might have expected more attitude change in treatments with deliberative components than in treatments with simply factual presentations.However, this did not appear to be the case.For example, the full deliberative poll group did not exhibit any more attitudinal change than the group with only the conference.All the groups had the same hour-long presentation in the morning and attended the luncheon conference, which may account for some of the similarities; however, we agree with Fishkin and Luskin (2005) that additional research on the different components is needed.

Civic Engagement
The interviews support a strong, consistent finding that the deliberative poll did not increase civic engagement, even among the highly interested group we studied.Although participants did discuss the event with close friends and family, they did not encourage others to look into energy options or become active.Participants themselves were, for the most part, not inclined to seek out additional information about energy after the event.Only two people partook in any type of political activism, and both these individuals were people who tended to be active anyway.These findings are inconsistent with those of Eggins et al. (2007) and various proponents of deliberative polling (e.g., Fishkin & Rosell 2004), who concluded that participation increases civic engagement.Our findings are more similar to those of Andersen and Hansen (2007), who did not find much increase in political behavior.Therefore, we belief caution is needed in making claims for the effects of deliberative activities on broader civic engagement.

Conclusion
This study adds to our understanding of the long-term attitudinal and behavioral effects of deliberative democracy efforts, as well as the different interpretations that may arise from different approaches to examining attitude change.The findings, albeit limited by the specifics of the case, do suggest that attitude change is a complex phenomenon, and a variety of analytic techniques is required to understand how people process and reason with new information via deliberation with peers and experts.Additionally, it cannot be assumed that participation in the types of activities prescribed by deliberative polling will necessarily lead to attitude shifts among a majority of participants, much less that shifts will be in a consistent direction.Nevertheless, some individuals do undergo substantive attitudinal changes, as a result of exposure to balanced information and dialog with other citizens.Finally, our results suggest that the long-term effects of deliberative polling in promoting civic engagement may be quite limited.
Researchers, consequently, may need to be attentive to the potential to overstate the civic engagement value of deliberative polling exercises.The results suggest further that the question of efficacy requires refinement and further study.Policy makers considering the use of deliberative polls and related techniques may want to carefully weigh the costs of recruiting a representative sample, which can be quite high, against the modest achievement of desirable civic outcomes that are realistically attainable.For some issues and in some contexts, such expense of taxpayer, citizen, and expert time and resources may be quite worthwhile, but in other cases, they may not.

Table 1 .
Number of Interview Participants by Treatment Group Attitude ChangeWe examined attitude change from pre-test to post-test, from post-test to delayed post-test, and from pre-test to delayed post-test for all treatment groups combined.

Table 2 .
Mean Rating of Energy Options by Treatment, Pre-Test vs. Post-Test and Post-Test vs. Delayed Post-Test (Paired T-Tests)

Table 3 .
Percentage of Respondents (n=61) Exhibiting Substantive Attitude Changes between Pre-Test and Post-Test, by Treatment and Energy Option

Table 4 .
Percentage of Participants Exhibiting Changes in Ratings of Energy Options between Pre-test and Post-test (n=61)

Table 5 .
Percentage of Participants Exhibiting Changes in Ratings of Energy

Table 10 .
Percentage of Participants Reporting Various Forms of Civic Engagement after the Deliberative Event