Response to ' Lessons from the Virtual Agora Project '

In a recent study, Peter Muhlberger and Lori Weber found strong evidence that participants in public deliberation acquired substantial new knowledge of the issues they had prepared to discuss. However, participants in their “control” group—a group that did not discuss the issue—learned just as much as those who did discuss it. The authors concluded that, by itself, discussion did not significantly increase overall decision knowledge. Nor did discussing the issue add substantially to the change in policy views they observed. Just reading and thinking about the information they were given produced the gain in knowledge and changes in attitudes that the post-event surveys revealed. To their credit, Muhlberger and Weber note that their study focuses narrowly on the acquisition of factual “decision knowledge.” They caution that their findings do not address whether discussion promoted the acquisition of knowledge other than factual knowledge. The authors also anticipate a criticism when they concede that they did not first “establish that participants ‘really’ deliberated.” They argue that their “approach to insuring deliberation...is typical for many studies, including those that purportedly show the learning effects of deliberation....” Though understandable, this acceptance of the prevailing research conception of deliberation takes for granted precisely what needs to be the focus of critical investigation. Whether deliberation occurred is a question whose answer observers too often assume is “yes.” In fact, there has been very little discussion as to what the essential characteristics of deliberation are. Hence it is difficult to say we know when deliberation has occurred. Nor has there been much discussion concerning how to determine the quality or the effectiveness of deliberation. Until discussion of such fundamental questions produces answers that are accepted widely and to general satisfaction, even exemplary empirical studies such as Muhlberger and Weber’s will be less illuminating and less useful for practice than they might be. The authors do suggest a number of useful lessons for practice.

Lessons from the Virtual Agora Project: The Effects of Agency, Identity, Information, and Deliberation on Political Knowledge * Peter Muhlberger and Lori M. Weber

Deliberation and Learning
Deliberation about issues of public policy lies at the heart of the notion that decisionmaking in a democracy should be an activity in which citizens establish priorities and set directions for policy by engaging each other in reasoned political discourse.Deliberation is the key to arriving at public decisions that will be widely regarded as both fair and sound.
An essential element of deliberation is learning: acquiring factual information pertinent to an issue.Deliberation depends on the possession of such knowledge, for without it deliberation's other tasks-weighing, judging, choosing, negotiating-can't be performed properly.
Peter Muhlberger is a political scientist at the University of Pittsburgh.He and his colleague, Lori Weber, of Cal State-Chico, observe that, although previous research has shown that deliberation does help educate citizens about political issues, this research hasn't looked into the question of which aspect(s) of deliberation make the biggest contribution to the learning that occurs when citizens participate in the various activities that make up the deliberative process.
In particular, Muhlberger and Weber point out, no one has tried to determine whether deliberative talk itself plays the chief role in helping citizens learn about an issue.For example, maybe simply reading information such as pre-forum policy briefs plays as great a role.Nor has previous research examined the question of whether characteristics of the participants or of the deliberative setting contribute as much to learning as does deliberative talk per se.
The authors have attempted to shed light on these matters by analyzing data from preand post-event surveys of a representative sample of 568 Pittsburgh residents who participated in a one-day deliberation (some face-to-face, others on-line).To answer their questions, Muhlberger and Weber employed a number of sophisticated statistical procedures, which, as a gesture of sympathy for the low threshold of fascination most people have for the intricacies of methodology, I will refrain from discussing here.

Five Findings
What did the authors discover?1.To begin with, Muhlberger and Weber found strong evidence that participants did in fact acquire substantial new knowledge of the four issues they had prepared to discuss.
After the event, participants were asked questions about the issues.On average, participants answered 73 percent of the questions correctly.The authors point out that almost none of the questions had answers that participants could have picked up from reading the newspaper or watching the news on TV.Rather, participants could have found the answers only in the policy reports and expert testimony they received prior to the actual deliberation.(Several participants commented that they were astonished by how poor and incomplete the information was that they had gotten through the media, compared with the content of the policy briefs they were given.) 2. The authors' second finding was that the "decision knowledge" gained by participants affected both participants' attitudes and their opinions of each other.The people who had high levels of decision knowledge-those who learned the most-showed significantly greater change in their policy views.Participants who acquired more knowledge than their fellow discussants had a greater impact on the policy attitudes of other group members than did those who demonstrated a lower level of knowledge.
3. Perhaps the most important discovery made by Muhlberger and Weber is that participants in their "control" group-a group that did not discuss the issue-learned just as much as those who did discuss it.The authors conclude from this that, by itself, discussion did not significantly increase overall decision knowledge.Nor did discussing the issue add substantially to the change in policy views they observed.Just reading and thinking about the information they were given produced the gain in knowledge and changes in attitudes that the post-event surveys revealed.
Muhlberger and Weber are quick to point out, however, that the findings of their study do not show that "deliberation," broadly defined, does not affect decision knowledge.Rather, the conclusion they draw is only that there is no evidence of an effect above that of readings and individual reflection on the information participants received.
4. In fact, the authors contend that the data do show that deliberation helps people learn.More precisely, for certain people deliberation promotes learning.Muhlberger and Weber looked at the effect on learning of two conceptions of citizenship.The first, which the authors label "authoritarian" (which may be an accurate characterization, though I wish they had chosen a less-pejorative term, such as "passive") involves the belief that a good citizen is one who obeys the law, shows deference and respect to political authorities, and approves punishment of those who fail to do these things.As might be expected, participants who hold this view of citizenship had the lowest levels of knowledge-acquisition.
The second conception is "active" (I'd call it "assertive"): a citizen's responsibilities go beyond obeying the law; they extend to active engagement with other citizens, including public officials, for the purpose of making sound and fair policy decisions.Again, as might be expected, participants who hold this view of citizenship had the highest levels of knowledge-acquisition.
Muhlberger and Weber believe the active conception of citizenship actually consists of two sub-types: (i) a non-deliberative form (e.g., voting, contacting public officials, making contributions to political groups), and (ii) a deliberative form (e.g., discussing a problem or issue with other citizens).The authors found that participants in the deliberative subcategory showed high levels of "political reflectiveness."Political reflectiveness has to do with the extent to which people feel personally responsible for forming their own political views.The more politically reflective people are, the more knowledge they gain.5. Finally-and this is related to point 3, above-the authors observe that the prospect of deliberation is crucial for motivating people to learn about a particular policy issue.Although actually deliberating had no effect on knowledge-gain above and beyond the effect of reading and thinking, the anticipation of deliberation may have promoted learning through reading and thinking about the materials received before the event.In part because some participants in the control group were very disappointed not to be able to discuss the issue, the authors believe far fewer control participants would have attended the event if they had known in advance that they wouldn't have a chance to talk about it.

Implications for Practice
What should practitioners of deliberative discourse take away from Muhlberger and Weber's research findings?What are the lessons for practice?Here are eight: 1.When a deliberative event is planned, well-chosen reading material should be made available to anyone who might participate.Reading has more of an impact on people's ability to acquire knowledge relevant to a policy issue than does deliberation itself.
2. A strong effort should be made to ensure that people read the materials they are provided.Allowing time for reading during the deliberative event itself will maximize the learning effect of the deliberative experience and might reduce inequalities in the amount of political knowledge different participants can draw on.
3. Giving people reason to believe that they will have an opportunity to deliberate with others is an important way to motivate them to prepare by reading information relevant to the issue.If people can't be counted on to read beforehand, though, time should be set aside for reading as part of the deliberative event.There should be enough time for participants to get through the material.(In the authors' study, the time allocated to reading was almost half as much as the time devoted to discussion time.) 4. If the chief aim of a public deliberation is to engage and inform as much of the public as possible, then finding ways to get large numbers of people to read may have a broader impact on the issue than holding big-meeting deliberations.But if the primary goal is to create a sense of community, build relationships, enhance people's personal sense of citizenship, or raise the level of active involvement, then smaller deliberations may be more useful.
5. Reminding participants of their roles as citizens enhances learning, but only if participants won't be discussing the issue together or, if they will, the discussion will take place on-line.Reminding participants of their roles as citizens does not enhance learning in face-to-face settings.Talking face-to-face with others inhibits people from disagreeing (because of politeness, for example, or out of fear of experiencing conflict), which may have a negative effect on learning.In contrast, on-line discussion lets participants feel freer to speak candidly, which in turn may enhance learning.
6. Challenging participants to question the number of people who share their views may also promote learning.One way to do this is to provide participants with factual information showing how diverse the public's views actually are.Another is to have the group adopt as a goal that participants will learn from each other.
7. Political reflectiveness might be strengthened by instructions to participants concerning how they should learn.For example, they might be asked to focus on the information their readings provide, rather than on whether different views are right or wrong.
"effective"-produces answers that are accepted widely and to general satisfaction, even exemplary empirical studies such as Muhlberger and Weber's will be less illuminating and less useful for practice than they might be.