Crowdsourcing civility: A natural experiment examining the effects of distributed moderation in online forums

https://doi.org/10.1016/j.giq.2013.11.005Get rights and content

Highlights

  • We describe distributed moderation to increase the civility in online discussion.

  • We discuss political discussion in a non-political forum.

  • We argue for more sophisticated moderation in online discussion.

  • We show effects of distributed moderation on civility.

Abstract

Participation in discussions about the public interest can be enhanced by technology, but can also create an environment in which participants are overwhelmed by the quantity, quality, and diversity of information and arguments. Political participation is at a greater disadvantage than non-political activities in that participants from different parties already start out with established differences, which requires them to reach some form of common ground before progress can be made. Those seeking authentic deliberation are discouraged to participate when confronted with uncivil and inflammatory rhetoric. These issues are often exacerbated in online discussions, where lack of identity cues and low barriers to entry can lead to heightened incivility between participants, often labeled as “flaming” and “trolling”. This paper explores the extent to which moderator systems, tools online discussion forums use to manage contributions, can reduce information overload and encourage civil conversations in virtual discussion spaces. Using the popular website Slashdot as an example of sound moderation in a public discourse setting, we found that users move toward consensus about which and how comments deserve to be moderated. Using these findings, we explore how transferable these systems are for participation in public matters specifically to the unique attributes of political discussion. Slashdot's political forum provides a comparison group that allowed us to find quantitative and qualitative differences in political posting, comments, and moderation. Our results show that large scale, civil participation is possible with a distributed moderation system that enables regularly lively debates to be conducted positively because the system provides tools for people to enforce norms of civility.

Introduction

The literature examining the use of information and communication technology (ICT) for deliberation has often focused on the influences of medium on the ability of people to access and engage in deliberative arguments (Davis, 1999) or how technological tools can be incorporated to enhance deliberation (Kavanaugh & Isenhour, 2005). Particularly, in the context of public administration, it has been found that successful adoption of advanced information and communication technologies can lead to increased civic engagement, co-production, transparency, and more efficient and cost effective delivery of public services (Noveck, 2009). However, a consistent concern of public administrators in leveraging advanced information and communication technologies to interact with constituents is the potential for vitriolic messages by and between those participants who are using the channel. Even in non-political discussions there are angry “flame” messages, designed to lash out at participants; “troll” messages that offer falsely polemic positions to elicit anger from other users, or simply the inability to have civil discussion in online forums are all concerns for those who are evaluating social media sites for public interactions (Pfaffenberger, 2002, Sproull and Kiesler, 1991). If these kinds of off-topic, uncivil messages are pervasive in online communications, they can dilute the value of the conversation by making users hard to find more fact-based, reasoned messages. This issue is important in the context of public administration, particularly e-government, where public administrators seek to recognize and respond to citizens' voices on public issues through online discussions. If online mechanisms for public discussion are derailed because of hostile and uncivil messages, government agencies and participants may not benefit from information and communication technologies that aim to encourage deliberative and critical discussions among users. Online communities have a long history of examining technical and social responses to counter-productive behaviors in online discussions, which may be of use for informing the increased prevalence of these efforts to use information and communication technology to share and discuss the work of governance (Lea et al., 1992, Poor, 2005). In addition, these online communities often reach a massive scale of interactions, with tens and hundreds of thousands of interested parties interacting. How the designers and managers of these systems share pro-social interactions in such massively-distributed, identity-constrained environments could be useful for future efforts in this area. Many scholars argue that new IT technologies, such as rating or moderation systems, can reduce information overload and improve civility of participants, and thus lead to deliberative discussions in online discourse arenas. This is because those technologies can help participants focus on discussion topics by filtering unnecessary, uncivil messages and comments (Goldberg et al., 1992, Resnick et al., 1994, Terveen and Hill, 2001).

In this paper we examine how a popular online community, Slashdot, which is an online news and discussion site with a large, persistent membership, has used a system of distributed moderation to facilitate deliberation in its discussion forums. We also present how facilitating political public discourse through social media ventures can contribute to public-to-government, government-to-public, and public-to-public engagement. By looking at previous work regarding necessary conditions for successful online communities, our goal is to define which conditions map to successful outcomes for communities that have a civic or public administration emphasis. As organizations at all levels better understand the barriers, opportunities, and design consequences in adopting social media technologies, their increased capacity and efficacy may affect the potential benefits such technologies allow.

Section snippets

Public administration goes digital

The use of information technology to exchange information and provide services from government organizations to citizens, businesses, and other branches of government has resulted in an increased interest among public administration scholars in e-government. The aim of e-government is not simply to reinforce existing forms of interactions, but to create new forms of participation that improve the relationship between the public and the government to increase government efficiency,

Slashdot as a research environment

Slashdot5 is a news and commentary site dedicated to technology issues, including when technology and policy intersect. Founded in 1997, it's one of the first sites to attract large-scale participation, and was an early example of the types of user-generated content that became characteristic of social media and online discussion forums. The site falls into a class of online forums referred to as “news aggregators”, meaning that they aggregate and list topical news

Research methods

Slashdot automatically records data whenever a user requests a page, posts a comment, or rates another's comment. These usage logs included information for each comment, moderation and meta-moderation that took place. User data included the karma scores of users and whether they were regular users or paid editors. Two datasets were constructed from these usage logs. The first dataset consisted of apolitical stories extending from May 31, 2003 through July 30, 2003. The dataset includes 293,608

Results

Our first research question asks how the moderator power is distributed among Slashdot users; and we found that participation is widespread throughout the system. During the two month period, 24,069 distinct users moderated with a median number of 7 (mean of 13) moderations per moderator. Because the system deliberately limited the amount of moderation any individual can perform, the maximum number of moderations completed by anyone other than paid staff was 164, less than three per day out of

Discussion

Slashdot provides an interesting example of using the distributed moderation system based on ratings to promote reasoned, civil discussions in online forums. As mentioned earlier, Slashdot users can achieve moderator eligibility by obtaining positive reputation scores, “karma scores” that are given to the users when they read threads regularly and their own comments get positive responses from moderators. In other words, when the users are more involved in the online discussions in a more

Conclusion

As our research has shown, the ability to have moderated, massively large and online political discussion spaces is both feasible and sustainable. Slashdot provides an existence proof that the basic idea of distributed moderation with widespread participation is sound. There seems to be a broad, though not perfect consensus about which comments deserve to be moderated up or down suggesting that the quality of the comment is more important than the position of the comment. Comment scores are

Cliff Lampe is an assistant professor in the School of Information at the University of Michigan. He studies the design and effects of online environments that support collaboration.

References (51)

  • A. Edmunds et al.

    The problem of information overload in business organizations: A review of the literature International

    Journal of Information Management

    (2000)
  • C. Avery et al.

    The market for evaluations

    American Economic Review

    (1999)
  • S. Bikhchandani et al.

    A theory of fads, fashion, custom and cultural change as informational cascades

    Journal of Political Economy

    (1989)
  • A. Bruckman et al.

    Managing deviant behavior in online communities

  • A. Chadwick

    Internet politics: States, citizens, and new communication technologies

    (2006)
  • R. Coleman et al.

    Public life and the Internet: If you build a better website, will citizens become more engaged?

    New Media and Society

    (2008)
  • P. Curtis

    Mudding: Social phenomena in text-based virtual realities

    Paper presented at the Conference on Directions and Implications of Advanced Computing, Berkeley, CA

    (1992)
  • P. Curtis et al.

    MUDs grow up: Social virtual reality in the real world

    Presented at COMPCON

    (1993)
  • R. Davis

    The web of politics: The Internet's impact on the American political system

    (1999)
  • S. Faridani et al.

    Opinion space: A scalable tool for browsing online comments

  • E. Gilbert et al.

    Understanding Deja reviewers

  • D. Goldberg et al.

    Using collaborative filtering to weave an information tapestry

    Communications of the ACM

    (1992)
  • S.C. Herring et al.

    Searching for safety online: Managing “trolling” in a feminist forum

    The Information Society

    (2002)
  • B.J. Hogan

    Networking in everyday life

    (2009)
  • S. Horn

    Cyberville: Clicks, culture, and the creation of an online town

    (1998)
  • J. Horrigan et al.

    The Internet and democratic debate

    (2004)
  • M. Huisman

    E-democracy, an illusion to be pursued but never attained

  • Q. Jones et al.

    User population and user contributions to virtual publics: A systems model

  • Q. Jones et al.

    An empirical exploration of mass interaction system dynamics: Individual information overload and Usenet discourse

  • Q. Jones et al.

    Information overload and the message dynamics of online interaction spaces: A theoretical model and empirical exploration

    Information Systems Research

    (2004)
  • A. Kavanaugh et al.

    Information technology in support of public deliberation

  • M.J. Kushin et al.

    Getting political on social network sites: Exploring online political discourse on Facebook

    First Monday

    (2009)
  • C. Lampe

    Ratings use in an online discussion system: The Slashdot case

    (2006)
  • C. Lampe et al.

    It's all news to me: The effect of instruments on ratings provision

  • C. Lampe et al.

    Slash(dot) and burn: Distributed moderation in a large online conversation space

  • Cited by (112)

    View all citing articles on Scopus

    Cliff Lampe is an assistant professor in the School of Information at the University of Michigan. He studies the design and effects of online environments that support collaboration.

    Paul Zube is a doctoral student in the Media and Information Studies Program at Michigan State University. He is an adjunct professor at Ferris State University. He studies technical features that support online deliberation.

    Jusil Lee is a doctoral student in the School of Public Affairs at Arizona State University. He studies how open information systems can be used to improve political processes.

    Chul Hyun Park is a doctoral student in the School of Public Affairs at Arizona State University. He studies the intersection of information systems and e-government.

    Erik Johnston is an associate professor in the School of Public Affairs at Arizona State University. He brings a complex systems approach to the study of public administration, and studies how information systems affect policy making.

    View full text