Crowdsourcing civility: A natural experiment examining the effects of distributed moderation in online forums
Introduction
The literature examining the use of information and communication technology (ICT) for deliberation has often focused on the influences of medium on the ability of people to access and engage in deliberative arguments (Davis, 1999) or how technological tools can be incorporated to enhance deliberation (Kavanaugh & Isenhour, 2005). Particularly, in the context of public administration, it has been found that successful adoption of advanced information and communication technologies can lead to increased civic engagement, co-production, transparency, and more efficient and cost effective delivery of public services (Noveck, 2009). However, a consistent concern of public administrators in leveraging advanced information and communication technologies to interact with constituents is the potential for vitriolic messages by and between those participants who are using the channel. Even in non-political discussions there are angry “flame” messages, designed to lash out at participants; “troll” messages that offer falsely polemic positions to elicit anger from other users, or simply the inability to have civil discussion in online forums are all concerns for those who are evaluating social media sites for public interactions (Pfaffenberger, 2002, Sproull and Kiesler, 1991). If these kinds of off-topic, uncivil messages are pervasive in online communications, they can dilute the value of the conversation by making users hard to find more fact-based, reasoned messages. This issue is important in the context of public administration, particularly e-government, where public administrators seek to recognize and respond to citizens' voices on public issues through online discussions. If online mechanisms for public discussion are derailed because of hostile and uncivil messages, government agencies and participants may not benefit from information and communication technologies that aim to encourage deliberative and critical discussions among users. Online communities have a long history of examining technical and social responses to counter-productive behaviors in online discussions, which may be of use for informing the increased prevalence of these efforts to use information and communication technology to share and discuss the work of governance (Lea et al., 1992, Poor, 2005). In addition, these online communities often reach a massive scale of interactions, with tens and hundreds of thousands of interested parties interacting. How the designers and managers of these systems share pro-social interactions in such massively-distributed, identity-constrained environments could be useful for future efforts in this area. Many scholars argue that new IT technologies, such as rating or moderation systems, can reduce information overload and improve civility of participants, and thus lead to deliberative discussions in online discourse arenas. This is because those technologies can help participants focus on discussion topics by filtering unnecessary, uncivil messages and comments (Goldberg et al., 1992, Resnick et al., 1994, Terveen and Hill, 2001).
In this paper we examine how a popular online community, Slashdot, which is an online news and discussion site with a large, persistent membership, has used a system of distributed moderation to facilitate deliberation in its discussion forums. We also present how facilitating political public discourse through social media ventures can contribute to public-to-government, government-to-public, and public-to-public engagement. By looking at previous work regarding necessary conditions for successful online communities, our goal is to define which conditions map to successful outcomes for communities that have a civic or public administration emphasis. As organizations at all levels better understand the barriers, opportunities, and design consequences in adopting social media technologies, their increased capacity and efficacy may affect the potential benefits such technologies allow.
Section snippets
Public administration goes digital
The use of information technology to exchange information and provide services from government organizations to citizens, businesses, and other branches of government has resulted in an increased interest among public administration scholars in e-government. The aim of e-government is not simply to reinforce existing forms of interactions, but to create new forms of participation that improve the relationship between the public and the government to increase government efficiency,
Slashdot as a research environment
Slashdot5 is a news and commentary site dedicated to technology issues, including when technology and policy intersect. Founded in 1997, it's one of the first sites to attract large-scale participation, and was an early example of the types of user-generated content that became characteristic of social media and online discussion forums. The site falls into a class of online forums referred to as “news aggregators”, meaning that they aggregate and list topical news
Research methods
Slashdot automatically records data whenever a user requests a page, posts a comment, or rates another's comment. These usage logs included information for each comment, moderation and meta-moderation that took place. User data included the karma scores of users and whether they were regular users or paid editors. Two datasets were constructed from these usage logs. The first dataset consisted of apolitical stories extending from May 31, 2003 through July 30, 2003. The dataset includes 293,608
Results
Our first research question asks how the moderator power is distributed among Slashdot users; and we found that participation is widespread throughout the system. During the two month period, 24,069 distinct users moderated with a median number of 7 (mean of 13) moderations per moderator. Because the system deliberately limited the amount of moderation any individual can perform, the maximum number of moderations completed by anyone other than paid staff was 164, less than three per day out of
Discussion
Slashdot provides an interesting example of using the distributed moderation system based on ratings to promote reasoned, civil discussions in online forums. As mentioned earlier, Slashdot users can achieve moderator eligibility by obtaining positive reputation scores, “karma scores” that are given to the users when they read threads regularly and their own comments get positive responses from moderators. In other words, when the users are more involved in the online discussions in a more
Conclusion
As our research has shown, the ability to have moderated, massively large and online political discussion spaces is both feasible and sustainable. Slashdot provides an existence proof that the basic idea of distributed moderation with widespread participation is sound. There seems to be a broad, though not perfect consensus about which comments deserve to be moderated up or down suggesting that the quality of the comment is more important than the position of the comment. Comment scores are
Cliff Lampe is an assistant professor in the School of Information at the University of Michigan. He studies the design and effects of online environments that support collaboration.
References (51)
- et al.
The problem of information overload in business organizations: A review of the literature International
Journal of Information Management
(2000) - et al.
The market for evaluations
American Economic Review
(1999) - et al.
A theory of fads, fashion, custom and cultural change as informational cascades
Journal of Political Economy
(1989) - et al.
Managing deviant behavior in online communities
Internet politics: States, citizens, and new communication technologies
(2006)- et al.
Public life and the Internet: If you build a better website, will citizens become more engaged?
New Media and Society
(2008) Mudding: Social phenomena in text-based virtual realities
Paper presented at the Conference on Directions and Implications of Advanced Computing, Berkeley, CA
(1992)- et al.
MUDs grow up: Social virtual reality in the real world
Presented at COMPCON
(1993) The web of politics: The Internet's impact on the American political system
(1999)- et al.
Opinion space: A scalable tool for browsing online comments
Understanding Deja reviewers
Using collaborative filtering to weave an information tapestry
Communications of the ACM
Searching for safety online: Managing “trolling” in a feminist forum
The Information Society
Networking in everyday life
Cyberville: Clicks, culture, and the creation of an online town
The Internet and democratic debate
E-democracy, an illusion to be pursued but never attained
User population and user contributions to virtual publics: A systems model
An empirical exploration of mass interaction system dynamics: Individual information overload and Usenet discourse
Information overload and the message dynamics of online interaction spaces: A theoretical model and empirical exploration
Information Systems Research
Information technology in support of public deliberation
Getting political on social network sites: Exploring online political discourse on Facebook
First Monday
Ratings use in an online discussion system: The Slashdot case
It's all news to me: The effect of instruments on ratings provision
Slash(dot) and burn: Distributed moderation in a large online conversation space
Cited by (112)
Managing the manosphere: The limits of responsibility for government social media adoption
2024, Government Information QuarterlyMapping HCI research methods for studying social media interaction: A systematic literature review
2022, Computers in Human BehaviorCivility: A concept analysis revisited
2022, Nursing OutlookContagion of offensive speech online: An interactional analysis of political swearing
2022, Computers in Human BehaviorPersonal attacks decrease user activity in social networking platforms
2022, Computers in Human BehaviorThe effect of crowdsourcing on organizational learning: Evidence from local governments
2021, Government Information Quarterly
Cliff Lampe is an assistant professor in the School of Information at the University of Michigan. He studies the design and effects of online environments that support collaboration.
Paul Zube is a doctoral student in the Media and Information Studies Program at Michigan State University. He is an adjunct professor at Ferris State University. He studies technical features that support online deliberation.
Jusil Lee is a doctoral student in the School of Public Affairs at Arizona State University. He studies how open information systems can be used to improve political processes.
Chul Hyun Park is a doctoral student in the School of Public Affairs at Arizona State University. He studies the intersection of information systems and e-government.
Erik Johnston is an associate professor in the School of Public Affairs at Arizona State University. He brings a complex systems approach to the study of public administration, and studies how information systems affect policy making.