Understanding the paradigm shift to computational social science in the presence of big data

Abstract

The era of big data has created new opportunities for researchers to achieve high relevance and impact amid changes and transformations in how we study social science phenomena. With the emergence of new data collection technologies, advanced data mining and analytics support, there seems to be fundamental changes that are occurring with the research questions we can ask, and the research methods we can apply. The contexts include social networks and blogs, political discourse, corporate announcements, digital journalism, mobile telephony, home entertainment, online gaming, financial services, online shopping, social advertising, and social commerce. The changing costs of data collection and the new capabilities that researchers have to conduct research that leverages micro-level, meso-level and macro-level data suggest the possibility of a scientific paradigm shift toward computational social science. The new thinking related to empirical regularities analysis, experimental design, and longitudinal empirical research further suggests that these approaches can be tailored for rapid acquisition of big data sets. This will allow business analysts and researchers to achieve frequent, controlled and meaningful observations of real-world phenomena. We discuss how our philosophy of science should be changing in step with the times, and illustrate our perspective with comparisons between earlier and current research inquiry. We argue against the assertion that theory no longer matters and offer some new research directions.

Introduction

With the rapid advances in technology, business interactions involving consumers and suppliers now generate vast amounts of information, which make it much easier to implement the kinds of data analytics that Gary Loveman, current CEO of Caesar's Entertainment, discussed in a 2003 Harvard Business Review article on data mining [70]. Today, this is referred to as the big data revolution in the popular press, and viewed as creating challenges and opportunities for business leaders and interdisciplinary researchers. The world's volume of data doubles every eighteen months, for example, and enterprise data are predicted to increase by about 650% over the next few years [45], [54]. Today, most firms have more data than they can handle, and managers recognize the potential for value, but the promise of big data still has not been realized, according to the leading academic [35], [78] and business media sources [38], [79].³ The potential arises from the use of data to support the way organizations operate and serve their stakeholders. A recent article in MIT Sloan Management Review [62] described the use of big data by an Atlanta-based public school, for example. High school graduation rates increased due to better-informed policy decisions that were based on the application of advanced analytics capabilities to student performance data. Likewise, organizations now are embedding analytics in their operations to support data-intensive strategies.

A recent McKinsey report has referred to big data as “data sets whose size is beyond the ability of typical database software tools to capture, store, manage, and analyze” [71].⁴ Such data come from everywhere: pictures and videos, online purchase records, and geolocation information from mobile phones. Big data are not just about sheer volume in terabytes though. Other important aspects have been emphasized in addition to volume, including variety, velocity and value [76]. Big data may be unstructured too: examples are text with social sentiments, audio and video, click streams, and website log files. Such data may flow in real-time streams for analysis, which can enable a firm to maximize business value by supporting business decisions in near to real-time. This new trend in decision support is evocative of what we saw in the 1990s with the emergence of data mining, and the new emphasis on data with a large number of dimensions and much higher complexity (e.g., spatial, multimedia, XML and Internet data). Most of the data sets were “one off” opportunities, rather than data that had become available due to systemic and technological advances.

Considerable challenges are present in the quest to capture the full potential of big data. The shortage of analytics and managerial talent is a significant and pressing problem, for example. CIO Magazine [72] and the Corporate Executive Board [79] have reported that it is difficult for firms to find the right people. The U.S. alone is reported to face a shortage of 140,000 to 190,000 people with deep analytical skills, as well as 1.5 million managers and analysts to make effective decisions [71]. (See Fig. 1.)

New perspectives in social science are now tracking the developments in big data. For example, computational organization science has broadened researchers' perspectives on social, organizational and policy systems, by adopting computational models that combine social science, computer science, and network science [22]. Other related developments have occurred, including the emergence of computational social science and e-social science [37], [63]. Computational social science involves interdisciplinary fields that leverage capabilities to collect and analyze data with an unprecedented breadth, depth, and scale. Computational modeling approaches now can predict the behavior of sociotechnical systems, such as human interactions and mobility, that were previously not studied with one-time snapshots of data for very many people [83]. We see a paradigm shift in scientific research methods — and one that prompts new directions for research. A useful perspective in this context is attributable to Runkel and McGrath [75], who characterized research methodologies based on three goals: generality, control and realism. They distinguished between their obtrusiveness and unobtrusiveness for the subjects of research.

With emerging collection techniques for big data sets, there seem to be fundamental changes that are occurring related to research methods, and the ways they can be applied too [58]. In e-business, for example, the contexts include social networks, blogs, mobile telephony, and digital entertainment. The new approaches we see are based on more advantageous costs of data collection, and the new capabilities that researchers have to create research designs that were hard to implement before. The research contexts include human and managerial decision-making, consumer behavior, operational processes, and market interactions. The result is a change in our ability to leverage research methodology to achieve control and precision in measurement, while maintaining realism in application and generality in theory development.

We will discuss the causes of the paradigm shift, and explore what it means for decision support and IS research, and more broadly, for the social sciences. How can we take advantage of big data in our research? What new perspectives are needed? What will the new research practices look like? What kinds of scientific insights and business value can they deliver in comparison to past research? And what research directions are likely to be especially beneficial for the production of new knowledge?

Section 2 reviews traditional methods for research and discusses the key factors that are creating the basis for a paradigm shift. Section 3 describes the new paradigm in the era of big data, and how it relates to decision support, IS and social science research. Section 4 assesses how the research has been changing, through the use of a set of specific comparisons between research that was conducted before and after the emergence of new methods associated with big data. Section 5 offers some new research directions, and section 6 concludes.