Elsevier

Computer Networks

Volume 75, Part B, 24 December 2014, Pages 560-577
Computer Networks

Mixed methods analysis of enterprise social networks

https://doi.org/10.1016/j.comnet.2014.08.025Get rights and content

Abstract

The increasing use of enterprise social networks (ESN) generates vast amounts of data, giving researchers and managerial decision makers unprecedented opportunities for analysis. However, more transparency about the available data dimensions and how these can be combined is needed to yield accurate insights into the multi-facetted phenomenon of ESN use. In order to address this issue, we first conducted a systematic literature review to identify available data dimensions and integrated them into a conceptual framework. We then adopted this framework as part of a mixed methods research approach to comprehensively analyze an empirical ESN case. With our results serving as a proof of concept we show the insights that can be derived from different data dimensions and how combining these can improve the validity of the analysis. The application of the framework also allows us to derive a detailed guideline for combining different data sources in ESN analysis to support researchers and decision makers.

Introduction

Over the past few years, many companies have witnessed their employees moving toward more flexibility and self-determination, especially in terms of spatial and time-independent work [1], [2]. Systems that support distributed and networked collaboration, such as enterprise social networks (ESN), have been playing an increasingly important role in changing communication practices [3], [4], [5]. Based on previous research [6], [7], [8], [9], we define ESN as web-based intranet platforms that support users in contributing persistent objects to a shared pool, which enables public responses to these objects, allows profile information to be presented, and connects users via features like Following, or Friendship request. Examples of widely adopted platforms include weblogs, wikis, microblogs, and social networking platforms, which tend to converge into an integrated ESN [10].

The increasing use of ESN produces a considerable amount of data, since almost every interaction in the system leaves a persistent digital trace [3]. This “revolution in the measurement of collective human behavior” [11 p. 66] gives researchers and managerial decision makers unprecedented opportunities to analyze and explain such systems.

The comprehensive variety of relevant ESN data is evidenced, for example, by studies on patterns of information exchange [12], [13] and their underlying network structures [14], [15], the distribution of ideas [16], rumors [17] and sentiments [18], as well as their representation [19]. Other studies have shed light on social technology’s effects on employee performance [20], [21], hierarchy [22], contribution behavior [23], and trust [24].

Whereas existing studies have led to insightful results, they have all applied only partial views based on different individual data dimensions and research approaches. However, the abundant data suggests that systematically combining and triangulating different data dimensions will lead to more robust or even novel insights for researchers or managers. For example, conducting an additional sentiment analysis of the postings and qualitative user interviews might support interpreting dissemination measures. In addition to the challenge of combining multiple such data sources, consistent academic theory development faces important methodological challenges. Researchers have argued that the application of current social network theories to online social networks is limited in that new and unknown aspects could affect the results of ESN analysis [10]. Trier [25], also emphasizes online social interaction’s high volatility and its resulting fast structural changes. Furthermore, the digital traces within an ESN are of a different nature than the data analyzed in earlier social network studies and may lack validity under certain aspects [26].

To address the above issues and in order to achieve consistent insights across future ESN research projects, more transparency of data dimensions and their interrelationships is required. This gap motivates our first research question (RQ):

  • RQ1. Which data dimensions are adopted in existing studies on enterprise social networks?

Contrary to the abundance of available data, analytical metrics, and perspectives on the data, there is a paucity of research that systematically develops analytical methods that effectively utilize this rich resource to inform academic inquiry and managerial decision making. In order to analyze ESN data, we explore and develop a conceptual framework as part of a dedicated mixed methods approach [27], [28] to address the diversity of data and the critical validity of online data traces.

Mixed methods research builds on a combination of qualitative and quantitative methods [29] and is particularly appropriate for domains marked by diversity [30]. In our context, this approach is appropriate for several reasons: ESN are a multi-dimensional and complex domain [31] that only one method is unlikely to capture appropriately. Basic social network analysis (SNA) alone is particularly insufficient due to new user behavior in online networks [5], [32], [10]. Furthermore, mixing methods allows researchers to draw on the strength of the methods used and to offset their weaknesses [33], [34]. It also helps to clarify the results that one method provides with the results of others [34], which expands the results and enhances the integrity of the findings [33]. To contribute to a better understanding of ESN analysis, our second research question thus focusses on a first step toward a dedicated mixed ESN research method approach:

  • RQ2. What are the implications of combining different data dimensions in a mixed methods approach in the context of an enterprise social network analysis?

By answering these questions, our contributions are threefold: We first identify relevant data dimensions based on a systematic literature review and suggest a framework to improve the transparency of the available options for analysis, which can serve as a basis for future ESN analysis. In order to not limit our contribution to conceptual work, we further apply and systematically evaluate our framework empirically, using an extensive and multi-dimensional case analysis of an ESN for military medical personnel as a proof of concept. To our knowledge, there is as yet no such empirical investigation of different data dimensions by means of a case study. In the context of this paper, we only reflect on the methodological aspects of the case study and its different data dimensions. Based on the empirical case analysis, we shed light on the unexploited potential, but also on the limitations of a mixed methods approach in the context of an ESN analysis. Third, to exploit the benefits and to address the limitations of this approach, we develop a set of guidelines for conducting an effective mixed methods analysis of the identified data dimensions that will be helpful to researchers and practitioners alike. Fig. 1 summarizes our research questions and contributions.

As we specifically intend to combine a theory-based concept development and an empirical proof of concept, the remainder of this paper is structured as follows: The next section contains the results of a structured literature review of current research into social network analysis, focusing on the research methods and utilized data dimensions. From this review, we systematically categorize all available data dimensions. Thereafter, in Section 4, we assess our framework [36] and investigate the identified dimensions and their interplay in an empirical case study, using a dedicated mixed methods approach. The presentation of the analysis results follows in Section 5. Based on the empirical results, our approach is discussed in Section 6; we also discuss mixed methods analysis’s observed issues, as well as present guidelines that address these issues, before we conclude this paper.

Section snippets

Identifying data dimensions in ESN research

As noted in the introduction, the existing studies on the multi-facetted phenomenon of ESN use are based on a variety of data sources and methods of data collection. Currently, there seems to be a lack of transparency about the available options and likely implications of concrete decisions on the sources that should be adopted in a study. To address this gap, we first conducted a systematic literature review, which aimed to identify, categorize, and conceptualize the variety of research

Case study

Our selected case organization is the medical service unit of the German Armed Forces (Deutsche Bundeswehr). It employs, amongst others, 2700 medical officers and 1600 trainee medical officers assigned to military medicine, military pharmacy, veterinary medicine, or dental medicine. Contrary to most officers who study at one of the two Bundeswehr universities in Munich and Hamburg, these employees study medicine at civil universities across Germany. The workforce and the students are

Methods and data collection

In the following sections, we describe the methods used in the case study. The theory section motivates these methods, which are based on the framework in Table 2. Later, we describe in detail the data dimensions used and how they were collected in our case setting.

Analysis results

This section presents selected results of our mixed methods approach, using our proposed data dimensions as the main structure. Elements are selected for presentation if they serve to illustrate the interplay between the different methods and their results.

Discussion

Based on our empirical inquiry, we will now refer to our initial research questions and discuss the respective results.

Conclusion

Communication within a networked organization creates a large variety of data, for example, usage data, user-generated data, structural data, and user perceptions. The analysis of this data yields valuable insights for relevant business decisions. The applicable analytical methods and the questions that these analyses can answer are equally diverse. In this paper, we proposed a framework of four basic data dimensions and identified their key characteristics. We argued that a mixed methods

Sebastian Behrendt is a Ph.D. candidate at Cooperation Systems Center at Bundeswehr University Munich. He graduated in Information Management at the University of Applied Science and Arts Hannover in 2006. After that, he worked several years as a consultant and project manager in the internet industry before going back to the academics. Now he is focusing on the evaluation of social software in organizations. In a current research project he analyzes the development of informal networks in such

References (79)

  • A. Bharadwaj et al.

    Visions and voices on emerging challenges in digital business strategy

    MIS Quart.

    (2013)
  • M. McLure Wasko et al.

    Why should I share? Examining social capital and knowledge contribution in electronic networks of practice

    MIS Quart.

    (2005)
  • D.M. Boyd et al.

    Social network sites: definition, history, and scholarship

    J. Comput.-Mediated Commun.

    (2007)
  • R. Agarwal et al.

    Editorial overview—the interplay between digital and social networks

    Inform. Syst. Res.

    (2008)
  • A. Richter, M. Koch, Functions of social networking services, in: Proceedings of the 8th International Conference on...
  • G.C. Kane et al.

    What’s different about social media networks?: A framework and research agenda

    MIS Quart.

    (2014)
  • J. Kleinberg

    The convergence of social and technological networks

    Commun. ACM

    (2008)
  • K. Berger, J. Klier, M. Klier, A. Richter, “Who is key…?” – Value adding users in enterprise social networks, in:...
  • M. Chau et al.

    Business intelligence in blogs: understanding consumer interactions and communities

    MIS Quart.

    (2012)
  • K. Lerman, R. Ghosh, Information contagion: an empirical study of the spread of news on Digg and Twitter social...
  • R.F. Ciriello, D. Hu, G. Schwabe, Identifying patterns of idea diffusion in innovator networks, in: Proceedings of the...
  • O. Oh et al.

    Community intelligence and social media services: a rumor theoretic analysis of tweets during social crises

    MIS Quart.

    (2013)
  • R. Hillmann, M. Trier, Dissemination patterns and associated network effects of sentiments in social networks, in: 2012...
  • J. Moody et al.

    Dynamic network visualization

    Am. J. Sociol.

    (2005)
  • X. Zhang et al.

    Explaining employee job performance: the role of online and offline workplace communication networks

    MIS Quart.

    (2013)
  • L. Wu

    Social network effects on productivity and job security: evidence from the adoption of a social networking tool

    Inform. Syst. Res.

    (2013)
  • X. Zhang et al.

    Network positions and contributions to online public goods: the case of Chinese Wikipedia

    J. Manage. Inform. Syst.

    (2012)
  • S. Sarker et al.

    The role of communication and trust in global virtual teams: a social network perspective

    J. Manage. Inform. Syst.

    (2011)
  • M. Trier

    Towards dynamic visualization for understanding evolution of digital communication networks

    Inform. Syst. Res.

    (2008)
  • J. Howison et al.

    Validity issues in the use of social network analysis with digital trace data

    J. Assoc. Inform. Syst.

    (2011)
  • J.C. Greene et al.

    Toward a conceptual framework for mixed-method evaluation designs

    Educ. Eval. Policy Anal.

    (1989)
  • A. Bryman

    Integrating quantitative and qualitative research: how is it done?

    Qual. Res.

    (2006)
  • P.J. Ågerfalk

    Embracing diversity through mixed methods research

    Eur. J. Inform. Syst.

    (2013)
  • J. Mingers

    The paucity of multimethod research: a review of the information systems literature

    Inform. Syst. J.

    (2003)
  • S. O’Riordan, J. Feller, T. Nagle, Exploring the affordances of social network sites: an analysis of three networks,...
  • A. Majchrzak et al.

    The impact of shaping on knowledge reuse for organizational improvement with wikis

    MIS Quart.

    (2013)
  • J.W. Creswell et al.

    Designing and Conducting Mixed Methods Research

    (2011)
  • V. Venkatesh et al.

    Bridging the qualitative–quantitative divide: guidelines for conducting mixed methods research in information systems

    Manage. Inform. Syst. Quart.

    (2013)
  • N.K. Denzin

    The Research Act in Sociology: A Theoretical Introduction to Sociological Methods

    (1970)
  • Cited by (74)

    • Social Collaboration Analytics Framework: A framework for providing business intelligence on collaboration in the digital workplace

      2021, Decision Support Systems
      Citation Excerpt :

      This observation is interesting because it could be expected that well-established data mining process models such as CRISP-DM [37] could also guide the analysis of ECS data. Behrendt et al. [25] argued that the analysis of ECS data is different from analyzing traditional sources and thus, new methods and approaches are required. Similarly, Hacker et al. [12] argue that the recommendations in CRISP-DM are not suitable for application in the context of SCA.

    • The citizen-led information practices of ICT4D in rural communities of China: A mixed-method study

      2021, International Journal of Information Management
      Citation Excerpt :

      Thus, a large majority of the ICT4D research involves qualitative studies that illustrate how ICTs can empower rural people to improve their lives (Qureshi, 2017; Zheng et al., 2018). However, quantitative methods still have strengths in the empirical investigation of different data dimensions as these methods can reveal data relationships and enhance the integrity of the findings (Behrendt et al., 2014). Thus, it is helpful to bring quantitative methods into qualitative ICT4D studies.

    • The Foundations of Social Media Analytics

      2024, Synthesis Lectures on Information Concepts, Retrieval, and Services
    View all citing articles on Scopus

    Sebastian Behrendt is a Ph.D. candidate at Cooperation Systems Center at Bundeswehr University Munich. He graduated in Information Management at the University of Applied Science and Arts Hannover in 2006. After that, he worked several years as a consultant and project manager in the internet industry before going back to the academics. Now he is focusing on the evaluation of social software in organizations. In a current research project he analyzes the development of informal networks in such tools and their influence on knowledge exchange. He furthermore co-developed a framework that has supported the selection, implementation and evaluation of social software at different companies like Allianz or Bosch.

    Alexander Richter is senior researcher at the Information Management Research Group at University of Zurich and head of Social Business at Cooperation Systems Center at Bundeswehr University Munich. In the last years he has been working in engaged scholarship projects on national and European level to understand the role of social software in the increasingly differentiated working practices of knowledge workers. By doing so, he studies the impact of social platforms in emergent structures such as informal networks, and new forms of leadership and innovation. Furthermore, he is interested in the development of methods to evaluate these developments and approaches to address them. He has for example co-developed a framework that has supported the selection, implementation and evaluation of social software at companies such as Allianz, Airbus, Bayer and Bosch. He is Chairman of the CSCW SIG in the German computer society and a mentor of a startup in the domain of ambient assisted living.

    Matthias Trier is associate professor at Copenhagen Business School. He studies phenomena related to electronic communication and social influence effects in online media within and outside the organization with a mixed methods approach that blends quantitative, qualitative and network analytical methods. Example topics include the implementation/appropriation of social media, online participation, framing electronic discourses (e.g. from a management perspective), information transfer, dissemination processes or bottom-up community emergence as a part of knowledge management initiatives. One special methodological focus is on developing an event-driven method for dynamic network analysis. The corresponding software Commetrix enables research into emerging structures and dynamic processes of networking among people. 

    View full text