Uncovering the structures of privacy research using bibliometric network analysis and topic modelling

2.1 Network analysis

This study re-uses a directed network of the bibliometric record of modern privacy research (van Dijk et al., 2021). The network consists of 83,159 publications from 1965 until 2022 as nodes and the 462,633 references between them as edges. Edges are unweighted, as each reference is considered equal. The network is scale-free and has a clustering coefficient of 0.076. This indicates that clusters and communities exist around much-cited publications. However, together with its average path length (5.6) and maximum path length (21), fragmented information exists within the network as well.

van Dijk et al. (2021) provided a visualisation of the network displaying the communities in the network and the primary topic(s) of each significant cluster (Figure 1). Communities in the network were identified through the Louvain method (Blondel et al., 2008). Each community was already named after its significant clusters, which were visually identified in the network and labelled by random sampling and agreement between two researchers. To further analyse each identified community, we use the following averaged structural measures for each community in the network.

1. In-degree, representing the citations received by the community from other publications in the network.

2. Out-degree, for the references made by the community within the network.

3. Eigenvector centrality, representing the quality of connections within the community. Communities with the most influential publications in the entire network will score higher.

4. Betweenness centrality, for the structural position of a community within the network. Communities with a high betweenness centrality are considered to be in a bridge position in the network.

Together, these centrality measures will provide an overview of the structural positioning of each community within the network.

2.2 Additional data collection

For the topic analysis, the network dataset was extended with the corresponding keywords and abstracts using Pybliometrics (Rose and Kitchin, 2019). Only keywords added by the author(s) were included in the data set to prevent the topic model from learning from previous categorisations, be it automatic or human. Only abstracts are considered, as the benefit from the inclusion of the full text is negligible and significantly increases the computational resources required (Syed and Spruit, 2017).

Since the initial query, 126 works had been retracted or otherwise removed. Of the 83,033 works left, 24,982 are without keywords, 3,534 of which without abstract. Sample inspection showed that these are primarily complete books and conference introductions. 67 works are missing just their abstract but have keywords. All documents without abstract were removed. No new publications were added to the network dataset. The final dataset used for the topic analysis consists of 79,432 publications.

2.3 Topic model

A Latent Dirichlet allocation (LDA) topic model was created from the collected data (Gadellaa, 2021). This generative probabilistic model uses the frequencies of words in the documents to uncover latent topics and assign every document a distribution over these topics (Blei et al., 2003). LDA is a mixed membership model, which means that documents are seen as part of multiple topics in different proportions. This makes it particularly suitable for exploratory and descriptive analyses (Elgesem et al., 2015). By capturing the heterogeneity of research topics that a specific paper can belong to, LDA overcomes the limitations of other techniques (Syed and Spruit, 2017). It is important to note that LDA is a bag-of-words (BOW) model, meaning it does not consider the order of words, just their presence and count within a document. Although this is an oversimplification from how humans understand text, it is well-founded for the purpose of uncovering latent structures (Blei and Lafferty, 2006).

2.4 Synthesis of network and topic model

The topic distribution of each publication and its network outcomes were combined to create an enhanced bibliometric record of the privacy field. This resulted in a dataset consisting of 83,159 publications with, for each publication, its centrality measures, community and main topic. From this dataset, a heat map visualisation was created to identify the subjects of interest in each research community. It displays the proportion of topic occurrence within each network communities. Furthermore, we extracted a sub-set of the 65 most cited publications within the network (in-degree) by combining the 50 most cited publications of the entire timespan with the 50 most cited publications of the last five years. Citations remain the baseline influence measure in the network analysis (Newman, 2010). The period of five years was applied to address the age bias inherent to bibliometric data and is a modal value of citation analysis research (Moody et al., 2010).

We use this combined dataset of the network analysis and topic model to answer the research questions as follows:

To answer RQ1. What communities and topics make up the privacy research field? We consider the outcomes of the community analysis and investigate patterns of interest, such as potentially latent communities.

RQ2. How are the identified communities positioned within broader privacy research? is answered by analysing the structural characteristics of the research communities within the network. We identify the central communities in privacy research, which form the disciplinary core of the research field and consider the findings against literature.

Section 3.1 presents the network, and its communities and their centrality measures. In 3.2, we show the labelled topic model. Section 3.3 brings together these results by displaying the relationship between communities and topics, as well as the topic distribution of the most influential publications. The research questions are then answered by considering the combined analysis in the discussion.

3.1 Network communities

A graphical rendering of the network of privacy research (Figure 1) provides insight into the structures of the field. It is a layered visualisation, made up of.

1. 83,159 nodes: dots representing each of the publications in the network. The size and colour of each node is determined by the number of references received and its community respectively.

2. 462,633 edges: references between publications and are depicted as a line between two nodes.

3. 94 clusters: a clustering of nodes of significant size within the network, named through 90 labels.

4. Communities: algorithmically identified groupings of one or more clusters. The accompanying Table 1 shows the 20 largest communities from a total of 42, with their corresponding colours. These 20 communities account for 93% of the network and are colour-coded in Figure 1.

While a visualisation is only one representation of a more complex dataset (Grandjean, 2016), it provides a high-level overview of the privacy research field.

Table 1 displays the average centrality measures for each of the 20 largest communities. The meaning of the network centrality measures in this context are explained in section 2.1. Like individual centrality measures, the average in-degree remains the primary indicator of influence as the number of citations received. The average out-degree shows a close relation with in-degree, suggesting an exchange between referencing in the network and getting referenced from the network. Whether that dynamic exists within or between communities cannot be said from this data.

Eigenvector centrality appears strongly correlated with in-degree, which can be explained by its focus on influential neighbours. The higher the average in-degree of a community, the more likely a publication in that community has influential neighbours.

Betweenness centrality does not follow in-degree. As an indicator for information flow through the network, it increases for the most central communities in the network – for example individual privacy and law, anonymity metrics and differential privacy – and those connecting a large, otherwise less connected community (with a lower average in-degree) to the larger body of knowledge – such as e-health and medical data and genetic data. Publications bridging the gap between these communities and the rest of the network have an excessively high betweenness centrality, as they are most often on the shortest path between nodes.

On the lower end of the spectrum, a combination of a lower average in-degree and eigenvector centrality appears in communities that form common exit points in the network. The two clearest examples are cybersecurity and physical privacy. Furthermore, there appears to be a sharp drop off in average betweenness centrality for each community with an average in-degree below 3 with more than 10 publications in them. Publications in these communities are more likely to consume information from the rest of the network than to connect otherwise separate disciplines.

3.2 Topic model

The 56 generated topic models were evaluated for their quality, Cv. Most of the models held a value between 0.32 and 0.49, showing a clear pattern where Cv scores increased sharply for the initial ’increase’ in the number of topics K. After the peak, Cv scores slowly decreased for models with more topics. The highest scoring model had a Cv score of 0.517 and a K of 36. This number of topics was not always the maximum across the different random states, suggesting an interaction between the random state and the effect of K, indicating that multiple coherent topic models exist for this dataset. Although the highest scoring model is chosen for interpretation, we have to keep in mind that we are looking at one of many possible representations of this data. A textual representation of the final topic model can be found in Table 2.

Notably, topic 19 seems to include both some recommender systems and healthcare. Inspecting the papers within this category does not yield a satisfying explanation of this grouping of two seemingly unrelated topics. The subjects seldom co-occur in one document, and none of the other most-frequent terms seem to connect the two. It could very well be a topic that would have been split up when 37 was chosen as value for K.

3.3 Combined models analysis

4.1 Subjects in privacy research

Anyone who has investigated privacy can attest to it being a multi-disciplinary field. The resolution limit inherent to manual review offers a challenge in obtaining a broad perspective. In this current analysis we sought for an ordering of privacy research through a combination of network and topic analysis. This section discusses RQ1 . What communities and topics make up the privacy research field?

What stands out immediately is the breadth and connectedness of privacy research. The 83,159 publications formed 94 notable clusters, with subjects ranging from operational healthcare and office planning to smart grids and quantum cryptography. The network analysis and topic model prove to be complementary methods in investigating the makeup of the research field with a strong coherence between their outcomes. Considered as a whole, the results show a diverse research field with communities that have distinct identities but share topics of interest with others.

Three community-topic sets stand out from this perspective. The first regards the community of cybersecurity. The community's primary topics are encryption, encryption schemes and image processing in order of proportion. However, topics such as security operations and access control primarily appear outside of this community. The topic model thus indicates a greater topical breadth of cybersecurity research within privacy than the network analysis alone suggests. As adequate cybersecurity is a pre-requisite for maintaining information privacy (Spiekermann and Cranor, 2009) and with secrecy, security and confidentiality considered to be central aspects of privacy (Smith et al., 2011), this distribution of topics over various communities shows a healthy relationship between privacy and security research.

Then, there are three interconnected communities that share distinct but related research interests, indicating a broader research community than the individual network communities suggest. The communities of data mining, anonymity metrics and differential privacy all have the highest topical proportions of data mining and privacy-preserving algorithms, albeit in different distributions. Especially noteworthy is that the data mining topic is proportionally largest within the community of anonymity metrics. While they are classified as distinct communities, their shared topical interests and interconnectedness suggest a larger macro-community in this corner of the network. This perspective is supported by its most cited literature survey, which draws heavily from each of these communities under the name of privacy-preserving data publishing (PPDP) (Fung et al., 2010).

The third result of interest concerns the topic of system design, which we would expect to strongly relate to the community of system architecture and design. While the community itself is relatively small (3% of the network) and spread out, the topic model suggests that much of the research relevant to this community happens all over the network. Illustrative is the only publication in the most cited publications of the field classified as a system design paper, as it appears in the community of data science. One use of the topic model is to discover potentially latent communities in the investigated area (Syed et al., 2019). While it is possible that system design is a distinct topic applied in all of privacy research, the existence of an overlapping community suggests a more structural issue. These results indicate that system design likely is a latent research community in the privacy field, which has yet to form its core theories or centralise around them. The absence of a sizeable and central system design research community is problematic when we consider the challenges faced by organisations globally in implementing Privacy-by-Design to comply with recent and upcoming privacy legislation. The field should take note of this theoretical gap of how privacy is achieved in information systems.

A final observation is that the community of physical privacy exists at the fringes of the network in a variety of small clusters, such as operational healthcare, sexuality in nursing homes and office planning. Its primary significance is that, however loosely connected this community may be, the distinction between physical and information privacy is mentioned in many of the field's influential theories (Belanger and Crossler, 2011; Phelps et al., 2000; Smith et al., 2011; Solove, 2006). While some theories consider aspects of physical privacy, most use it to distinguish information privacy as a separate concept. However, aspects of both information and physical privacy do appear in many communities, among them location data, Internet of things, vehicular ad hoc networks, RFID, biometrics and smart grid. For example, a smart speaker can be an intrusion in the physical space when it collects audio data during a fight. The physical nature of these information technologies is absent from the field's influential theories and physical privacy is often not considered at all. With further developments in information technology, this line between physical and information privacy will continue to blur. A monitoring device in the operation room can nowadays be connected to an Artificial Intelligence system that offers the operating team recommendations on physical procedures. A smartphone can continuously monitor someone's location. The disappearing boundaries between physical and information privacy offer a new challenge for privacy researchers that have yet to be represented in its theories.

4.2 Central communities in privacy research

While our approach produced new insights in the makeup of privacy research, a major concern with this dataset is that privacy is rarely considered a field of its own. When comparing the network against the full bibliometric record, it only curtails 14.7% of all references made by the collected publications. This is a significant portion, but also includes research communities with different views towards privacy. Due to the broad inclusion criteria in establishing the bibliometric record, only a sub-set of the identified communities forms the core of the privacy field. Others are more likely to apply the field's theories while still identifying as privacy researchers. Others again will view privacy as a reference discipline. In this section, we interpret the results through answering RQ2. How are the identified communities positioned within broader privacy research?

Four communities stand out by their positioning in the network as well as frequent occurrence in the most cited publications. These four communities can be considered the central communities in privacy research around which other research organises itself.

1. Individual privacy and law. This is the largest community in the network with 16.2% of all publications. It primarily concerns itself with consumer privacy, the legal aspects of privacy, social media and data privacy. This view is strengthened further by its theoretical contributions, as the most influential theories in privacy research primarily revolve around individual privacy (van Dijk et al., 2021; Smith et al., 2011).

2. Cloud computing. The second-largest community in the network nominally concerns the current computing paradigm in providing Information Technology capabilities to users (Wang et al., 2010). Researchers in this community, however, focus on a more fundamental privacy issue highly relevant to cloud computing: remote and centralised data processing. As such, many of its disciplines are not specific to cloud computing: untrusted storage, databases, encryption and searching encrypted data. Rather, cloud is the paradigm in which these fundamental issues converge.

3. PPDP is a macro-community consisting of the data mining, anonymity metrics and differential privacy. PPDP is defined as the methods and tools designed to protect the disclosure of any individual's identity while sharing and transporting data (Fung et al., 2010). All three communities share a topical interest in data mining and privacy-preserving algorithms. As evidenced by the high centrality measures of its individual communities, the macro-community of PPDP holds an essential position within privacy research.

4. Location data. The location data community is primarily concerned with its topical namesake. Its most influential publications relate to k-anonymity (Gedik and Liu, 2008) and privacy guarantees in location-based queries (Ghinita et al., 2008). What differentiates it from the other central communities is its combination of physical and information privacy, as many techniques revolve around the obfuscation of a physical location while still retaining usefulness of information (Beresford and Stajano, 2003). It appears a key subject in privacy research, evidenced by its central position in the network and its usefulness in a wide variety of application domains.

Surrounding these central communities we find applied privacy research. Communities in this category are highly connected to the central communities, such as cloud computing, networking, vehicular ad-hoc networks and mobile devices and apps. Each of these communities has mid-range centrality values and strong connections to one or more central communities. While privacy is at the heart of these research communities, they are defined by their application domain.

Then there are communities with little influence in the network. Scoring relatively low on most or all centrality measures, they are less connected to the wider network and thus of less structural importance. Most notable among them are the communities of e-health and medical data and genetic data. Despite their cumulative 14.4% of publications, they hold little authority within the network. A pattern that was also identified in privacy's most influential publications, where a small number of little referenced publications were found to monopolise the connection of medical research to the broader network (van Dijk et al., 2021). Exploring this divide between medical and broader privacy research may be a fruitful direction for future research.

A notable observation is the absence of a central community regarding organisational privacy knowledge, such as privacy management, privacy governance or a similar term. A wide variety of influences have been prescribed to the organisational context, including but not limited to legislation, organisational policies, (organisational) culture, privacy expertise and the business model (Acquisti et al., 2015; Belanger and Crossler, 2011; Culnan and Armstrong, 1999; Smith et al., 2011; Spiekermann and Cranor, 2009; Warkentin et al., 2011). Privacy professionals, besides operating in this complex environment, are asked to weigh organisational interests, privacy concerns, legal requirements, organisational and technical capabilities, social norms and ethics (Bieker et al., 2016; Clarke, 2009; Raab, 2020; Wright, 2013). However, despite various and frequent calls for more organisational research within the field's most influential publications (Belanger and Crossler, 2011; Smith et al., 2011) and an abundance of research on organisational outcomes (Lanier and Saini, 2008; Smith et al., 2011), no influential research community on how to manage privacy at an organisational level has risen to the surface. This absence is also apparent in the field's core theories, which concern themselves with privacy concerns, the individual level of analysis and, in one case, system design (van Dijk et al., 2021). There appears to be a scientific knowledge gap in organisational privacy management in a complex organisational context, while grey literature provides a variety of models and tooling (Swartz et al., 2019). New privacy regulations and increased societal privacy awareness have only increased the need for organisational privacy knowledge, making the absence of such a community even more apparent.

The positioning of legal privacy research next to, and in the same community as, individual privacy research, furthers concerns on the absence of a central community for organisational privacy management. While individual concerns undoubtedly need to be understood and addressed for privacy regulations to be effective, the close relationship between research on the individual level of privacy and legal privacy appears to colour the perspective of legal researchers. Its most influential publications concern themselves with individual privacy harms (Solove, 2006), privacy as contextual integrity (Nissenbaum, 2004) and the necessity of privacy to individual development (Cohen, 2012). However, understanding organisational privacy management is required for the design of effective interventions in the application of privacy legislation. One example of conflict between organisational reality and legislation is the European Union's General Data Protection Legislation (GDPR), which is intentionally technology-agnostic. A summary of unforeseen complications in the implementation of the GDPR finds conflicts in the areas of backups, non-volatile storage, auditability, biometrics, data integrity and increased costs for certain processing activities (Politou et al., 2018). While this cannot be classified as an oversight of existing research, legal privacy researchers should be aware that we currently lack a shared understanding of the domain of organisational privacy management, factors influencing privacy decision-making and what effective treatments can be applied.

4.3 Future research and limitations

This research contributes to our overall understanding of the privacy research field. Its primary data source, Scopus, is considered a reliable source for bibliometric data (Baas et al., 2020), though it provides several limitations. First, various publications have no indexed references or lack an abstract, limiting their usefulness in or excluding them altogether from the analyses performed. Second, the earliest paper was published in 1965 and thus does not include influential publications as least as far back as 1890 (Warren and Brandeis, 1890). The impact of this missing data on the network and topic model has not been quantified. Keeping these limitations in mind, the outcomes of this research provide an overview of current privacy research. Future research could expand the data sources used to gain a more complete picture of privacy research over time and further investigate the temporal developments of the field.

As can be expected of a mapping study, its outcomes also raise new questions. In considering privacy as a singular research field, a number of findings ask for further exploration. First is the presence of system design research as a potentially latent community. It requires in-depth investigation to assess whether this is indeed a latent community within privacy research or rather a distinct and influential reference discipline. Then, the lack of influence and connection of medical research to the broader research field. There is a disconnection between the medical sciences and other privacy research that cannot be explained within the scope of this research. Finally, the absence of organisational privacy management in the network, the topic model and the most cited publications provides what may be the currently most significant knowledge gap in privacy research.

After intermediate presentation of the results we were approached by several scholars interesting in repeating our approach to bibliometric analysis in their own domains. We invite interested researchers in re-using and expanding on this methodology and software used (van Dijk, 2021; Gadellaa, 2021).