Cohesion and segregation in the value migration network: Evidence from network partitioning based on sector classification and clustering

Abstract

Cluster structure detection of the network is a basic problem of complex network analysis. This study investigates the structure of the value migration network using data from 499 stocks listed in the S&P500 as of the end of 2021. An examination is carried out whether the process of value migration creates a cluster structure in the network of companies according to economic activity. Specifically, the cohesion and segregation of the extracted modules in the network division according to (i) sector classification, (ii) community division, and (iii) network clustering decomposition are assessed. The results of this study show that the sector classification of the value migration network has a non-cohesive structure, which means that the flow of value in the financial market occurs between companies from various industries. Moreover, the divisions of the value migration network based on community detection and clustering algorithm are characterized by intra-cluster similarity between the vertices and have a strong community structure. The structure of the network division into modules corresponding to the classification of economic sectors differs significantly from the partition based on the algorithms applied.

1 Introduction

Due to the many complex interactions among a number of agents, financial markets exhibit a collective behavior classified as an evolving complex system with a huge amount of data sets. A large number of complex systems can be represented as complex networks. For this reason, complex network theory has been extensively utilized to investigate the organization of stock market and describe the features of financial market networks. In the case of the complex organization of a set of financial time series, the complex network consists of vertices denoting stocks, and the edges connecting a pair of vertices denote the interrelationships between the two stocks.

Clusters or communities are hidden information embedded in the structure of complex networks. Various clustering methods have been applied in research related to the classification problem in diverse fields. Clustering or community finding is a crucial and one of the most common tasks in the field of complex networks (Radicchi et al. 2004; Palla et al. 2005; Reichardt and Bornholdt 2006; Boccaletti et al. 2006; Rosvall and Bergstrom 2008; Gómez et al. 2009; Collingsworth and Menezes 2014; Jia et al. 2015; Duan et al. 2021; Kuikka 2021; Kumar and Dohare 2021; Su et al. 2021), where the topological structure of complex networks can be characterized by partitioning networks into densely connected subgraphs (Salter-Townshend et al. 2012). Recognizing cohesive clusters or communities and their boundaries allows the classification of nodes according to their topological position in the network (Fortunato 2010; Piccardi et al. 2010) or retaining the same properties. Therefore, clustering has the potential to uncover patterns in the large underlying network. Disclosure of the cluster structure within financial networks enables the modeling of financial systems in an implicit dimension, such as the design of portfolio strategies and risk management.

Network clustering is a data mining procedure that involves assigning a set of nodes to disjoint clusters in such a way that vertices belonging to one group are similar according to the edge structure of the network, while nodes belonging to a distinct cluster are dissimilar. Cluster cohesion in a network occurs when the number of edges connecting nodes within a given group is higher than the number of vertices dispersed in different clusters.

Community structure represents network partitioning in which highly interconnected subgraphs, so-called communities, clusters, or modules occur. Communities are groups of densely connected nodes, and inter-community connections are sparse (Newman and Girvan 2004; Ferreira and Zhao 2016). Community structure detection and exploration is one of the main issues of network analysis and enables to reveal an interesting pattern hidden in the complex network and identify significant clusters containing vertices with similar features. Communities may represent functional units in a networked system (Newman 2012) and help understand the interactions of a complex system. In general, community detection is one of the commonly used methods of clustering network vertices (Piccardi et al. 2011). Clustering (Tumminello et al. 2010; Song et al. 2012; Ross 2014; Musmeci et al. 2015; Bhattacharjee et al. 2019) and community detection (Arai et al. 2015; Wang et al. 2017; Nie 2017; Lee and Nobi 2018; Nie and Song 2018; Hu et al. 2019; Memon et al. 2019; Millington and Niranjan 2020, 2021; Li and Yang 2021) approach are frequently utilized in financial networks.

The economic sector is also called a cluster because its stocks usually share common economic properties (Jiang et al. 2014) or play a similar role within the network (Fortunato 2010). Therefore, in this work, a cluster or group corresponds to a business sector, community or module as a result of the clustering procedure.

A network filtering procedure is performed to simplify the complexity of the financial market graph into a sparse relevant subnetwork. Among the several possible procedures of filtering out noise, the minimal spanning tree (MST) is particularly useful for extracting the most significant information from the original network and generating the network backbone. To extract the value migration network I use the MST-based filtering procedure to prune the correlation-based financial complex network into a simplified form that comprises the significant part of the network connectedness.

The aim of this work is to broaden our knowledge of the structure of value migration on the financial market using the complex network approach. In this study, the value migration network has been divided into groups, and the segregation and cohesion of the three performed partitions have been examined: (i) sectoral classification as a natural division according to the conducted economic activity; (ii) communities; and (iii) PAM clustering. A network partition analysis can assist in revealing important properties of the VMN. The designed research can allow to determine whether the flow of value between enterprises occurs within the sectoral classification as a result of microeconomic factors, or whether the dominant are macroeconomic factors shaping communities in the VMN, which are different from the sector division. If the division of the VMN reflects the sectoral classification, which means that a substantial part of the value flow takes place between companies that belong to the same economic sector, then microeconomic factors related to the specificity of a particular sector are dominant. Otherwise, when value migration between companies of different sectors is observed, the value flow is determined by macroeconomic factors. As a result, the partition of the VMN into clusters using an appropriate algorithm is different from the sector classification determined by macroeconomic factors indicating intersectoral value migration. On this basis, the following research question is formulated: Is value migration between companies in the financial market strongly related to the economic sector to which stocks belong?

In this work, the structure of the value migration process is investigated from the perspective of the US stock market network. For this purpose, the construction of the VMN, previously presented in (Siudak 2022a, b), has been applied. As empirical data, the daily closing prices of 499 stocks of the S&P500 index from the end of 2020 to the end of 2021 have been selected. Specifically, I have studied the level of cohesion of connections between stocks represented as vertices grouped into modules as a result of the decompositions of the network according to the economic activity of enterprises, communities, and cluster. The following research methods have been applied in the study: complex networks, community detection algorithms, clustering algorithms, segregation and cohesion indicators and methods for comparing classification using the Adjusted Rand Index, the indicator w, the normalized van Dongen distance and multivariate analysis of variance (MANOVA).

The empirical results reveal that network partition based on sector classification does not show the segregation and cohesion structure. In this partition of the VMN, the external out-going edges dominate over the internal links within the same sector. In other words, network division based on economic activity has a non-cohesive structure. In contrast, both algorithmic network decompositions into communities and clusters demonstrate a high level of cohesion and segregation. A significant difference in measures of segregation and cohesion was found between the sector and community/cluster divisions of the network. Based on the MANOVA analysis, it has been shown that there are statistically significant differences in the mean values of all segregation and cohesion indicators between algorithmic divisions of the VMN and the partition of the network based on the classification of economic sectors. In other words, in terms of cohesion and segregation, the structure of the compared divisions of the network is significantly different. In addition, based on the Adjusted Rand Index and other measures of comparison between the divisions (normalized von Dongen distance, index $w$), the two algorithmic network divisions differ significantly from the stock arrangement according to sector classification. A comparison of the sector classification with two algorithmic network divisions reveals non-overlapping partitions, which means that these divisions are independent.

Further analysis showed that the VMN has a strong community structure with regard to the decomposition of the network into communities and clusters. On the other hand, intra-sector edges are indistinguishable from the uniformly random counterpart of the network.

The results of this study suggest that a noticeable structure of value migration is observed in the financial market, although this flow occurs between enterprises belonging to different economic sectors.

The results obtained provide a new perspective for the analysis of VM processes in the financial market. A study using a complex network shows the general directions of the movements in the market value of a company on the stock market. This study contributes to the extension of value-based management theory. Unlike the cross-correlation of log-return of stock price networks, the VMN demonstrates links between companies in different economic sectors. In this light, one potential application of this study is to provide direction for optimal portfolio selection by investment funds. The optimal portfolio should contain a set of stocks that maximizes the rate of return while minimizing the risk of volatility of the stock price. Companies that skillfully apply the concept of value-based management, through decisions focused on the creation of economic value added, are able to absorb market value from other enterprises. The value migration map, presented in the form of a graph, provides information about companies in the phase of value inflow and value outflow. Value migration between enterprises of various sectors, as shown in the study, indicates a general intersectoral flow of value. This creates an imbalance in the degree of value migration at the level of economic sectors, which means that there are sectors with a greater potential for capturing value than others. While designing an optimal portfolio, selected stocks from sectors with a greater potential for market value growth should be taken into account. In other words, the high diversification of the portfolio of stocks, due to sector affiliation, can contribute to reducing its effectiveness.

This paper has been divided into the following parts. Section 2 deals with a review of related literature. Section 3 outlines the methodology, and Sect. 4 provides a description of the dataset. Section 5 presents the results. This is followed by Sect. 6, which presents an analysis of the robustness of the main findings. Finally, the last Sect. 7 provides some concluding remarks.

2 Related literature

Complex networks of financial systems contain a lot of information about the dependency structure of assets. Most research on cross-correlation networks has focused on the task of finding clusters of stocks. As has been found in earlier works (Mantegna 1999; Bonanno et al. 2003; Onnela et al. 2003a), stocks in correlation-based networks form clusters corresponding to the economic sector classification taxonomy. Several studies have confirmed the deep similarity of the detected clusters to business sectors, using different methods and filtering procedures to build the stock return correlation network, such as MST based on the Pearson correlation (Borghesi et al. 2007; Coelho et al. 2007; Brida and Risso 2008; Tabak et al. 2010; Kantar et al. 2011; Ulusoy et al. 2012; Tang et al. 2018; Yao and Memon 2019); MST based on the dynamical conditional correlation approach (Lyócsa et al. 2012); threshold based on the Pearson correlation (Li and Yang 2021); fraction of the largest absolute values of the Pearson correlation (Millington and Niranjan 2020); Planar Maximally Filtered Graph (PMFG) based on Pearson correlation (Tumminello et al. 2007; Millington and Niranjan 2021); MST and PMFG based on the wavelet correlation coefficient (Wang et al. 2017); asset graph based on Pearson correlation (Heimo et al. 2007); MST based on normalized mutual information (Guo et al. 2018) and coherence-based distance (Materassi and Innocenti 2009). In contrast, the clustering of companies by economic activity does not occur (Jung et al. 2006; Eom and Park 2017); nevertheless, both studies refer to the Korean stock market.

The theoretical justification for the above findings is the expectation that stock returns in the same economic sectors are influenced by similar economic factors and information flows within the same industry (Borghesi et al. 2007; Musmeci et al. 2015). The cross-correlation network with clusters of a specific economic industry is created as a result of removing weaker links in the network filtration process that have a low correlation. The structural role of weak and strong edges in the financial correlation network has been shown in Garas et al. (2008). After removing 99.5% of the weakest correlated edges of the initial fully connected network based on 1062 stocks from the NYSE, a sparse graph with nodes clustered according to economic sector classification is obtained as opposed to the network after removing 99.5% of the strongest correlated links. The correlation between assets within the same sector is generally expected to be stronger than between stocks from different economic activity.

It should be emphasized that Coelho et al. (2007), showed that for the correlation financial network (MST) based on random time series, computed using the approach presented in studies (Bonanno et al. 2003; Onnela et al. 2003a), clustering of stocks by sector classification is not observed. This indicates that stocks belonging to the same industry tend to cluster together which are observed in the real-world network.

The study (Millington and Niranjan 2021) using the PMFG filtering method, has shed more light on the comparison of the community structure with sector partition for cross-correlation networks based on stock returns of the US (S&P500), UK (FTSE100) and German (DAX50) financial markets. Based on the Adjusted Rand Index, a significantly low level of similarity is observed between the sector classification and the community division using the Louvain algorithm. The ARI value for the US market ranges from 0.1 to 0.5. This means that, for the correlation network, the sector classification is not perfectly reflected in the clusters due to the different activities of companies belonging to the same sector and macroeconomic effects.

The presented literature review concerns a stock market network based on the correlation coefficient between log-return series of stocks. However, there is a lack of studies in relation to the VMN, which differs significantly in its structure from the cross-correlation network of log-return stocks. Detailed information on building the VMN can be found in Sect. 3.1. To the best of my knowledge, no one has so far investigated weather companies in the same sector are clustered together in the value migration network. This is a research gap in terms of the structure of the VMN.

3 Methodology

3.1 VMN construction

The graph corresponding to the VMN should fulfill the following criteria: (i) directed network; (ii) planar graph; (iii) connected graph; (iv) contain a tree structure without a cycle; (v) graph without a self-edges; (vi) simplicity of graph complexity.

The aforementioned features are fully met by the MST-based financial network filtering method (Mantegna and Stanley 1999), compared to approaches that reduce network complexity, such as the PMFG (Tumminello et al. 2005), threshold network (TN) (Boginski et al. 2005), or asset graph (AG) (Onnela et al. 2003b). The MST approach extracts the most relevant information from the fully connected correlation network while preserving the simplest graph structure. In this way, the hierarchical organization of assets can be obtained directly based on the subdominant ultrametric distance matrix. The minimum spanning tree is a simply, connected and acyclic graph that contains the same number of $N$ vertices as the original graph, and $N(N-1)/2$ edges are reduced to $N-1$ in such a way that the sum of all edges weights is minimized.

Additional premises for using the MST are its simplicity, robustness and good network visualization capabilities with acceptable computational time complexity. For the study of the value migration process, the MST-based filtering information approach is regarded as the most for identifying the tree-based value flow paths of companies in the financial market.

The MST extracted from the cross-correlation network is unique if all Pearson correlation coefficients are different. To eliminate the robustness problem of the non-uniqueness of the MST (Djauhari 2012), the PFNET algorithm parametrized by a particular set of values $\left(r=\infty ;q=N-1\right)$ has been used to the simplicity of the negative cross-correlation network. The parameter $r$ denotes the Minkowski metric applied to measure the distance in the path, and the parameter $q$ sets a limit on the number of edges allowed to violate the triangular inequality (Quirin et al. 2008). The MST-based Pathfinder approach allow to obtain the ${\text{PFNET}}\left(\infty ;N-1\right)$ as the equivalent to the union of all the MSTs extracted from the original graph.

The construction of the value migration network based on the procedure presented in Siudak (2022a, b) consists of the following 11 steps:

• 1. Time series of individual stocks are selected.

• 2. The logarithmic returns time series of closing stock price are computed

  $${r}_{i}=\mathrm{ln}{P}_{i}\left(t\right)-\mathrm{ln}{P}_{i}\left(t-\Delta t\right),\; \mathrm{for }\;i,\;j=\mathrm{1,2}, \dots , N; t=\mathrm{1,2}, \dots ,T$$

  (1)

  where $\Delta t=1$ day.

• 3. The symmetric return correlation matrix C with dimensions $N\times N$ is obtained based on the Pearson correlation coefficient between log-returns of two assets $i$ and $j$, defined as follows:

  $${c}_{ij}=\frac{\langle {r}_{i}{r}_{j}\rangle -\langle {r}_{i}\rangle \langle {r}_{j}\rangle }{\sqrt{\left(\langle {r}_{i}^{2}\rangle -{\langle {r}_{i}\rangle }^{2}\right)\left(\langle {r}_{j}^{2}\rangle -{\langle {r}_{j}\rangle }^{2}\right)}}$$

  (2)

  where

  $$\langle {r}_{i}\rangle =\frac{1}{T}\sum_{t=1}^{\mathrm{T}}{r}_{i}\left(t\right)$$

  (3)

• 4. Considering only negative correlation coefficients, the cross-correlation matrix C is transformed to form ${\text{C}}^{\prime}$ with the same dimensions according to the following relation

  $${c}_{ij}^{\prime}=\left\{\begin{array}{ll}{c}_{ij} &\quad {\text{if }} {c}_{ij}<0\\ 0 &\quad {\text{otherwise}}\end{array}\right.$$

  (4)

• 5. The following required quantities are computed as follows:

  • (i) Yearly logarithmic return of the company’s market capitalization:

    $${R}_{{\mathrm{MV}}_{i}}\left(t\right)={\mathrm{lnMV}}_{i}\left(t\right)-{\mathrm{lnMV}}_{i}\left(t-1\right) i=\mathrm{1,2}, \dots , N$$

    (5)

    where ${\mathrm{MV}}_{i}\left(t\right)$ is the market capitalization of asset $i$ at time $t$; $t$ denotes the end of the year.

  • (ii) Return of market capitalization for the entire market:

    $${R}_{\sum {\text{MV}}_{i}\left(t\right)}=\mathrm{ln}\sum_{i=1}^{N}{\text{MV}}_{i}\left(t\right)-\mathrm{ln}\sum_{i=1}^{N}{\text{MV}}_{i}\left(t-1\right)$$

    (6)

  • (iii) Each asset is assigned to a given value migration phase (1) inflow phase ($IP$) or (2) outflow phase ($OP$), according to the following rule:

    $$i\in {\text{IP}} \quad {\text{if }} {R}_{{\text{MV}}_{i}}\left(t\right)\ge \theta {R}_{\sum {\text{MV}}_{i}\left(t\right)}$$

    (7)
    $$i\in {\text{OP}} \quad {\text{if }} {R}_{{\text{MV}}_{i}}\left(t\right)<\theta {R}_{\sum {\text{MV}}_{i}\left(t\right)}$$

    (8)

• 6. For each pair of stock of the reduced cross-correlation matrix C’, the polytomous variable ${x}_{ij}$ is specified as follows:

  $${x}_{ij}=\left\{\begin{array}{ll}1 & \quad{\text{if}}\; {R}_{{\text{MV}}_{i}}\left(t\right)\ge \theta {R}_{\sum {\text{MV}}_{i}\left(t\right)} \;{\text{and}}\, {R}_{{\text{MV}}_{j}}\left(t\right)<\theta {R}_{\sum {\text{MV}}_{j}\left(t\right)} \equiv i\in {\text{IP}} \,{\text{and}}\, j\in {\text{OP}}\\ -1& \quad {\text{if}} \;{R}_{{\text{MV}}_{i}}\left(t\right)<\theta {R}_{\sum {\text{MV}}_{i}\left(t\right)} \;{\text{and}}\, {R}_{{\text{MV}}_{j}}\left(t\right)\ge \theta {R}_{\sum {\text{MV}}_{j}\left(t\right)} \equiv i\in {\text{OP}} \,{\text{and}}\, j\in {\text{IP}}\\ 0 &\quad{\text{if}}\,\, {\text{otherwise}}\end{array}\right.$$

  (9)

• 7. Based on the matrix ${\bf{C}}^{\prime}$and variable ${x}_{ij}$ the symmetric matrix ${\text{A}}=\left[{a}_{ij}\right]$ with dimensions $N \times N$ is built, where

  $$a_{ij} = \left| {c^{\prime}_{ij} \cdot x_{ij} }\; \right| {\text{for }}i, j = 1, 2, \ldots , N$$

  (10)

  within the range $[0; 1]$. In this way, an undirected network is obtained.

• 8. The symmetric distance matrix D = [d_ij] of dimensions $N \times N$ is extracted based on the matrix A, using the metric distance

  $${d}_{ij}=\sqrt{2\left(1-{a}_{ij}\right) }\,\;{\mathrm{for} }\,\,i, j=1, 2, \dots , N$$

  (11)

  such that $0\le {d}_{ij}\le \sqrt{2}$ and fulfills the three axioms of a metric distance—(i) ${d}_{ij}=0$ if and only if $i=j$; (ii) ${d}_{ij}={d}_{ji}$; (iii) ${d}_{ij}\le {d}_{ik}+{d}_{kj}$.

• 9. The distance matrix D is pruned using the MST-based Pathfinder method based on Kruskal’s algorithm (Kruskal 1956). The PFNET ($r=\infty$, q = N − 1) network denoted by T = [t_ij], which is an undirected and weighted graph with a tree form, is created using the Pathfinder procedure (Quirin et al. 2008).

• 10. The PFNET ($\infty$, N—1) network is dichotomized where the elements ${t}_{ij}^{\prime}$ of matrix $\mathbf{T}\mathrm{^{\prime}}$ are transformed according

  $${t}_{ij}^{\prime}=1\; \mathrm{if} \;{t}_{ij}>0$$

  (12)

• 11. To determine the direction of value migration, T′ matrix is transformed into an adjacency matrix ${\mathbf{A}}^{\prime}=\left[{a}_{ij}^{\prime}\right]$ such that elements are computed using the following rule:

  $$a_{ij}^{\prime} = \left\{ \begin{gathered} \begin{array}{*{20}c} {a_{ij}^{\prime} = 1} \\ {a_{ji}^{\prime} = 0}\\ \end{array}\big \} \quad \quad {\text{if}} \;\;t_{ij}^{\prime} = t_{ji}^{\prime} = 1 \;\;{\text{and}}\;\; x_{ij} = - 1 \hfill \\ \begin{array}{*{20}c} {a_{ij}^{\prime} = 0} \\ {a_{ji}^{\prime} = 1} \\ \end{array}\big\} \quad \quad {\text{if}} \;\; t_{ij}^{\prime} = t_{ji}^{\prime} = 1 \;\;{\text{and}}\;\; x_{ij} = 1 \hfill \\ \end{gathered} \right.$$

  (13)

The annual window applied in step 5 is used to reduce the effects of short-term volatility in the company value.

3.2 Community structure

In a complex network, the community is a subgraph $V$ with a few edges that connecting the rest of the graph (Fortunato 2010) and with cohesive connections between vertices belonging to the same module. In other words, a community is a subset of vertices with a high density of internal connections and sparse edges with the rest of the modules. For consistency of further considerations, the formal definition of the community in the weak sense (Radicchi et al. 2004) has been adopted, where for a directed network $c\subset G$ is a module if the sum of all out-going edges within the community is greater than all out-going connections toward the rest of the graph.

To assess the goodness of a given network division in the process of studying cluster structure, a quality function has been defined, the so-called modularity (Newman and Girvan 2004) based on the assumption that random networks are not expected to exhibit community structure. Modularity is expressed as the difference between the actual number of edges in a graph that connects nodes within modules and the expected number in an equivalent network (Newman 2006a, 2012), referred to as the null model, with the same number of vertices if edges are assigned at random while preserving vertex degrees.

Assume that $N$ vertices in the graph are partitioned into communities where node $i$ is assigned to the module ${c}_{i}\in \left\{\mathrm{1,2},\dots ,r\right\}$. Using the adjacency matrix ${{\varvec{A}}}^{\boldsymbol{^{\prime}}}$ and the cluster assignment vector $\mathbf{c}=\left[{c}_{1},\dots ,{c}_{N}\right]$, the Newman–Girvan modularity of the graph partition for a directed network is as follows (Leicht and Newman 2008):

$$Q=\frac{1}{m}\sum_{ij}\left({a}_{ij}^{\prime}-\frac{{k}_{i}^{\mathrm{out}}{k}_{j}^{\mathrm{in}}}{m}\right)\delta \left({c}_{i},{c}_{j}\right)$$

(14)

where $m$—total number of edges in the directed network, expressed as follows:

$$m={\sum }_{i=1}^{N}{k}_{i}^{\mathrm{out}}={\sum }_{j=1}^{N}{k}_{j}^{\mathrm{in}}$$

(15)

${k}_{i}^{\mathrm{out}},{k}_{j}^{\mathrm{in}}$—out- and in-degree of nodes $i$, $j$, respectively

$${k}_{i}^{\mathrm{out}}=\sum_{j=1}^{N}{a}_{ij}^{\prime}$$

(16)

$${k}_{j}^{\mathrm{in}}=\sum_{i=1}^{N}{a}_{ij}^{\prime}$$

(17)

${a}_{ij}^{\prime}$: elements of the adjacency matrix A^’ with values 0 and 1.

$\delta \left({c}_{i},{c}_{j}\right)$: Kronecker delta function, where $\delta \left({c}_{i},{c}_{j}\right)=1$ when both vertices $\left(i,j\right)$ belong to the same module $\left({c}_{i}={c}_{j}\right)$, and $\delta \left({c}_{i},{c}_{j}\right)=0$ when both nodes $\left(i,j\right)$ are not classified into the same group $\left({c}_{i}\ne {c}_{j}\right)$.

Modularity $Q$ takes values less than 1, where the closer the value of $Q$ is to 1, the stronger the community structure as it deviates more from the null model. In other words, a higher positive value of modularity implies better partitioning, where the division contains groups in which the greater number of edges exceeds the expected value of occurrence under a random assignment of ties with a preserved degree of nodes. If the number of integral links in modules is equal to the expected value in a uniformly random network, the modularity is 0, which indicates that all nodes are in the same community. Modularity can be negative in the case where each vertex is assigned to a separate module. In conclusion, when there is no positive modularity value, the community structure is not defined in the graph. A network is considered to have a significant community structure if the modularity $Q$ value is higher than 0.3 (Clauset et al. 2004; Newman 2004). It should be noted that the maximum modularity of a network increases as the number of vertices in the network grows and the number of distinct divisions extends (Good et al. 2010). Therefore, modularity should not be applied to compare the cluster structure of the entire network, especially when two networks differ significantly in the number of well-separated modules (Fortunato 2010).

In the problem of detecting the community structure, the optimal partition of the network is searched in which the maximization of modularity $Q$ is utilized. In the case of community structure identification, the complex network is simplified, and general areas in the network structure are specified.

The modularity measure will be used not only to assess the quality of the occurrence of the community structure in the VMN using different algorithms for decomposing the graph into modules but also to evaluate and compare the natural division into groups according to economic sectors and the partition into clusters according to the applied clustering method.

3.3 PAM-based clustering

There are several techniques for identifying the cluster structure in complex networks. In this study, I employed the Partitioning Around Medoids (PAM) method, called k-medoids, which is a clustering procedure developed in (Kaufman and Rousseeuw 1990). The aim of the PAM algorithm is to find, for each cluster discovered, an archetypal object called a medoid, which is the most centrally located element within the disjoint clusters (Halkidi et al. 2001). The algorithm uses an interactive procedure to optimize the clustering criterion function to decompose the vertices into a specified number of clusters $\left(r\right)$. The criterion function of PAM $\left({TC}_{ih}\right)$ is to minimize the distance of the nodes in the VMN belonging to the cluster from its medoid. The classification of shares in the VMN into disjoint groups applying the PAM algorithm is dictated by the following premises (Ng and Han 1994):

(i) The PAM algorithm is robust to outliers in the data set.

(ii) Clusters determined by the PAM are independent of the order in which the elements are inspected.

(iii) The PAM method is invariant in relation to translations and orthogonal transformation of data points.

The Partitioning Around Medoids algorithm is as follows (Ng and Han 1994; Halkidi et al. 2001; Musmeci et al. 2015):

• 1. Select randomly $r$ elements as medoids for each of $r$ clusters among the $N$ nodes;

• 2. Assign each non-selected element $\left({O}_{j}\right)$ to the medoid $\left({O}_{i}\right)$ to which the representative object is most similar;

• 3. For each medoid, swap the medoid with each point assigned to it and compute the clustering criterion function $\left({\mathrm{TC}}_{ih}\right)$ for all pairs of elements currently selected $\left({O}_{i}\right)$ and currently non-selected $\left({O}_{h}\right)$, where

  $${\mathrm{TC}}_{ih}=\sum_{j}{\varphi }_{jih}$$

  (18)

  and ${\varphi }_{jih}$ denotes the total cost of swapping ${O}_{i}$ with ${O}_{h}$;

• 4. Pick out the configuration consisting of the pair ${O}_{i}$, ${O}_{h}$, that is consistent with the minimum of the criterion function, $min\left({TC}_{ih}\right)$, for all non-selected objects. Swap ${O}_{i}$ with ${O}_{h}$ when the clustering criterion function is negative;

• 5. Reiterate steps (2)–(4) until it is impossible to replace the medoids with other non-selected elements.

Unlike other hierarchical clustering methods, the PAM algorithm produces a dendrogram-free partition (Musmeci et al. 2015).

3.4 Segregation–cohesion indicators

Several indicators are used to assess the degree of cohesion and segregation of the clustering of stocks in the VMN.

• 1. Assortative mixing—the assortativity of a network is the tendency of nodes to connect with similar ones in a group (Newman 2003; Catanzaro et al. 2004; Silva et al. 2016) due to the considered feature, which is measured by the assortativity coefficient. For the designed study of the VMN, the network has an assortative pattern when the fraction of links between nodes belonging to the same cluster, community, or economic sector is significantly greater than would be expected by random chance. The assortativity coefficient is evaluated in a different way than for the measure of the degree correlation coefficient presented in (Newman 2002). Let ${c}_{i}$ be the cluster to which node $i$ is assigned. The assortativity coefficient is the normalized fraction of the difference between the actual and expected number of links in a graph that combine nodes of similar types to the number of edges (Newman 2010). For a directed network, the coefficient can be written as follows:

  $$ac=\frac{\sum_{ij}\left({a}_{ij}^{\prime}-\frac{{k}_{i}^{\mathrm{out}}{k}_{j}^{\mathrm{in}}}{m}\right)\delta \left({c}_{i},{c}_{j}\right)}{m-\sum_{ij}\left(\frac{{k}_{i}^{\mathrm{out}}{k}_{j}^{\mathrm{in}}}{m}\right)\delta \left({c}_{i},{c}_{j}\right)}$$

  (19)

  Significant positive values of the assortativity coefficient indicate that out-going edges in the VMN connect more pairs of vertices within the same cluster than would be expected by random chance. On the contrary, negative values of the assortative coefficient mean that there are fewer connections in the network between nodes belonging to the same group than would be expected by random chance. One can say that the network is disintegrated in terms of clusters demonstrating disassortative behavior. The maximum value of the assortativity coefficient $\left(ac=1\right)$ means the perfect assortative mixing behavior. In other words, all edges are within the same cluster. It should be noted that the three classifications of assets in the value migration network contain more than two groups and, as emphasized in (Bojanowski and Corten 2014), the value of the assortativity coefficient equals to 0 does not mean that the graph is reconcilable with proportionate mixing.

• 2. Group modularity, in contrast to the modularity applied to the problem of assessing the community structure of the entire network, is used to evaluate how the observed connectivity within group $c$ differs from the expected value of a randomly created network with the same degree sequence. The network modularity formula (14) previously discussed in the context of the community structure detection problem can be expressed in an equivalent way (Good et al. 2010; Barabasi 2016)

  $$Q=\sum_{c=1}^{r}\left[\frac{{e}_{c}^{\mathrm{out}}}{m}-{\left(\frac{{k}_{c}^{\mathrm{out}}}{m}\right)}^{2}\right]$$

  (20)

  where ${e}_{c}^{\mathrm{out}}$—number of out-going links inside group $c$; ${k}_{c}^{\mathrm{out}}$—total out-degree of vertices in module $c$.

  Then the definition of group modularity $\left({Q}_{c}\right)$ takes the form

  $$Q_{c} = \frac{{e_{c}^{{{\text{out}}}} }}{m} - \left( {\frac{{k_{c}^{{{\text{out}}}} }}{m}} \right)^{2}$$

  (21)

  which is simply the difference between the fraction of out-going edges within group $c$ and the expected number of out-going connections from vertex $i$ to $j$ in a graph with a randomly connected edge.

  The interpretation of ${Q}_{c}$ is similar to the modularity of the entire network. The higher the group modularity $\left({Q}_{c}\right)$, the more internal edges within module c than expected by random chance and the better cohesion within the module. Considering the formulas (20) and (21), the values of group modularity decrease with the increase in the number of selected groups. This should be taken into account when directly comparing their values for two network partitions with a different number of modules.

• 3. E-I index (${{\text{EI}}}_{c}$) (Krackhardt and Stern 1988) is a measure of domination of the external edges over internal edges in group c. Let ${{\text{InE}}}_{c}$ and ${{\text{ExE}}}_{c}$ are internal edges and external edges, respectively, in cluster $c$. The internal edges of the cluster $c$ are the total number of links on the graph connecting vertices belonging to the same cluster $c$

  $${\mathrm{InE}}_{c}=\sum_{i\in c}\sum_{j\in c}{a}_{ij}^{\prime} \left(i,\;j=1, 2, \dots ,N;i\ne j;\;c=\mathrm{1,2},\dots ,r\right)$$

  (22)

External edges of cluster $c$ are defined as the total number of links out-going from nodes belonging to cluster $c$ that are directed to vertices belonging to other clusters than $c$

$${\mathrm{ExE}}_{c}=\sum_{i\in c}\sum_{j\notin c}{a}_{ij}^{\prime}\left(i,j=1, 2, \dots ,N;\;i\ne j;\;c=\mathrm{1,2},\dots ,r\right)$$

(23)

The E-I index for cluster c is specified as follows:

$${\mathrm{EI}}_{c}=\frac{{\mathrm{ExE}}_{c}-{{\text{InE}}}_{c}}{{{\text{InE}}}_{c}+{{\text{ExE}}}_{c}}$$

(24)

and the E-I index for the entire network is defined as follows:

$${\text{EI}}=\frac{\sum_{c=1}^{r}{{\text{ExE}}}_{c}-{\sum }_{c=1}^{r}{{\text{InE}}}_{c}}{{\sum }_{c=1}^{r}{{\text{InE}}}_{c}+\sum_{c=1}^{r}{{\text{ExE}}}_{c}}$$

(25)

The E-I index is in the interval $\langle -1, 1\rangle$, where the value of ${{\text{EI}}}_{c}$ equals −1 indicates that all edges are inside the cluster $c$, while the value of ${{\text{EI}}}_{c}$ equals 1 corresponds to all edges out-going outside the cluster $c$. When the E-I index equals 0, it means that the number of internal edges in cluster c is the same as the number of external edges (Bojanowski and Corten 2014). A similar interpretation is formulated for the E-I index for the entire network where internal and external edges apply to the vertices of all emerged clusters, not only group $c$.

• 4. Segregation matrix index (SMI) is a measure introduced in (Fershtman and Chen 1993) and designed for directed networks (Bojanowski and Corten 2014). The SMI is based on density of connection between vertices belonging to cluster $c$ and the density of connection between vertices from group c toward nodes outside of cluster $c$

  $${\mathrm{SMI}}_{c}=\frac{{\rho }_{c}-{\rho }_{r}}{{\rho }_{c}+{\rho }_{r}}$$

  (26)

  where

  $${\rho }_{c}=\frac{{{\text{InE}}}_{c}}{{N}_{c}\left({N}_{c}-1\right)}$$

  (27)
  $${\rho }_{r}=\frac{{{\text{ExE}}}_{c}}{{N}_{c}\left(N-{N}_{c}\right)}$$

  (28)

  ${N}_{c}$– number of vertices of group $c$, with SMI values in the interval of $\langle -1, 1\rangle$.

  The higher the segregation matrix index, the higher the group segregation level. If the connections between the vertices belonging to the cluster c reveal self-preference, the segregation matrix index exceeds 0 $\left({\text{SMI}}_{c}>0\right)$. On the contrary, when group $c$ reveals other preferences, SMI is below 0 $\left({{\text{SMI}}}_{c}<0\right)$. In a special case, SMI takes the extreme values from its range, when all out-going edges of cluster $c$ connect elements from other clusters $\left({{\text{SMI}}}_{c}=-1\right)$ or in the case of complete group segregation, when all edges belonging to cluster $c$ are intra-group $\left({{\text{SMI}}}_{c}=1\right)$ (Fershtman 1997). SMI = 0 indicates that the intra-group density is equal to the extra-group density, which means that the created connections are indistinguishable from what is expected by a random chance.

• 5. Cohesion index (${{\text{CI}}}_{c}$) (Bock and Husain 1950) is expressed as the ratio of the cohesion of the cluster $c$ to the dispersion of the out-going edges of nodes belonging to the cluster $c$ directed to vertices belonging to other groups in the network (Wasserman and Faust 1994)

  $${{\text{CI}}}_{c}=\frac{{\rho }_{c}}{{\rho }_{r}}=\frac{\frac{{{\text{InE}}}_{c}}{{N}_{C}\left({N}_{c}-1\right)}}{\frac{{{\text{ExE}}}_{c}}{{N}_{c}\left(N-{N}_{c}\right)}}$$

  (29)

  If the value of ${{\text{CI}}}_{c}$ is above 1 $\left({{\text{CI}}}_{c}>1\right)$, the internal group density prevails over the external group density. On the contrary, the value of ${{\text{CI}}}_{c}$ below 1 corresponds to the dominance of the external cluster density over the internal cluster density. A cohesion index value of 1 indicates that internal group density and external group density are equal to each other.

• 6. Cluster density is the fraction of the number of edges that are actually present in each group and the maximum possible number of connections in a cluster. For a directed network, the cluster density is defined as follows:

  $${\nu }_{c}=\frac{{m}_{c}}{{N}_{c}\left({N}_{c}-1\right)}$$

  (30)

  where ${m}_{c}$ denotes the number of edges observed in the cluster $c$; ${\nu }_{c}$ values are in the interval of $\langle 0, 1\rangle$.

  Cluster density is interpreted as the probability of the occurrence of an edge between any two randomly selected vertices belonging to cluster $c$. The lower the value of cluster density, the lower the group cohesion and vice versa. It should be pointed out that the density of a cluster decreases as the number of vertices in the group increases. This property determines the limitation of the use of density as a measure of cohesion for inter-group comparisons of heterogeneous size.

• 7. Relative cluster link is the fraction of the number of internal edges of cluster $c$ to all links observed in the same cluster $c$

  $${RCL}_{c}=\frac{{{\text{InE}}}_{c}}{\left({{\text{InE}}}_{c}+{{\text{ExE}}}_{c}\right)}$$

  (31)

  ${\mathrm{RCL}}_{c}$ is in the interval $\langle 0;1\rangle$, where 0 means that there are only external connections in the cluster, and 1 indicates that all vertices in the same module are connected only by internal edges.

• 8. Relative cluster ratio is the average balance of internal/external edges of cluster c

  $${\text{RCR}}_{c}=\frac{1}{{N}_{c}}\left({{\text{InE}}}_{c}-{{\text{ExE}}}_{c}\right)$$

  (32)

• 9. Community centrality is a measure developed by Newman (Newman 2006b) that determines how central vertices are in the community structure to which they are assigned. Let $f$ denote the positive eigenvalues of the modularity matrix $\mathbf{B}$. Community centrality is based on the eigenvector of the modularity matrix, defined as the vector of magnitude $\left|{\mathbf{x}}_{\mathrm{i}}\right|$ (Newman 2006b)

  $$\left|{{\varvec{x}}}_{i}\right|=\sum_{j=1}^{f}\sqrt{{\beta }_{j}}\cdot {U}_{ij}$$

  (33)

  where ${\varvec{U}}$—matrix of eigenvectors of the modularity matrix $\mathbf{B}$, ${\beta }_{j}$—eigenvalue of modularity matrix $\mathbf{B}$ corresponding to the eigenvector ${\mathbf{u}}_{\mathrm{i}}$, $\mathbf{B}$—modularity matrix with elements

  $${B}_{ij}={a}_{ij}-{P}_{ij}$$

  (34)

  ${P}_{ij}$—expected number of out-going links from node i to j

  $${P}_{ij}=\frac{{k}_{i}{k}_{j}}{2m}$$

  (35)

A high level of community centrality indicates that a vertex has more edges than expected within its local neighborhood and is likely to make a substantial contribution to the total network modularity (Newman 2006b). Large values of community centrality indicate greater power of vertex in its own community. In other words, vertices with a high value of community centrality are the most networked nodes belonging to their assigned communities. These hubs play a dominant role within the structure of their own group.

Community centrality is computed from the aggregated network at the cluster level, where individual nodes of the VMN within the same group are merged into one cluster vertex. The sum of the number of links between stocks belonging to a given cluster is calculated, where the internal edges and the links out-going from the cluster are counted separately, respectively

$${a}_{cc}=\sum_{i\in c}\sum_{j\in c}{a}_{ij}^{\prime}={{\text{InE}}}_{c} \left(i, j=\mathrm{1,2},\dots ,N;c=\mathrm{1,2},\dots ,r\right)$$

(36)

$${a}_{cl}=\sum_{i\in c}\sum_{ j\in l}{a}_{ij}^{\prime} \left(i, j=\mathrm{1,2},\dots ,N;c=\mathrm{1,2},\dots ,r;l=\mathrm{1,2},\dots ,r-1;c\ne l\right)$$

(37)

where ${a}_{cc}$—number of edges between nodes belonging to cluster $c$; ${a}_{cl}$—number of edges between nodes belonging to cluster c and $l$.

The cluster-based network is a directed and valued graph with loops and self-edges. The aggregated network is dichotomized and symmetrized before computing community centrality for each group of a given division. Equations (33)–(35) regarding the community centrality of cluster $c$ in the network aggregated at the cluster level are defined as follows:

$$\left|{{\varvec{x}}}_{c}\right|=\sum_{l=1}^{f}\sqrt{{\beta }_{l}}\cdot {U}_{cl}$$

(38)

$${B}_{cl}={a}_{cl}-{P}_{cl}$$

(39)

$${P}_{cl}=\frac{{k}_{c}{k}_{l}}{2m}$$

(40)

A high value of community centrality of the specified groups in a given division indicates the cohesion of nodes in the network aggregated at the cluster level.

3.5 Classification comparison

The Adjusted Rand Index (Hubert and Arabie 1985)is applied to assess the clustering similarity. This measure is based on the Rand Index (Rand 1971), which compares the percentage of correct decisions carried out by the clustering method to the maximum possible number of links. Let denote by $S$ a set of N vertices and let two partitions $X$ and $Y$ of $S$ into non-empty disjoint subsets. Let $p=1, 2, \dots P$ and $q=1, 2,\dots , Q$ be clusters of $X$ and $Y$ partitions, respectively, of the set of elements $S$. The Rand Index is computed as follows:

$$\mathrm{RI}\left(X,Y\right)=\frac{2\left(a+b\right)}{N\left(N-1\right)}$$

(41)

where $a$ is the number of pairs of elements correctly classified in the same cluster in $X$ and $Y$ partitions; $b$ is the number of pairs of elements correctly classified in two different clusters in $X$ and $Y$ partitions.

The Adjusted Rand Index can be defined as the difference between the RI and its mean value under the null hypothesis associated with two independent partitions, normalized by the maximum attainable for the above difference (Musmeci et al. 2015):

$$\mathrm{ARI}\;\left(X,Y\right)=\frac{\mathrm{RI}\left(X,Y\right)-\left[\frac{2{t}_{1}{t}_{2}}{N\left(N-1\right)}\right]}{\frac{\left({t}_{1}+{t}_{2}\right)}{2}-\left[\frac{2{t}_{1}{t}_{2}}{N\left(N-1\right)}\right]}$$

(42)

where

$${t}_{1}=\sum_{i}^{k}\left(\genfrac{}{}{0pt}{}{\left|{X}_{p}\right|}{2}\right), {t}_{2}=\sum_{j}^{l}\left(\genfrac{}{}{0pt}{}{\left|{Y}_{q}\right|}{2}\right).$$

(43)

ARI values are in the range $\langle -1;1\rangle$. A perfect match between the two partitions $X$ and $Y$ is demonstrated when ARI = 1. If ARI = 0, then the two partitions are completely uncorrelated, which means that clustering is independent. Negative ARI values indicate an anti-correlation of the two partitions. This means that the match between the two partitions $X$ and $Y$ is weaker than expected under the random chance assumption.

A measure of the compatibility of the two classifications is used to compare the two partitions. In the contingency table $M=\left\{{m}_{pq}\right\}$, the number of elements in the intersection of clusters $p$ and $q$ of the partitions $X$ and $Y$ are written as follows:

$${m}_{pq}=\left|{X}_{p}\cap {Y}_{q}\right| \left(p=\mathrm{1,2},\dots P;\;q=\mathrm{1,2},\dots ,Q\right)$$

(44)

The measure of agreement of the two classifications w is defined as follows (Nowak 1990):

$$w=\frac{1}{P+Q}\cdot \left({\sum }_{p=1}^{P}{h}_{p0}+{\sum }_{q=1}^{Q}{h}_{0q}\right)$$

(45)

where P, Q are the numbers of groups of partitions X and Y

$${h}_{p0}=\underset{q}{\mathrm{max}}\left\{{h}_{pq}\right\}\; \left(p=\mathrm{1,2},\dots ,P\right)$$

(46)

$${h}_{0q}=\underset{p}{\mathrm{max}}\left\{{h}_{pq}\right\} \;\left(q=\mathrm{1,2},\dots ,Q\right)$$

(47)

$${h}_{pq}=\frac{{m}_{pq}}{\mathrm{max}\left({m}_{p.},{m}_{.q}\right)}\;\left(p=\mathrm{1,2},\dots P; q=\mathrm{1,2},\dots ,Q\right)$$

(48)

$${m}_{p.}={\sum }_{q=1}^{Q}{m}_{pq}, {m}_{.q}={\sum }_{p=1}^{P}{m}_{pq}$$

(49)

The indicator $w$ is in the interval $\left[\frac{1}{N}\le w\le 1\right]$, where N is the number of vertices. A value of 1 for $w$ corresponds to two homogeneous partitions, and the lower value of the above range $\left(1/N\right)$ indicates an almost complete incompatibility of the two cluster partitions. In other words, when the first partition consists of N single-element groups and the second partition consists of one group of N elements, then the indicator $w$ equals $\left(1/N\right)$. $w=\left(1/N\right)$ occurs if two equalities are satisfied $P=N, Q=1$ or $P=1, Q=N$ (Nowak 1990).

The Adjusted Rand Index is a criterion for comparing partitions by counting pairs. The van Dongen distance (Meilǎ 2007) is a symmetric metric referred to as a set-matching index. The normalized form of the van Dongen distance metric is defined as follows:

$${D}^{n}\left(X,Y\right)=1-\left(\frac{\sum_{p=1}^{P}\underset{q}{\mathrm{max}}\left\{{m}_{pq}\right\}+\sum_{q=1}^{Q}\underset{p}{\mathrm{max}}\left\{{m}_{pq}\right\}}{2N}\right)$$

(50)

The normalized van Dongen distance index ranges from 0 to strictly less than 1 $\left({D}^{n}\left(X,Y\right)\in [0;1)\right)$. The index takes value 0 for perfect clustering matching of partitions $X$ and $Y$. The value of ${D}^{n}\left(X,Y\right)$ close to 1 means that the partitions are completely independent.

Multivariate analysis of variance (MANOVA) is used to test the equality of the segregation and cohesion indicators in a multidimensional approach and in more than two groups. The null hypothesis, which assumes equality in the mean measures of segregation and cohesion for three different divisions, takes the form

$${H}_{0}:{{\varvec{\mu}}}_{1}={{\varvec{\mu}}}_{2}={{\varvec{\mu}}}_{3}$$

(51)

to an alternative hypothesis

$$H_{1} :\exists k_{1} k_{2} :{{\varvec{\upmu}}}_{{{\mathbf{k}}_{1} }} \ne {{\varvec{\upmu}}}_{{{\mathbf{k}}_{2} }}$$

(52)

assuming that the mean values differ in at least two partitions.

If the null hypothesis is rejected, there is a significant difference between at last two divisions. If the null hypothesis cannot be rejected, the measures in the three decompositions are significantly equal. Assessment of the equality test of segregation and cohesion measures in partitions is evaluated using four statistics: (1) Wilks’ Lambda (L); (2) Pillai's trace (PT); (3) Lawley–Hotelling Trace (LWT); (4) Roy’s Largest Root (RLR).

4 Data set

In the empirical study, time series of N = 499 constituent stocks belonging to the S&P500 index was selected, and the data of daily closure stock prices from December 31, 2020, to December 31, 2021, containing 252 daily returns of each stock were collected. The S&P500 Index is based on a set of the 500 most influential companies in the USA market that can be considered representative of the US economy. For the purposes of the construction of the VMN, the stock market capitalization of listed companies was recorded at the beginning and the end of the analysis period, i.e., the end of 2020 and the end of 2021. One company that was not continuously quoted on the stock exchange in the analyzed period was excluded from the analysis. The data were obtained from Yahoo Finance (http://finance.yahoo.com). The threshold value used in formula (7)–(9) was set to 0.6667.

The analyzed stocks are classified into 11 sectors according to their economic activities based on the classification of Yahoo Finance (http://finance.yahoo.com). The sector partition of the analyzed enterprises is as follows: Basic Materials (BM; 21 stocks), Communication Services (CS; 23), Consumer Cyclical (CC; 63), Consumer Defensive (CD; 35), Energy (EN; 21), Financial Services (FS; 70), Healthcare (HE; 64), Industrials (IN; 72), Real Estate (RE; 29), Technology (TE; 73); Utilities (UT; 28).

For 499 nodes, there are N(N−1)/2 = 124,251 possible correlation coefficients, which are contained in the Pearson correlation matrix C with the dimensions $\left(N\times N\right)$ $499\times 499$. Of the 124,251 possible correlation coefficients of log rate of return, 9,198 are negative correlations (7.40%), and the rest are positive correlation coefficients (115,053; 92.60%). According to the VMN construction methodology, only negative correlation coefficients are taken into account.

As a result of network filtration using the MST-Pathfinder procedure, the number of links in matrix A was reduced from 4.795 to (N−1) 494 of the most important ones, representing 10.30% of the original number of edges. This means that the number of interconnected nodes in the network is 495 out of 499 analyzed stocks (99.2%). In other words, the largest component of the VMN has 495 vertices, and only 4 stocks are isolated. These isolated companies belong to the following sectors: Utilities, Technology, Industrials, and Real Estate. This means that the sources (sinks) of the inflow (outflow) of value to (from) these companies are other assets on the financial market. The value migration network is directed and unweighted and is shown in Fig. 1 (the largest component). The color of the node corresponds to one of the 11 economic sectors.

Fig. 1

Largest component of the value migration network (N = 495 out of 499 stocks) based on the visualization algorithm: Stress Majorization (Gansner et al. 2004). The color of the node indicates the economic sector to which the stock is assigned

The VMN exhibits a tree-like structure with a few distinct areas located in the center of the network and with several peripheries of the graph, which are clustered by single high-degree vertices. The reason for this characteristic network structure is the disassortative behavior of the VMN, and the node degree distribution of the VMN obeys the power-law (Siudak 2022a). This means that there are a small number of nodes in the network that have a high degree, and these hub-like vertices are connected with low-degree nodes. All calculations were performed with the following software: (TIBCO Software Inc 2017; Cyram 2022). All data underlying this study is available in the research data repository (https://data.mendeley.com/datasets/gs424fsr8g/1).

5 Results

The company partitions created were used for comparison based on: (1) sector classification, (2) community, and (3) the PAM clustering algorithm. The analysis of the clustering structure in the VMN is carried out after removing the isolated stocks. In other words, the community search, PAM clustering, and sector division are performed based on the largest component of the VMN.

5.1 Network partition

5.1.1 Community structure of the value migration network

Community structure detection in the VMN is not a trivial task, especially as the algorithms applied do not allow for a predetermined number of clusters. The aim of community detection is to divide stocks into a number of groups similar to the number of economic sectors. To obtain a reasonable community structure with a similar number of modules for partitioning assets into 11 clusters according to the classification of economic sectors, 7 algorithms have been used and compared based on the modularity measure $\left(Q\right)$: (i) CNM (Clauset et al. 2004); (ii) HE; (iii) HE′; (iv) NE (Wakita and Tsurumi 2007); Louvain Fast Unfolding method (Blondel et al. 2008); (v) Label Propagation (Raghavan et al. 2007); (vi) Eigenvector (Newman 2006a). CNM, Louvain, and Eigenvector methods are algorithms for dividing networks into communities in which the optimal community structure is detected in a greedy manner to maximize modularity. HE, HE′, and NE are the three types of variations of the CNM algorithm. The Label Propagation technique uses the network structure without any external parameter setting.

Table 1 presents the results of the community structure detection using seven algorithms. As shown in Table 1, the value of modularity $Q$ reaches over 0.86 for all community detection algorithms. This means that a non-trivial community structure has been detected using different algorithms beyond what can be expected under the degree-preserving null model of random link assignment. The modularity value for real-world networks is observed in the range of 0.3–0.7, according to Newman and Girvan (Newman and Girvan 2004) or 0.3–0.8 according to Dogorostev (Dorogovtsev 2010). A network is considered to have a significant community structure when modularity is higher than 0.3 (Clauset et al. 2004), while networks with modularity higher than 0.7 are rare (Newman and Girvan 2004). This indicates that VMN has a strong community structure robust to the detection approach.

Table 1 Modularity and the number of communities obtained as a result of searching for the community structure in the VMN using specific algorithms

Of the 7 community detection algorithms used, Label Propagation has the lowest modularity value $\left(Q=0.866\right)$; therefore, this method is rejected. The remaining approaches show consistent results in terms of maximizing the modularity value, where the obtained $Q$ is in the range $\langle 0.894{-}0.899\rangle$. Despite the closer modularity value is to 1, the better the community partition, and due to the high similarity of the result in terms of modularity score, the final choice of the method of dividing the network into communities is based on the numbers of obtained communities. The number of modules closest to the natural division of the network into 11 economic sectors is obtained by employing the HE heuristic algorithm, which is 15 clusters. In other words, the HE algorithm is selected to divide the VMN into communities that identify 15 modules with a high value of the modularity function $\left(Q=0.894\right)$, which is only 0.005 below the maximum value. This implies that the vertices are strongly connected within their community and that the HE partition is most comparable to the VMN partition by economic sector.

The community structure of the VMN is shown in Fig. 2, where the nodes are color-coded according to the module to which the stocks are assigned. The community analysis utilized to the value migration network reveals a strong and distinct community structure. This means that connections within each module are denser than would be expected on the random chance.

Fig. 2

Community structure of the VMN performed using the HE algorithm. The nodes denoting individual stocks are marked with an appropriate color according to the assignment to a given community

5.1.2 Value migration network clustering

The third partition of the VMN is carried out using the PAM method. The advantage of this approach is the ability to set a predetermined number of clusters before performing the clustering procedure. I set the number of clusters at the level corresponding to the number of economic sectors ($r=11$). This approach ensures relative comparability with the natural division of the VMN by sector classification. The mean distance to the nearest medoid is 0.699, and the average silhouette coefficient is 0.018. The results of the performed division are presented in Fig. 3.

Fig. 3

Network partition based on the PAM clustering. The vertices denoting individual companies are marked with an appropriate color due to the PAM classification

5.2 Comparison of the obtained divisions and network characteristics of cohesion and segregation

The following section includes the comparison of the three partitions in the sector–community and sector–PAM clustering pairs order. The heatmaps of the number of vertices assigned to a given group in the first division and to a given cluster in the second partition are shown in Fig. 4.

Fig. 4

Heatmap of the number of elements in the intersection of clusters of the first pair of partitions: sectors and communities (a) and the second pair of divisions: sector and PAM clustering (b)

The heatmaps show that there is no convergence of the two pairs of the VMN partition. It should be noted that cluster 4 of PAM (Fig. 4b) contains a large number of nodes assigned to different economic sectors. In other words, the PAM clustering demonstrates an over-expression of cluster 4. This is a consequence of the inconsistency of the results of the sector division and the PAM approach, as well as the large size of cluster 4 (more than 50% of nodes). These observations are confirmed by a graphical summary of the number of assets of individual sectors classified into individual communities and PAM clusters, reported in Appendix 1 (Fig. A1), included in the supporting information file associated with this article.

Next, comparative clustering measures are applied to assess the similarity between the sector classification and the partitions detected by (i) HE community searching and (ii) PAM clustering. The Adjusted Rand Index, Normalized van Dongen distance, and $w$ index are computed for two pairs, sector classification–clustering, as shown in Table 2.

Table 2 Comparative clustering measures

The ARIs obtained are 0.026 and 0.006 for partitions between sector and community divisions and between sector and PAM decompositions, respectively. The values of ARI are close to 0, which indicates that sector classification and clustering based on HE and PAM algorithms are independent. This means that the overlap between the two partitions is similar to what would be expected under the assumption of random chance. The value of the index $w$ is close to the lower value of its range $\left(1/N\right)$, confirming that the two compared pairs of partitions are independent.

Based on the ARI and the $w$ index, the community-based partition of the VMN displays greater similarity to the sector classification than the PAM grouping. However, the reverse relationship applies to the normalized von Dongen distance, where the value close to 1 occurs for non-overlapping partitions. The significantly lower value of ${D}^{n}\left(X,Y\right)$ for the PAM clustering is due to the presence of one large cluster obtained by this clustering approach. This is important because van Dongen distance compares clustering by set matching.

Three measures of cohesion and segregation for the entire network are used to assess the strength of connections between vertices belonging to the same cluster and between nodes assigned to different groups for the three partitions (Table 3).

Table 3 Cohesion and segregation measures for the entire network

The modularity obtained for community decomposition of the VMN is 0.894, previously presented in the description of the selection of the adopted algorithm for community structure detection. For the network division by sector and PAM algorithm, the corresponding modularity value as the sum of all groups is equal to  −0.07 and 0.611, respectively. The divisions corresponding to community detection and the PAM clustering reveal a strong community structure (modularity $Q>0.3$). The value of modularity close to 0 for the sector classification means that the number of connections in the groups is close to the expected value by random chance.

Additionally, the assortativity coefficients corresponding to the sector, community, and PAM partitions are -0.071, 0.961, and 0.816, respectively. The assortativity coefficients close to 1 mean that the decomposition of the VMN according to HE community detection and the PAM clustering exhibits a tendency to occur vertices from the same cluster at the two ends of the edge. On the contrary, the assortativity coefficient close to 0 obtained for the sector classification indicates that edges between nodes within the same sector are rarely encountered in a manner indistinguishable from what would be expected on a random chance.

A Quadratic Assignment Procedure (QAP) (Krackhardt 1987) is carried out to assess whether the assortativity coefficients obtained for each network partition differ significantly from those that would be expected by a random chance. The QAP is a standard approach in social network analysis used to evaluate the statistical significance of a network indicator. Based on the original matrix, the columns and rows are randomly permuted computing artificial networks independent of the previous network and the properties, especially autocorrelation, of the randomized networks are preserved. I set a permutation of 3 500 combinations per node (a total of 1,732,500 QAP permutations) to compute the expected value of the assortativity coefficient and then compare with the observed value based on the original partition of the VMN. Table 4 presents the results of the QAP analysis for the assortativity coefficient for three partitions of the VMN.

Table 4 Results of the QAP analysis for the assortativity coefficients

Since exactly the same network is considered for each partition, the expected value of the assortativity coefficients and its standard deviation are equal to each other. In the case of community and the PAM clustering, the original observed values of the assortativity coefficient are higher than the expected values with a probability of exactly 1.00.

Significant positive values of the assortativity coefficient for community and PAM partitions reveal assortative behavior where out-going edges in the VMN link more pairs of stocks within the same module than would be expected by a random chance. This means that these two methods of clustering assets in the VMN are significant. However, sector classification does not show a statistically significant at a significance level of 0.05 division in terms of edges between stocks within the same sector.

Returning to the cohesion and segregation indicators for the entire network (Table 3), the E-I index clearly shows a significant difference between the compared network divisions. The value of the E-I index for the sectoral classification is close to 1 (0.927), indicating that the vast majority of out-going edges occur between sectors. On the contrary, for the network partitioning into communities and PAM clusters, the E-I index values are close to −1, revealing a strong segmentation of the VMN.

5.3 Analysis of the cohesion and segregation of partitions of the value migration network

In this section, I conduct the segregation and cohesion analysis for specific divisions of the VMN. Intra- and extra-group connections based on out-going edges are shown in Fig. 5, where the size of the vertex is proportional to the number of internal edges. The sizes of the vertices are not comparable between the different partitions of the network. The aggregated network is constructed as presented in the description of the community centrality computation.

Fig. 5

Intra- and extra-group connections for (a) sector classification, (b) community partitioning, and (c) PAM grouping. The node thickness is proportional to the number of internal edges. These proportions are preserved only in a given division

As can be seen in Fig. 5, the sector classification is characterized by a dense network of intersectoral connections. In contrast, the partitioning of the network into communities and PAM clusters is characterized by a sparse structure of inter-group connections.

Appendix 2 (Table A1) presents a summary of the segregation and cohesion indicators for all clusters. The mean values of these measures for a given network division are shown in Table 5. In addition, box plots of the segregation and cohesion indicators are provided in Appendix 3 (Fig. A2). Appendices 2 and 3 are included in the supporting information file.

Table 5 Mean value of the segregation–cohesion indicators for each network partition

One can see from Table 5 and from Appendices 2 and 3 that measures of segregation and cohesion indicate a substantial difference between the sector classification and community/PAM cluster divisions, referred to as algorithmic partitions.

Based on the analysis presented in Appendix 4, a high level of segregation and cohesion should be stated in both algorithmic network divisions into (i) communities and (ii) clusters according to the PAM algorithm. The natural division of the network according to sector classification does not demonstrate the cohesion and segregation structure in which the external edges prevail over the internal edges within a given sector.

The MANOVA test is used to assess the equality of the mean value of the segregation and cohesion indicators between the three partitions of the VMN. In other words, the aforementioned differences in mean values between the individual network partitioning (Table 5) are tested. The null hypothesis of MANOVA, which assumes equality of mean segregation and cohesion measures for the three different decompositions, is tested at a specified significance level. The results of the multivariate MANOVA test for differences between the decompositions in terms of segregation and cohesion indicators are summarized in Table 6.

Table 6 Multivariate tests for partition differences in the segregation and cohesion measures

All four tests, Wilks’ lambda, Pillai’s trace, Lawley–Hotelling trace, and Roy’s largest root, indicate that the set of mean values of the segregation and cohesion measures have a highly statistically significant (p < 0.001) difference across the three partitions of the VMN. We therefore have a basis for rejecting the null hypothesis. The value of the $\Lambda$ statistic is close to zero (0.002) and $1-\Lambda =0.998$, which means that 99.8% of the variance of the dependent variables is explained by the considered effects. It should be noted that Pillai’s trace statistic is also statistically significant. Pillai’s criterion is considered to be the strongest and most robust statistic for violating the MANOVA assumptions. Pillai’s trace statistic is particularly useful when the sample size is small and when unequal cell sizes occur. The observed power for the statistical test is 1.0 (computed using α = 0.05), indicating that the effect size and sample size are appropriate.

In addition, each measure should be assessed separately for differences between the three network partitions. Univariate tests for the individual indicators of segregation and cohesion are presented in Table 7. Univariate tests for each dependent variable indicate that all individually considered indicators are also highly statistically significant at the 0.1% level. This means that each of the dependent variables has a significant impact on the rejection of the null hypothesis.

Table 7 Univariate tests for partition differences in segregation and cohesion measures

On the basis of multivariate and univariate tests, it can be concluded that measures of segregation and cohesion differ at a statistically significant level in the three partitions of the VMN. This is confirmed by the differences in the mean segregation and cohesion indicators, as shown in Table 5 and Fig. A2.

Although a significant main effect has been found, both collectively and individually, we can only conclude that at least one significant difference is observed between a pair of partitions of the network. In other words, the statistical significance of the main effect does not ensure that every one of the division differences is also significant. Post hoc procedures are applied to determine the statistical significance of differences between network partitions by testing all differences between network divisions. The results of the post hoc test using the two most conservative tests for Type I error, the Scheffe method and Turkey’s HSD for unequal group size, are shown in Table 8.

Table 8 Post hoc tests for individual group differences in segregation and cohesion measures

For all indicators of segregation and cohesion, there are statistically significant differences between the division according to the sector classification and the division according to communities. The observed difference in mean values of the segregation and cohesion measures is statistically significant at the 0.01% level. Statistically significant differences in the mean values between the sector classification and the PAM clustering division are observed for all measures (p < 0.001), excluding the cohesion index. The two divisions do not differ for this variable. The results above are consistent according to Scheffe and Turkey’s HSD method.

The comparison of network partitions according to the community and PAM clusters shows a statistically significant difference at the level of 5% for 4 of 8 variables, i.e. (i) E-I index, (ii) cohesion index, (iii) RCL, and (iv) RCR. For the remaining 4 variables, (i) SMI, (ii) density, (iii) group modularity, and (iv) community centrality, the two algorithmic divisions do not differ from each other. This analysis indicates that the dominance of the community division in terms of segregation and cohesion is only observed to a limited extent.

In conclusion, it can be pointed out that both algorithmic divisions of the value migration network (into communities and PAM clusters) show significantly higher average levels of segregation and cohesion than the division of the network according to the sector classification.

6 Robustness analysis

It is noticeable that community detection algorithms automatically determine the number of modules, which is instead an adjustable parameter for the PAM clustering. It should be emphasized that the number of groups produced is important. To ensure the reliability of the presented results, additional checks were performed to verify the robustness of the results with regard to the choice of alternative community finding algorithms and the different number of clusters parameterized in the PAM method. In order to save space, the fully tabulated results of the robustness analysis are included in the supporting information file attached to this article (Appendices 5 and 6).

Firstly, a test is conducted to see if the results are robust to the use of the community search algorithm. Six different algorithms (HE, CNM, HE', NE, Blondel, Eigenvector) are employed, and the mean values of the segregation and cohesion measures for each method are calculated (Table A2 and Fig. A3). The MANOVA approach is then used to verify the null hypothesis of equality of mean values of segregation and cohesion measures for the six different community detection algorithms. The results of the multivariate and univariate tests are listed in Table A3 and Table A4, respectively. Three multivariate tests indicate that there is no basis to reject the null hypothesis. This means that the mean values of the set of segregation and cohesion indicators for all used community detection algorithms are equal. Only Roy’s largest root statistic shows a statistically significant difference in the collective set of dependent indicators across the six community partitions. Although this test is most appropriate when the dependent variables are highly correlated, and the p value of Roy’s largest root statistic is always the lowest and, therefore, most often ignored when statistically significant.

Univariate tests (Table A4) and post hoc tests (Table A5) support the conclusion that the null hypothesis cannot be rejected. This indicates that the mean values of the segregation and cohesion indicators for the basic HE algorithm are not statistically significantly different from the mean values for the other five community division algorithms. The use of a different community partitioning algorithm does not bias for the baseline results obtained in Sect. 5.3.

Secondly, the results are checked with regard to the sensitivity to the number of given clusters into which the network is to be divided using the PAM clustering. For this purpose, with respect to the basic division into 11 clusters, four additional divisions of the network are carried out using the PAM algorithm. Alternative network divisions into 12, 13, 14, and 15 clusters are performed, labeled as follows: PAM-11, PAM-12, …, PAM-15.

Again, the MANOVA method is used to evaluate differences in the set of mean values of the segregation and cohesion measures between the individual divisions. The results of the analysis performed in the same way as previously are presented in TablesA6, A7, A8, A9 and Fig. A4, included in Appendix 6 (see supplementary materials). All multivariate tests lead to the unambiguous conclusion that the null hypothesis cannot be rejected. Univariate and post hoc tests lead to the same conclusions, which show that increasing the number of clusters in the PAM division does not cause statistically significant changes in the set of mean values of cohesion and segregation measures.

In summary, the robustness test demonstrates that the significant differences in segregation and cohesion measures between partition of the network according to sector classification and algorithmic division into communities or PAM clusters remain robust to the choice of an alternative community detection algorithm and the diverse number of clusters obtained by network division according to the PAM clustering.

7 Concluding remarks

In summary, I have studied the MST-based value migration network using data form 499 companies listed on the S&P500 and examined whether some stocks belonging to the same economic sector cluster together. In line with the aim of the study, I have extracted communities and clusters in the VMN to compare their structure with the natural division of the network by economic sector.

In particular, it is clearly revealed that the majority of companies are linked to companies belonging to another economic sector. It should be stated that value migration between companies on the financial market is determined by macroeconomic factors that cause the flow of value between economic sectors. On the basis of appropriately selected measures, it can be concluded that there is no structure of cohesion and segregation in the sector network division. On the contrary, strong segmentation of the value migration network is observed for algorithmic network divisions, i.e., division into communities and clusters. Importantly, MANOVA analysis showed statistically significant differences in segregation and cohesion measures between algorithmic network partitioning and sector classification. This means that the flow of value occurs between companies operating in different economic sectors. This conclusion is confirmed by the demonstration of the independence of sector classification from divisions into communities and PAM clusters based on comparative clustering measures (ARI, normalized van Dongen distance, and w index).

My research provides evidence that the VMN has a strong community structure and is robust to the detection algorithm. This applies both to the divisions of the network into communities applying the HE procedure and into clusters using the PAM method.

Finally, I observe that, in contrast to sector classification, division of the network into communities and clusters reveal assortative behavior. This means that a more intensive flow of value occurs between companies belonging to the same cluster compared to a randomized counterpart of the network. In other words, with the use of algorithmic divisions of the VMN, the network is divided into areas where a significant part of the value flow between companies within one cluster and a small-scale value flow with firms outside the cluster is observed.

The results are encouraging and should be verified with a sample from other financial markets, both developed and emerging. Future research should focus on developing a sectoral map of value migration in the financial market using a network approach to identify sectors in the inflow and outflow phases. Further studies are needed to investigate the volatility of the process of value migration through the analysis of the dynamics of the value migration network.