Star graphs induce tetrad correlations: for Gaussian as well as for binary variables

Tetrad correlations were obtained historically for Gaussian distributions when tasks are designed to measure an ability or attitude so that a single unobserved variable may generate the observed, linearly increasing dependences among the tasks. We connect such generating processes to a particular type of directed graph, the star graph, and to the notion of traceable regressions. Tetrad correlation conditions for the existence of a single latent variable are derived. These are needed for positive dependences not only in joint Gaussian but also in joint binary distributions. Three applications with binary items are given.


Introduction
Since the seminal work by Bartlett (1935) and Birch (1963), viewed in a larger perspective by Cox (1972), correlation coefficients were barely used as measures of dependence for categorical data. Instead, functions of the odds-ratio emerged as the relevant parameters in log-linear probability models for joint distributions and in logistic models, that is for regressions with a binary response. Compared with other possible measures of dependence, the outstanding advantage of functions of the odds-ratio is their variation independence of the marginal counts: odds-ratios are unchanged under different types of sampling schemes that result by fixing either the total number, n, of observed individuals or the counts at the levels of one of the variables; see Edwards (1963).
As one consequence of the importance of odds-ratios for discrete random variables, it is no longer widely known that Pearson's simple, observed correlation coefficient, r 12 say, coincides in 2 × 2 contingency tables with the so-called Phi-coefficient, so that √ nr 12 is, asymptotically and under independence of the two binary variables, a realization of a standard Gaussian distribution. As we shall see, some properties of correlation coefficients for binary variables, make them important for data generating processes that incorporate many conditional independences.
In particular, we look here at directed star graphs such as the one shown in Figure  1(a). Such graphs have one inner node, L, from which Q arrows start and point to the uncoupled, outer nodes 1, . . . , Q. To simplify notation, the inner node L denotes also the corresponding random variable and both, the node and the variable, are called root. The random variables X i corresponding to the outer nodes, also called the leaves of the graph, are identified just by their index i taken from {1, . . . , Q}. The independence structure of the directed star graph is mutual independence of the leaves given the root. In the condensed node notation, this is written as In general, the types of variables can be of any kind. Densities, f N , that are said to be generated over the directed star graph with node set N = {1, . . . , Q, L}, may also be of any kind, provided that they have the above independence structure.
imsart-ejs ver. 2013/03/06 file: papTetradFinal.tex date: May 11, 2014 Each generated density is defined by Q conditional densities, f i|L , that are called regressions, and a marginal density, f L , of the root. In the condensed node notation, a joint density with the independence structure of a directed star graph, factorizes as Directed star graphs belong to the class of regression graphs and to their subclass of directed acyclic graphs. Distributions generated over regression graphs have been named and studied as sequences of regressions and one knows when two regression graphs capture the same independence structure, that is when they are Markov equivalent, even though defined differently; see Theorem 1 in Wermuth and Sadeghi (2012). In particular, each directed star graph is Markov equivalent to an undirected star graph with the same node and edge set, such as in Figure 1(b), since it does not contain a collision V: • ≻ • ≺ •, that is two uncoupled arrows meeting head-on.
Sequences of regressions are traceable if one can use the graph alone to trace pathways of dependence, that is to decide when a non-vanishing dependence is induced for an uncoupled node pair. For this, each edge present in the regression graph corresponds to a non-vanishing, conditional dependence that is considered to be strong in a given substantive context. In evolutionary biology, the required changes in dependences that are strong enough to lead to mutations, were for instance called 'drastic' by Neyman (1971). In general, special properties of the generated distributions are required in addition; see Wermuth (2012).
In the last century, regression graphs were not yet defined and properties of traceable regression unknown. Then, distributions generated over directed star graphs, in which the root L corresponds to a variable that is never directly observed, have been studied separately under different distributional assumptions and with changing main objectives. Many of them were named item response models. In these contexts, the unobserved root L is also called latent or hidden, the items are the observed variables.
For instance, Spearman (1904) suggested to measure general intelligence with Q similar quantitative tasks, a method now called factor analysis with a single latent variable. For confirmatory factor analyses, typically, a Gaussian distribution is assumed, after the observed variables are standardized to have mean zero and unit variance. Heywood (1931) and Anderson and Rubin (1956) derived necessary and sufficient conditions for the existence of one or more latent variables, without the constraint of non-vanishing, positive dependences of each item on the latent variables. The latter assumption arises however naturally when the items are to measure a specific latent ability or attitude or when they are the symptoms of a given disease.
For instance, when psychologists try to measure for children in a given age range, what is called the working memory capacity, then each item is the successful repetition -in reverse order -of a sequence of numbers. Typical tasks of the same difficulty are imsart-ejs ver. 2013/03/06 file: papTetradFinal.tex date: May 11, 2014 the sequences (3,5,7) and (2,4,6). The item difficulty increases, for instance, by presenting more numbers or numbers of two digits. For children with a higher capacity of the working memory, one expects more successes for tasks of the same difficulty as well as for tasks of increasing difficulty.
When instead, the leaves and the root of the star graph are both categorical, the resulting model is a latent class model, as proposed by Lazarsfeld (1950), again with an extensive follow-up literature. An important warning was given by Holland and Rosenbaum (1986): such a model can never be falsified when the number of levels of the latent variable is large compared to the number of cells in the observed contingency table. Expressed in other words, merely requiring conditional independence of categorical items given the latent variable imposes then no constraints on the observed distributions.
A general, testable constraint suggested by Holland and Rosenbaum (1986), that is now widely adopted in nonparametric item response theory, is to have a monotonically non-decreasing association of each item on L; see van der Ark (2012) or Mair and Hatzinger (2007). However, the underlying notion of 'conditional association', that had been proposed in probability theory, includes conditionally independent variables as being 'conditionally associated'; see Esary, Proschan and Walkup (1967). By contrast, when one uses traceable regressions, each edge present in a graph excludes explicitly a corresponding conditional independence statement.
In this paper, we study similarities of joint Gaussian and of binary distributions generated over directed star graphs where all dependences of leaves, 1, . . . , Q, on the root, L, are positive and the root is unobserved but does not coincide with any leaf or with any combination of the leaves. In Section 2, we summarize results for star graphs and for their traceable regressions. In Section 3, we describe joint Gaussian distributions, so generated, and in Section 4, we study joint binary distributions, especially correlation constraints on the distribution of the leaves. Section 5, gives applications to binary distributions, Section 6 a general discussion. The Appendix contains a technical proof.

Marginalizing over the root in star graphs
A regression graph is said to be edge-minimal when each of its edges, that is present in the graph, indicates a non-vanishing dependence. Definition 1. Traceable regressions are generated over an edge-minmal, directed star graph, if (a) the density factorizes as in equation (2) and (b) is dependence-inducing by marginalizing over the root L, that is yields for each pair of leaves i, j a bivariate dependence, denoted by i ⋔ j.
Thus, for traceable regressions with exclusively strong dependences, i ⋔ L, in the generating star graph, each ij-edge in the induced complete covariance graph of the leaves, drawn as in Figure 2 distributions that do not induce such a dependence violate singleton-transitivity; see Table 1 in Wermuth (2012) for an example of such a family of distributions.
More consequences can be deduced, using strong dependences i ⋔ L and the Markov equivalence of the directed to the undirected star graph with the same node and edge set, as in Figures 1(a) and 1(b). For a traceable distribution with an undirected star graph, such as in Figure 1(b), each edge in the induced complete concentration graph, obtained by marginalising over the root L and drawn as in Figure 2(b), indicates a non-vanishing conditional dependence of each pair of leaves given the remaining leaves, denoted by i ⋔ j|{1, . . . , Q} \ {i, j}. In exponential family distributions, a covariance graph corresponds to bivariate central moments, a concentration graph to joint canonical parameters and a directed star graph to regression parameters. If Markov equivalent models are also parameter equivalent, then their parameter sets are in a one-to-one relation. This property does not hold in general, but for instance in joint Gaussian distribution, and in joint binary distributions that are quadratic exponential. More generally, Markov equivalence and parameter equivalence coincide when only a single parameter determines for each variable pair whether the pair is conditionally dependent or independent.
For traceability of a given sequence of regressions, the generated family of distributions needs to have three properties of joint Gaussian distributions, that are not common to all probability distributions. In addition to the dependence-inducing property (singletontransitivity) stated in Definition 1, these are the intersection (downward combination) and the composition property (upward combination of independences); see Lněnička and Matúš (2007) equations (9) to (10)  In the context of directed star graphs, these two properties are a consequence of the special type of generating process. After removing any two arrows for Q ≥ 5, a directed star graph in Q − 2 arrows remains. Thus, by introducing two additional pairwise independences in a directed star graph, these independence statement combine downwards. The Markov equivalence of the directed to the undirected star graph implies that for any two nodes, the statement i ⊥ ⊥ j|L can be modified to i ⊥ ⊥ j|Lc with c ⊆ {1, . . . Q} \ {i, j} so that pairwise independences combine upwards.

Gaussian distributions generated over directed star graphs
In Gaussian distributions, each dependence is by definition linear and proportional to some (partial) correlation coefficient. Furthermore, there are no higher than two-factor interactions. In a traceable Gaussian distribution generated over a directed star graph, each directed edge indicates a simple, strong correlation, ρ iL , called the loading of item i on L. Then for each item pair (i, j) a simple correlation, ρ ij is induced via where ρ ij|L denotes the partial correlation coefficient In factor analysis, the latent root L is assumed, without loss of generality, to have mean zero and unit variance. Typically, when the observed variances of the items are nearly equal, the items are transformed to have mean zero and unit variance, so that their correlation matrix is analyzed.
In general, the model parameters are known to be identifiable for Q > 2; see for instance Stanghellini (1997). For Q = 3 items, the positive loadings ρ 1L , ρ 2L , ρ 3L can be completely recovered using equations (3) for the positive, partial correlations ρ ij|k of each leaf pair. The maximum-likelihood equations (Lawley, 1967) reduce to the same type of three equations so that a unique, closed form solution can be obtained, provided it exists, and be written asλ T = (ρ 1Lρ2Lρ3L ) witĥ ρ 1L = r 12 r 13 /r 23 ,ρ 2L = r 12 r 23 /r 13 ,ρ 3L = r 13 r 23 /r 12 .
Clearly, these equations require for permissible estimates: 0 <ρ iL < 1. In that case, there can be no zero and no negative marginal or partial item correlation that cannot be removed by recoding some items. We give in Table 1    In the example on the left, ρ 23|1 = 0. By equation (4), the estimated loading of item 1 is then equal to one, i.e.ρ 1L = 1, so that item 1 cannot be distinguished from the latent variable itself. For the example in the middle,ρ 1L = 0.6 · 0.5/0.1 = √ 3 is larger than one, hence leads to an infeasible solution for a correlation coefficient. Thus even for a positive, invertible item correlation matrix, there may not exist a generating process via positive loadings on a single latent variable. The example on the right, has a perfect fit for the vector of estimated positive loadings in equation (4) For Q > 3 items, proper positive loadings in equation (3), that is 0 < ρ iL < 1 for all items i, lead directly to a positive correlation matrix of the items, denoted in this paper by P > 0, that is to exclusively positive correlations, ρ ij = ρ iL ρ jL for i = j in {1, . . . , Q}, and to a tetrad structure. Vanishing tetrads were defined by Spearman and coauthors and nowadays a popular search algorithm for models with possibly several latent variables is named Tetrad. This algorithm is based on and extends work by Spirtes, Glymour andScheines (1993, 2001).
Definition 2. A positive tetrad correlation matrix S has dimension Q > 3 and elements s ij such that 0 < s ij < 1 for all item pairs i, j and s ih /s jh = s ik /s jk for all distinct i, j, h, k taken from {1, . . . , Q}.
Thus for any pair i, j, the ratio of its correlations to variables in row h of S is the same as to variables in row k, or, equivalently, there are vanishing tetrads: s ih s jk −s jh s ik = 0. Let λ T denote a row vector of proper positive loadings, ∆ a diagonal matrix of elements 1 − ρ 2 iL , and δ T a row vector of elements iL ), then, as proven in the Appendix, the correlation matrix of the leaves P and its inverse, the concentration matrix P −1 , are Some important direct consequences of (6) are given next. Lemma 1 uses the notion of M(inkowski)-matrices that were named and studied by Ostrowski (1937Ostrowski ( , 1956) and discussed in connection to totally positive Gaussian distributions much later using the name MTP 2 distributions; see e.g. Karlin and Rinott (1983). General MTP 2 distributions are characterized by having no variable pair negatively associated given all remaining variables. We will use a more strict form of MTP 2 that also excludes any variable pair being conditionally independent given all remaining variables. Definition 3. A square, invertible matrix is a complete M-matrix if all its diagonal elements are positive and all its off-diagonal elements are negative. Lemma 1. A positive tetrad correlation matrix, P , generated over a star graph with proper positive loadings, 0 < ρ iL < 1, is invertible and P −1 is a symmetric, complete M-matrix with vanishing tetrads.
When we denote the elements of P −1 by ρ ij , then by Lemma 1 and Definition 3, all the precisions, ρ ii , are positive and all concentrations are negative, that is ρ ij < 0, for all i = j, if P −1 is a complete M-matrix. The following important properties of complete M-matrices result from Ostrowski (1956), Section 1.

Lemma 2.
A symmetric matrix m, which has an inverse complete M-matrix, (i) has exclusively positive elements, that is m > 0, and (ii) the inverse of every principal submatrix of m is a complete M-matrix.
Thus, if the concentration matrix P −1 of observed Gaussian items has exclusively negative off-diagonal elements, then all concentrations in every marginal distribution of the items are negative as well and P > 0. Proof. The result follows with the following general relation of elements of a concentration matrix to partial correlations, together with Lemma 1 and Lemma 2.
Thus, for a complete concentration matrix of Gaussian items, negative concentrations mean positive dependence and every item pair is positively dependent, no matter which conditioning set is chosen. Now, a known rank-one condition for the existence of a Gaussian item correlation matrix P * of a single factor simplifies as follows.
Proposition 1. Equivalent necessary and sufficient conditions for a Gaussian distribution. For Q > 3, the following statements are equivalent: (i) a P * > 0 is generated over a star graph and with proper positive loadings, ρ * iL , (ii) a P * > 0 minus a diagonal matrix, of elements δ * ii with 0 < δ * ii < 1, has rank one, (iii) a tetrad P * > 0 of the items is formed by proper positive loadings, (iv) there is a concentration M-matrix (P * ) −1 of the items having vanishing tetrads, (v) there are item partial correlations, given the Q−2 other items, which form a positive tetrad correlation matrix.
Conditions (i) to (iii) in Proposition 1 involve dependences of the items on the latent variable L, that is the unobserved loadings. In some applications, prior knowledge may be so strong that a positive loading can be safely predicted for each selected item, but otherwise, these characterizations involve unknown parameters.
By contrast, conditions (iv) and (v) in Proposition 1 concern directly the distribution of only the observed items. Equivalence to the former conditions for existence become possible by the added special properties of the concentration matrix (P * ) −1 being a complete M-matrix with vanishing tetrads.
The following example shows that a positive, tetrad correlation matrix alone, does not assure that its inverse is a M-matrix. The example is another Heywood case, conditions (iv) and (v) of Proposition 1 are violated: where the dot-notation indicates symmetric entries. This correlation matrix cannot have been generated over a directed star graph with proper positive loadings. If these correlations were observed, one would get with equation (4) thatρ 1L > 1, that is not a permissible solution. By contrast, every P −1 that is a complete M-matrix with vanishing tetrads implies a positive tetrad correlation matrix.

Binary distributions generated over directed star graphs
When Q binary items are mutually independent given a binary variable L and the factorization of the joint probability in equation (2) cannot be further simplified, since each item has a strong dependence on L, then it is a traceable regression generated over a directed star graph. The reason is that binary distributions are, just like Gaussian distributions via equation (3), dependence-inducing, that is (i ⋔ L and j ⋔ L) =⇒ at most i ⊥ ⊥ j|L or i ⊥ ⊥ j but never both.
The property assures for star graphs with the independences of equation (1), i ⊥ ⊥ j|L, that i ⋔ j is implied for each item pair. In applications, strong dependences of each item i on L are needed to obtain relevant dependences for each leaf pair i, j. The joint binary distribution generated over a directed star graph is quadratic exponential since the Q largest cliques in this type of a decomposable graph contain just two nodes, an item and L. Expressed equivalently, in the log-linear model of an undirected star graph, as in Figure 1(b), the largest, non-vanishing log-linear interactions are positive 2-factor terms, α iL . These are canonical parameters in the generated binary quadratic exponential family. Marginalizing over L in such distributions with α iL > 0 gives a tetrad form for the canonical parameters in the observed item distribution; see Cox and Wermuth (1994), Section 3: With α iL > 0 for all i, the joint binary distributions of the items have exclusively positive dependences. In general, the parameters α iL are identifiable for Q ≥ 3, see Stanghellini and Vantaggi (2013). But equation (9) does not lead to a an explicit form of the induced bivariate dependences for the item pairs. For this, we write for instance for any two items A, B, and L, each with levels 0 or 1, as well as There are three equivalent expressions of Pearson's correlation coefficient for the binary items 1 and 2. To present these, we use κ 12 = √ π 0++ π 1++ π +0+ π +1+ and abbreviate the operation of taking a determinant by det(.) : = (π 11+ − π 1++ π +1+ )/κ 12 , Equation (10) shows that the correlation is given by the cross-product difference, that it is zero if and only if the odds-ratio, defined as the cross-product ratio, equals one and that a positive correlation is equivalent to a positive log-odds ratio. Equation (11) gives as numerator the covariance and as denominator the product of two standard deviations. This is the usual definition of Pearson's correlation coefficient, here for binary variables, and it implies that ρ 12 together with the one-dimensional frequencies of items 1 and 2 give the counts in their 2 × 2 table. Equation (12) leads best to our main new result.
From equations (2) and (12), we have for each source V of the star graph, that is for each configuration A≺ L ≻B, a trivariate binary distribution with A ⊥ ⊥ B|L and π AB = π AL (π L ) −1 π LB , .
Hence, the existence of a correlation matrix P * becomes relevant for binary variables. Proposition 2. Equivalent necessary conditions for joint binary distributions. For Q > 3 items to be generated over a directed star graph with a binary root L (i) there is a tetrad correlation matrix P * > 0 formed by proper positive loadings, ρ iL , (ii) there are item partial correlations, given the Q−2 other items, which form a positive tetrad correlation matrix.
Proof. Premultiplying equation (13)  gives, for 0 < ρ iL < 1 for all i, with equation (12), that is after taking determinants, ρ 12 = ρ 1L ρ 2L > 0 . Since this holds for all item pairs, and the positive tetrad correlation matrix is a consequence of the generating process. The same arguments as in Proposition 1 give the equivalence of the two statements.
This result corrects a claim in Cox and Wermuth (2002) that a tetrad condition does not show in correlations of binary items but only in their canonical parameters in the induced distribution for the Q items: just as in Gaussian distributions, a positive, invertible tetrad correlation matrix, P , is induced for binary items if the dependence of each item on the latent binary L is positive, that is if ρ iL > 0 for all i.
Nevertheless, the directly relevant dependences for the induced, complete concentration graph model are the canonical parameters, the log-linear interaction terms that are functions of the odds-ratios. For instance, for a binary distribution generated over a star graph to have a general MTP 2 distribution, the condition is α iL ≥ 0, while for a strictly positive subclass, the condition is α iL > 0 for all items i.
Necessary and sufficient conditions, that a density of equation (2) has generated the observed distribution of Q = 3 items, have been derived as nine inequality constraints on the probabilities of the item by Allman et al. (2013) without noting their relation to MTP 2 distributions. For π ijk > 0, the first three reduce to π 111+ /π 011+ ≥ π 100+ /π 000+ , π 111+ /π 101+ ≥ π 010+ /π 000+ , π 111+ /π 110+ ≥ π 001+ /π 000+ , so that for each leaf, the odds for level 1 to 0 when the levels of the other two leaves match at level 0 do not exceed those with matches at level 1. Their last six constraints require nonnegative odds-ratios for each leaf pair, that is a binary MTP 2 distribution.
It can be shown that the above three inequalities are satisfied, whenever each leaf pair i, j has a positive conditional dependence given the Q − 2 remaining leaves, that is if the leaves have a strictly positive distribution. Therefore, this strict form of a MTP 2 binary distribution of the leaves is also sufficient for just Q = 3 leaves to have been generated over a star graph with positive dependences of the leaves on the root. This implies but is not equivalent to P −1 having exclusively negative off-diagonal elements.
imsart-ejs ver. 2013/03/06 file: papTetradFinal.tex date: May 11, 2014 More complex characterizing inequality constraints on probabilities of the leaves for Q > 3 categorical variables, when leaves and root have the same number of levels, are due to Zwiernik and Smith (2011). It remains to be seen how they simplify for binary MTP 2 distributions of the leaves or with a complete, tetrad P −1 .

Applications
We use here three sets of binary items. The first is a medical data set, the last two are psychometric data sets where the questions were chosen to expect strong positive dependences of each item on a latent variable. As discussed above, to check conditions for the existence of a single latent variable, that might have generated the observed item dependences, we use here mainly the observed item correlation matrices and the observed marginal tables of all item triples. In the first two cases, no violations are detected. In the third case, item correlation matrices alone provide already enough evidence against the hypothesized generating process. Algorithms to compute maximum-likelihood estimates for latent class models, are widely available; see for instance Linzer and Lewis (2011).

Binary items indicating gestosis that may arise during pregnancy
Worldwide, EPH-gestosis is still the main cause for a woman's death during childbirth and a major risk for death of the child during birth or within a week after birth. It is until today not a well-understood illness, rather it is characterized by the occurrence of two or more symptoms, of edema (E:=high body water retention), proteinuria (P:=high amounts of urinary proteins) and hypertension (H:=elevated blood pressure).
Little research into causes of EPH-gestosis appears to have been undertaken during the last 50 years, possibly because in higher developed countries its worst negative consequences are avoided by intervening when two of the symptoms are observed. Our data are from the prospective study 'Pregnancy and Child Development' in Germany, see the research report of the German Research Foundation (DFG, 1977). The symptoms were recorded before birth for 4649 pregnant women.
As a convention, we order in this paper counts reported in vectors such that the levels change from 0 to 1 and the levels of the first variable changes fastest, those of the second change next and so on. The observed counts for the gestosis data are then exclude negative dependences. The observed relative frequencies satisfy the relations of equation (4) directly. Thus, the above observed counts support the hypothesis of a generating directed star graph with a latent binary root.
Some additional features of the data are given next. The first symptom (E) is present for 3.7%, the second (P) for 4.2% and the third (H) for 24.8% of the women. Of the symptom pairs, E,P are seen for 65, E,H for 181 and P,H for 166 of 1000 women and all three symptoms for 41 of 1000 women. The bivariate dependences are strong and positive; with values for the odds-ratios of 5.5 for E,P, 3.0 for E,H and 2.0 for P,H.
The corresponding correlations look smaller than expected for quantitative features because E and P are rare symptoms. They have values 0.13 for E,P, 0.11 for E,H, and 0.07 for P,H and the inverse of the observed correlation matrix is a complete M-matrix.   Table 2; used is the order E,P,H,L. The identity matrix within the matrix of partial correlations given the remaining two variables indicates the perfect fit of the correlations to the hypothesized generating process via a star graph.

Binary items in a small depression scale
From an evaluation study of a short depression scale developed by Hardt (2008), we use four binary items. The answers (no:=0, yes:=1) are to the questions: feeling hopeless (item 1, with 35.1% yes), dispirited (item 2, with 27.6% yes), empty inside (item 3, with 24.2% yes), loss of happiness (item 4, with 33.0% yes). The observed row vector of counts is, again ordered as described in section 5.1, All conditional odds-ratios for items 1 and 2 are positive, with values 12.6, 1.9, 17.3, 6.0, respectively for levels (00, 10, 01,11) of items 3 and 4. Even though there are 1008 respondents, some subtables contain only small numbers. Especially for many items, tests in such tables have little power and may therefore not be very informative. We therefore concentrate on trivariate subtables.
For each triple of the items, we show in Table 3 studentized log-linear interaction parameters for the 2-factor terms and the 3-factor term. Each is a log-linear term estimated under the hypothesis that it is zero and divided by an estimate of its standard deviation; see for instance Andersen and Skovgaard (2010). To simplify the display, we list the involved item numbers, but use the same notation for interaction terms, for instance AB is for the first two listed items, for items 1,2 in the triple 1,2,3 but for items 2,3 in the triple 2,3,4. The 3-factor interactions for items 1,2,3 and for 1,3,4 are not negligible but all interaction terms are positive and the 3-factor term is always smaller than any of the 2-factor terms.
The similarity of the positive dependences at each level of the third variable shows here best in the two conditional relative risks for the first pair in each item triple: (5.9, 1.5; 5.6, 1.9; 5.7, 1.3; 5.7, 2.0).
The marginal observed correlations, are shown next with Table 4(a), in the lower triangle, and the partial correlations computed with equation (7) in the upper triangle. The same type of display is used in Table 4(b) with an estimated vectorλ T of loadings, ρ iL , added in the last row, computed as if the correlation matrix were a sample from a Gaussian distribution.  also positive. Thus, these two necessary conditions for the existence of a single latent variable are satisfied by the given counts. Withρ iL added in Table 4(b), the partial correlations for each item pair, given the remaining two items and the latent variable, are quite close to zero, as they should be under the model.

Binary items in a failed attempt to construct a scale
In the same study of the previous section, the participants were asked whether their parents fought often. For 538 respondents who answered yes, answers to reasons of the fighting were (no:=0, yes:=1) to: hot temper (item 1, with 50.0% yes), money (item 2, with 46.5% yes), alcohol (item 3, with 36.1% yes), jealousy (item 4, with 24.3% yes). The observed vector of counts is, again ordered as described in section 5.1, 46 113 63 55 59 14 42 15 10 23 17 17 13 10 19 22).
Tables 5(a) and 5(b) contain marginal correlations in the lower half of a matrix; Table 5(a) for all four items and Table 5(b) for items 1 to 3. In the upper half are the partial correlations, computed with equation (7) from the corresponding overall concentration matrices. In Table 5(a), item 1 has negative dependences on items 2 and 3, the one on item 3 is strong ( √ 538(−0.28) = −6.5). This correlation cannot result even when ρ 3L = 0, let alone when ρ 3L > 0. For the three remaining items, the correlations in Table 5(b) are all positive, but too small to support the hypothesized generating process with a useful indicator variable L.

Factor analysis applied to data with concentration M-matrices
After Spearman had introduced factor analysis in 1904, he and others claimed that the vanishing of the tetrads is necessary and sufficient for the existence of a single latent factor. It was Heywood (1931) who proved that the condition was only necessary when imsart-ejs ver. 2013/03/06 file: papTetradFinal.tex date: May 11, 2014 negative correlations (marginal or partial) are also permitted. His results implied that 0 ≤ ρ ij.k ≤ 1 for all distinct item triples is needed, in addition to vanishing tetrads, for a necessary and sufficient condition, in general. The same result was proven by Andersen and Rubin (1956) using a rank condition.
Heywood mentions that under some sort of strict positivity, a vanishing of tetrads may indeed give a necessary and sufficient condition, but the relevant properties of a Mmatrix were unknown at the time. These relevant features, used here in Proposition 1 (iv), were derived by Ostrowski (1956) without having any applications in statistics in mind. The connection of M-matrices to Gaussian concentration matrices was only recognized much later by Bolviken (1982). It may also be checked that Spearman's (1904) applications lead to complete concentration M-matrices.
For Gaussian distributions, complete concentration M-matrices define a subclass that is even more constrained than the one that is MTP 2 , where off-diagonal zeros may arise and indicate conditional independence; see e.g. Karlin and Rinott (1983).
Proposition 1 concerns a generating process via a directed star graph for a Gaussian correlation matrix. Important are the two equivalent constraints, (iv) and (v), on only the observable distribution: the concentration matrix of the leaves is a complete M-matrix with vanishing tetrads and partial correlations of leaf-pairs i, j, given the remaining Q − 2 leaves form a positive tetrad correlation matrix. The second of these two is easier to recognize due to the scaling of correlations.

Applications of binary star graphs
Joint binary distributions generated over general directed star graphs are extremely constrained; see Figure 1 in Allman et al. (2013) and even more for strictly positive conditional dependences of the leaves on the root, as these define a special subclass of the general binary MTP 2 family, studied by Bartolucci and Forcina (2000).
Nevertheless, the structure in the medical and in one psychometric data set of Section 5.3 can be explained by such a generating process. There, strong prior knowledge about ρ iL , based on observing many patients, permits to select suitable sets of symptoms or items: these are three symptoms for EPH-gestosis and four suitable binary items that may indicate depression.
Inequality constraints on probabilities of the joint distribution of the leaves have been given by Zwiernik and Smith (2011) in their Proposition 2.5, restated in simplified form for P > 0 and Q = 3 by Allman et al (2013). The latter are discussed here using equation (4). Compared to Gaussian distributions, Proposition 2 contains the same conditions on the correlations of the leaves and on the partial correlations of the leaves. The latter are easy to check but they are for general types of binary variables only necessary conditions.
Applications of phylogenetic star graphs and trees, as started by Lake (1994), were based on incomplete characterizations that have been completed only recently by Zwiernik and Smith (2011). It is still unclear whether the history of factor analysis may repeat itself in this context: the current applications are plagued by infeasible solutions, but possibly there are unknown characterizing features for some situations, under which these problems are always avoided. Though Lake's 'paralinear distance measure' reduces to − log(|r ij |) for Gaussian and for binary variables, little is known about the distribution of the corresponding random variables. Even for a Gaussian parent distribution, the variance in the asymptotic Gaussian distribution ofρ involves the unknown ρ unless ρ = 0. To construct confidence intervals for ρ = 0, one uses Fisher's z-tranformation, which lacks this undesirable feature. However, for other than Gaussian parent distributions, even the z-transformed, correlation coefficient estimator depends in general on the unknown ρ; see Hawkins (1989).
Sometimes, as for the data in Section 5.3 above, the absolute value of an observed negative simple or partial correlation is so large that it clearly contradicts the existence of a generating star graph with only proper positive dependences on the root. But, correlations alone or constraints on the population probabilities alone, such as in equations (4), cannot help to decide whether an observed negative dependence, r ij ≤ 0, may arise from a population in which ρ ij > 0.

Machine learning procedures for star graphs with a latent root
In the machine learning literature, it is considered to be one of the simpler tasks to decide whether a joint binary distribution has been generated over a directed star graph.
However, when a learning strategy is based on only the bivariate binary distributions, no joint distribution may exist for a set of given bivariate distributions. In the spirit of Zentgraf's (1975)  The conditional odds-ratio of A, B given C, at level 0 of 0.13 and given C at level 1 of 53.1, show qualitatively strongly different dependences of A, B given C. When inference is based only on the bivariate tables, one implicitly sets the third-order central moment to zero and keeps all others unchanged. Transforming this vector back to probabilities gives negative entries and hence shows that no joint binary distribution exists when the log-linear, three-factor interaction is falsely taken to be zero. 7. Appendix: The inverse of a positive tetrad correlation matrix It was known already to Bartlett (1951), that an invertible tetrad correlation matrix implies a tetrad concentration matrix. His proof is in terms of the general form of the inverse of sums of matrices. It is more direct to give the overall concentration matrix in explicit form. For this, we use the partial inversion operator of Wermuth, Wiedenbeck and , described in the context of Gaussian parameter matrices in Marchetti and Wermuth (2009). It can be viewed as a Gaussian elimination technique (for some history see Grcar, 2011) and as a minor extension of the sweep operator for symmetric matrices discussed by Dempster (1972). This extension is to invertible, square matrices so that an operation is undone by just reapplying the operator to the same set.
Let M be a square matrix of dimension d for which all principal submatrices are invertible. To describe partial inversions on d, we partition M into a matrix m of dimension d − 1, column vector v, row vector w T and scalar s The transformation of m is also known as the vector form of a Schur complement; see Schur (1917). For M a covariance matrix, v/s contains d − 1 linear, least-squares regression coefficients and m − vw T /s is a residual covariance matrix. For partial inversion on a set a ⊆ {1, . . . , d}, one may conceptually apply equation (16) repeatedly for each index k of a: one first reorders the matrix M so that k is the last row and column, applies equation (16) and returns to the original ordering. Several useful and nice properties of this operator have been derived, such as commutativity and symmetric difference.
For Q = 3 leaves and the correlation matrices of a star graph models, more detail is .
where the . -notation indicates an entry that is symmetric, the ∼ -notation an entry that is symmetric up to the sign. For Q items, a single root L and Ψ denoting their joint correlation matrix, we have for instance inv L Ψ = inv 1,...,Q Ψ −1 , Ψ −1 = inv 1,...,Q (inv L Ψ).
For Q > 3, the structure of these matrices is preserved in the sense that there is a diagonal matrix ∆ containing 1 − ρ 2 iL as elements, a row vector λ T with loadings ρ iL , the precision of L as s = 1 + i ρ 2 iL /(1 − ρ 2 iL ), and a row vector δ T with elements −ρ iL /{ √ s(1 − ρ 2 iL )}. For P the correlation matrix of the Q items that are uncorrelated given L, one gets P as the submatrix of rows and columns {1, . . . , Q} of inv L (inv L Ψ) and P −1 as the submatrix of rows and columns {1, . . . , Q} of inv L Ψ −1 so that Thus, for P −1 a complete M-matrix with vanishing tetrads, P > 0 has tetrad form.