On the Geometry of Discrete Exponential Families with Application to Exponential Random Graph Models

There has been an explosion of interest in statistical models for analyzing network data, and considerable interest in the class of exponential random graph (ERG) models, especially in connection with difficulties in computing maximum likelihood estimates. The issues associated with these difficulties relate to the broader structure of discrete exponential families. This paper re-examines the issues in two parts. First we consider the closure of $k$-dimensional exponential families of distribution with discrete base measure and polyhedral convex support $\mathrm{P}$. We show that the normal fan of $\mathrm{P}$ is a geometric object that plays a fundamental role in deriving the statistical and geometric properties of the corresponding extended exponential families. We discuss its relevance to maximum likelihood estimation, both from a theoretical and computational standpoint. Second, we apply our results to the analysis of ERG models. In particular, by means of a detailed example, we provide some characterization of the properties of ERG models, and, in particular, of certain behaviors of ERG models known as degeneracy.


Introduction
Our motivation for the work described in this paper comes from the analysis of network data using models representable by graphs, where the nodes correspond to individuals and the edges to relations or linkages among them. Such graphical representation has a long history, dating back to Moreno (1934), and was recast within the exponential family framework by Holland and Leinhardt (1981) and Frank and Strauss (1986) (see also Strauss and Ikeda, 1990). Their work led to the development of the broader class of exponential random graph (ERG), or p * , models for social networks (see, e.g. Wasserman and Pattison, 1996), but likelihood methods for their analysis remained out of reach until earlier this decade. For a broad review of these and other network models, see Goldenberg et.al. (2009). Recent work on maximum likelihood estimation for ERG models, however, has pointed to difficulties that have been characterized as "degeneracies" or "near degeneracies" by Handcock (2003) and Hunter et al. (2008). The explanation for these difficulties lies within broader characterizations of "degeneracies" for discrete exponential families.
Exponential families are one of the most important and widespread class of parametric statistical models, whose remarkable properties have long been established in the statistical literature (see, e.g., Bardoff-Nielsen, 1978;Brown, 1986;Letac, 1992). Among the most interesting features of exponential families is the notion of the closure of the family, known as the extended exponential family, whose mathematical theory has been recently worked out in great generality (see Csiszár and Matúš, 2001, 2003, 2005, 2008. The study of the extended families is particularly important, as it may directly pertain to the existence of the maximum likelihood estimates and to the estimability of the natural parameters. This is the case for discrete exponential families, for which the maximum likelihood estimates may not exist with some positive probability. A notable instance is the class of log-linear models, for which existence of the MLE and closure of the family can be characterized in a purely geometric fashion (see, e.g., Eriksson et al., 2006;Geiger et al., 2006;Rinaldo, 2006a).
In this article we are concerned with discrete linear exponential families. In the first part of the paper, we show that the geometric and statistical properties of the extended family depend in a fundamental way on the normal fan of the convex support. In particular, the normal fan can be used to characterize nonidentifiability of the families in the closure, to represent the densities in the extended family as almost sure limits of the densities in the original family along certain directions of the parameter space and to describe the directions of recession of the (negative) log-likelihood function.
As an application of our results, in the second part of the paper we turn our attention to exponential random graph models, a particular class of discrete linear exponential families. Our discussion is based an the detailed analysis of the ERG model on a the graphs on 9 nodes with two-dimensional sufficient statistics consisting of the number of edges and the number of triangles. We use Shannon's entropy function to illustrate graphically how concentrated the distributions in this family are, viewed as functions of both the natural and mean value parameters. Besides illustrating the theoretical results derived in the first part of the article, our analysis sheds light on a variety of pathological behaviors observed in practice while fitting ERG models known as degeneracy (see, e.g., Handcock, 2003), and, more generally, on the qualities and attributes of ERG models. Our analyses indicate that perhaps network analysts and methodologists attribute to ERG models a degree of regularity that they may not possess.
The remainder of this article is organized as follows. In Section 2 we provide the derivation of our ket theoretical results. In Section 2.1, we begin by describing our settings and briefly review the theory of extended exponential families and their fundamental properties. Then Section 2.2, we introduce the notions of normal cones and the normal fan to the convex support of the family. In Section 2.3 we state our main result and a discussion of its corollaries, while Section 2.4 presents come computational considerations concerning maximum likelihood estimation for extended exponential families. Section 3 consists of an application of our results to ERG models. First in Section 3.1 we introduce the class of ERG models and then in Section 3.2 we present our running example of an ERG model on the set of all graphs on 9 nodes. We next introduce the concept of degeneracy for ERG models in Section 3.3, while in Section 3.4 we use our theoretical results to illustrate graphically the features of the model in the running example of Section 3.2 to show how degeneracy arises. The appendices contains the proofs and some additional result on how to establish existence of the maximum likelihood estimates in discrete linear exponential families using linear programming.
We end this section by establishing the notation we will be using throughout. For two vectors x and y in x i y i denotes their inner product. The Eucludean norm of a vector x is x 2 = x, y . If A is a subset of R d , we indicate with convhull(A) its convex hull and with cone(A) the set of all of its conic combinations. Finally, for any A ⊂ R d , possibly of dimension less than d, its relative interior ri(A) is its interior relative to convhull(A).

Settings
In this section we introduce the statistical and geometric background needed for our results. We will assume throughout some familiarity with the general theory of exponential families and the basics of polyhedral geometry. For more complete treatments, consult Bardoff-Nielsen (1978), Brown (1986), Csiszár and Matúš (2001, 2003, 2005, 2008 and Rinaldo (2006a) for material on exponential families, and Ziegler (1996) and Schrijver (1998) for introductions to polyhedral geometry.
We consider an exponential family of distributions E P on R k with densities with respect to some base measure ν, where is the natural parameter space and ψ(θ) = log R k exp x,θ dν(x) the log-partition function. The support of E P is the closure of the set {x : ν(x) > 0}, while the convex support P is the closure of the convex hull of the support of E P . We will assume throughout the paper that (A1) ν has countable support; (A2) P is a full-dimensional polyhedron in R k , that is, P does not belong to any proper affine subspace of R k ; (A3) for each face F of P, F = convhull(S F ), for some set S F ⊂ supp(ν); (A4) the natural parameter space Θ is an open set.
Assumptions (A1) and (A2) imply, in particular, that the family is in minimal form and, therefore, identifiable. We remark that assumption (A2) is not necessary and is imposed to simplify the exposition; our results would still hold with some minor changes without assumption (A2), and the cost of additional technicalities in the proofs. In fact, any degenerate exponential family can be made full by taking appropriate affine transformations, a procedure known as reduction to minimality (see, e.g., Theorem 1.9 in Brown, 1986or Lemma 8.1 in Bardoff-Nielsen, 1978. Assumption (A3) is needed to guarantee the existence of probability distributions supported over the boundary of P, which is an indispensable feature of the extended exponential family, described in the next section. It could be easily relaxed by allowing some faces to have zero ν measure. Finally, assumption (A4) is a standard. In particular, for our discussion of ERG models, Θ = R k .

Basics of Extended Exponential Families
Letting X = x be the observed sample from an unknown distribution in E P , the random set is the maximum likelihood estimate, or MLE, of θ. If θ = ∅ the MLE is said to be nonexistent. Existence of the MLE is determined by the geometry of P, as indicated by the following well-known, fundamental result (see, e.g., Theorem 5.5 in Brown, 1986or Proposition 4.2 Rinaldo, 2006a for different proofs).
Theorem 2.1. Under the current settings, the MLE θ exists and is unique if and only if x ∈ relint(P).
Furthermore, setting E θ (X) = R k zp θ (z)dν(z), because of the minimality of E P , the mean value parametrization map ∇ψ : int(Θ) → relint(P) given by is a homeomorphism, so that one can equivalently represent any distribution in E P using the natural parameter θ or the mean value parameter µ = E θ (X) ∈ relint(P). In particular, if the MLE exists, it is determined by the equation which translates into the moment equation E b θ (X) = x. For any proper face F , let ν F be the restriction of ν to F . Then, ν F determines a new exponential family of distributions E F , with densities with respect to ν F given by By assumption (A3), the convex support of this new family is F and the existence result of Theorem 2.1 carries over: the MLE exists if and only if the observed sample x belongs to relint(F ). However, since E F is supported on a lower-dimensional affine subspace of R k , it is no longer minimal, hence the MLE is not unique, and it consists instead of many solutions to (1); see Corollary 2.9 below for details. Nonetheless, via reduction to minimality (see, e.g., Brown, 1986, Theorem 1.9), it can be verified that, when θ is not empty, it consists exactly of those points satisfying the first order optimality conditions with the corresponding moment equations E F θ (X) = R k zp F θ (z)dν F (z) = x, ∀θ ∈ θ, still holding. In fact, lack of minimality bears not effect on the mean value parametrization: for every θ ∈ Θ F , there exists one point x ∈ ri(F ) such that and, similarly, for any x ∈ ri(F ), there exists a set θ F ⊂ Θ F , depending on x, such that (4) holds for all θ ∈ θ F . See equation (10) below for a characterization of θ F . The collection of distributions E = F E F as F ranges over all the faces of P, including P itself, is called the extended exponential family of distribution. With respect to the extended family E, for any observed sample X = x, the MLE, or extended MLE, is always well defined and is the set of solutions to (3), where F is the unique face containing x in its relative interior.

Extended Exponential Families and The Normal Fan of P
In this section we introduce the notion of normal fan of P and establish its relevance for the extended family E. See Lemma 7.2 in Appendix B for some basic properties of the normal cones and of the normal fan.
By assumption (A1) and (A2), there exists a m × k matrix A and a vector b ∈ R m such that where the system contains no implicit equalities. A proper face F of P is a subset of P defined by for some subsystem A F x ≤ b F of Ax ≤ b and, therefore, it is itself a polyhedron. The whole polyhedron P is regarded as the improper face of itself associated to the full system of inequalities, so that P is representable as the disjoint union of the relative interiors of all its faces. The dimension of a face F , dim(F ), is the dimension of the affine subspace it generates or, equivalently, the dimension of the null space of A F . Faces of dimension k − 1 are called facets of P and, if the system (5) has no redundant inequality, something which can always be assumed without loss of generality, the number m of rows of A match the number of facets. Equation (5) is known the H representation of P. Alternatively, P could be described using the V representation as the sum of a polytope and a polyhedral cone: where the sign + denotes Minkowski addition, and Q = convhull(Q) and C = cone(C), with Q and C two finite sets of vectors in R k . Throughout the paper, we will rely on the H representation (5), which we find more suited to our purposes, although our results could be established using (7). For every face F of P, let be the polyhedral cone consisting of all the linear functionals on P that are maximal over F , called the normal cone of F . Then, dim(N F ) = k − dim(F ), so that larger faces of P correspond to smaller normal cones. By Lemma 7.2 part 5., the normal cone of a proper face F can be equivalently defined as where a i denotes the transpose of the i-th row of the submatrix A F given in (6), where i = 1 . . . , m F . The collection of cones forms a polyhedral complex in R k (see, e.g. Sturmfels, 1995), called the normal fan of P. Notice that, since dim(P) = k, N P = {0} and N (P ) is pointed. Furthermore, where C * = {x ∈ R k : x, y ≤ 0, ∀y ∈ C} is the polar of C in the V representation (7) of P and denotes disjoint union. In particular, if C = {0}, i.e. if P is a full-dimensional polytope, the cones in N (P) partition R k : We mention that, more generally, if assumption (A2) is not in force, then N P is a linear subspace of R k of codimension k − dim(P).
Let lin(N F ) denote the subspace generated by N F , which is the linear subspace spanned by the vectors (a 1 , . . . , a m F ). The following lemma shows that, for every face F of the convex support, the parameter space of the extended family E F can be fully described using lin(N F ).
Lemma 2.2. The family E F is non-identifiable and Θ F is the quotient space of Θ modulo lin(N F ). Furthermore, for any ζ ∈ lin(N F ), rank (I F (θ + ζ)) = dim(F ), where I(·) and I F (·) denote the Fisher information matrices for E P and E F , respectively.
The previous result characterizes Θ F as the set of equivalence classes of points in Θ, where θ 1 and θ 2 are in the same class if and only if θ 1 − θ 2 ∈ lin(N F ), and the class containing θ ∈ Θ is the set which we call the congruence class of θ modulo lin(N F ). Notice that if Θ = R k , then Θ F is comprised of affine subspaces of dimension dim(N F ) = k − dim(F ) parallel to lin(N F ), each identifying a single distribution. In particular, when F = P, lin(N F ) = {0}, so that θ F is an atomic set and we recover the original, fully identifiable family E P .

Main result
We will utilize the normal fan N (P) to chracterize the following convergence statements: where µ n = E θn [X]. We take note that, because of the one-to-one correspondence between natural and mean value parameters for the families comprising E, the two statements imply each other. Equation (11) is of relevance as it explicitly provides various representation of the extended family E as the closure of the original family E P in both natural and mean value parameterization and also in terms of almost sure limits of the densities in E P . As a preliminary observation, we point out that (11) holds true only if the parameters θ n have diverging norms, so that p F θ F cannot belong to E P . Formally, Lemma 2.3. If (11) is verified, then θ n 2 → ∞.
In our main result, we establish establish sufficient conditions under which (11) holds or fails, based on the cones in the normal fan of P.
Theorem 2.4. Consider the settings describe above and assumptions (A1)-(A4). Let {θ n } ⊂ Θ be a sequence of natural parameters satisfying θ n = η + ρ n d n , where {ρ n } is a sequence of non-negative scalars tending to infinity, η ∈ θ F ∩ Θ and {d n } is a sequence of unit vectors.
1. If {d n } ⊂ R, with R a compact subset of ri(N F ), then Equation (11) which, in particular, implies that (11) is not verified.

1.
The assumption d n 2 = 1 for all n is imposed for mathematical convenience and does not entail any loss in generality.
2. The Theorem shows that (11) will hold or fail uniformly over compact subsets of ri(N F ), for all faces F of P.
Below, we will concern ourselves with sequences {θ n } of natural parameters of a certain simplified form, as described in below.
where θ ∈ Θ, d ∈ R k and {ρ n } is a sequence of non-negative numbers tending to infinity.
The restriction to (θ, d, {ρ n })-sequences is a strong enough condition to yield a full characterization of (11), as described in the next corollary, and yet sufficiently mild to unveil some of the fundamental features of the extended family E. Furthermore, it will allow us to recast some of our results in the language of convexity theory and gain some insights on the computational aspects of calculating the extended MLE.

If d ∈ N (P), then (12) is verified.
In essence, Corollary 2.6 characterize the extended family E as the compactification of the original family E P under both natural and mean value parametrization. For the natural parametrization, each density in E F is obtained as the point-wise limit of sequences of densities parametrized by sequences of points in Θ along any direction in ri(N F ) with norms diverging to infinity. In contrast, the corresponding sequence of mean value parameters converges gracefully to the corresponding point of finite norm on the boundary of P. This is a striking difference between natural and mean value parametrization, which is entirely captured by the norma fan of P. See Figures 4 and 3 below and related discussion for more details in the context of ERG models. See also the short movies available http://www.stat.cmu.edu/ ∼ arinaldo/ERG/ for a direct graphical illustration of these claims.
In the remaining of this Section, we will explore some of the consequences of Theorem 2.4 and, in particular, of Corollary 2.6, with the goal of illustrating some of the key properties of the extended family E.
We begin by observing that, as shown in Equation (20) in the proof of Theorem 2.4, if d ∈ N F , the sequence of distributions parametrized by the points θ n = θ + ρ n d corresponds to distributions in the original family E P whose mean value parameters µ n ∈ relint(P) are such that µ n 2 → ∞, with µ n bounded away from rb(P). It is clear that this can occur only if C = {0}, i.e. if the convex support is unbounded. In fact, when P is a polytope, we have N (P) = R k (see Equation 8), so that Corollary 2.6 further yields that each density in the family E F can be obtained as lim n p θn , where {θ n } is any (θ, {ρ n }, d)-sequence with θ ∈ Θ and d ∈ ri(N F ). Formally, Corollary 2.7. If P is a polytope, then, for any d ∈ R k , any (θ, {ρ n }, d)-sequence {θ n } and any face F , if and only if d ∈ ri(N F ).
In fact, our analysis of exponential random graph models of Section 3.4 is almost entirely an illustration of the previous Corollary.
Another implication of Corollary 2.6 is that, when the MLE does not exist, the directions of increase of the likelihood function for a given observed sample x ∈ relint(F ) are precisely the points in the associated normal cone N F . Formally, let X = x be the observed sufficient statistics and let x : Θ → R be the loglikelihood function, given by x (θ) = log p θ (x). Then, − x is a proper convex function, strictly convex if and only if x ∈ ri(P). This follows from Lemma 2.2 and the well-known convexity properties of the cumulant generating function ψ (see, e.g., Brown, 1986, Theorem 1.13). Then, following Rockafellar (1970, Chapter 8 for one, and thus for all, θ ∈ dom( x ) = Θ. The set of all directions of recession of − x is called the recession cone of x . It is clear that convex functions admitting directions of recession might not achieve their infimum at any point in their effective domain. On the account of the next result, the recession cone of − x is a cone of the normal fan of P, almost everywhere ν.
Corollary 2.8. For any observable sufficient statistics X = x ∈ P, the polyhedral cone N F is the recession cone of the negative log-likelihood function − x , where F is the unique, possibly improper, face of P such that x ∈ relint(F ).
In particular, when x ∈ relint(P), i.e. when the MLE exists, the corresponding recession cone is just the point {0} (since dim(P) = k), so that the negative log-likelihood function does not have any direction of recession and, therefore, its supremum is achieved at one parameter point θ ∈ R k with finite norm, namely the MLE. On the other hand, when the MLE is nonexistent, the likelihood function increases for any sequence of natural parameters with norm diverging to infinity along any direction d ∈ N F , where N F is the normal cone of the face of P containing the observed sufficient statistics in its relative interior.
We note that Corollary 2.8 could be stated in a more general form. Indeed, for any ξ ∈ P, letting ξ : Θ → R be given by ξ (θ) = θ, ξ − ψ(θ), it can be verified that the proof of Corollary 2.8 still holds with x replaced by ξ . Though theoretically relevant, this fact has little practical value.
During the preparation of the paper, we learned of similar results in Geyer (2008), which are based on the characterization of the convex support in term of the tangent cones and normal cones. While his analysis applies to more general classes of exponential families, our results are more refined, as we take full advantage of the polyhedral assumption and establish a more direct connections between the extended families and the cones in the normal fan.
By combining the results derived so far, we next show that, when x ∈ relint(F ), the extended MLE will be the affine subspace of dimension dim(N F ) given by Though not entirely a new result (see Brown, 1986, Chapter 6), our proof and the characterization of θ F in terms of N F is novel.
For completeness, we conclude this section by linking our discussion with alternative characterizations of the closure of the family E P existing in the literature, which could be easily obtained using Theorem 2.4 (see, in particular, Csiszár and Matúš, 2001, 2003, 2005, 2008.

Computational considerations
Based on our findings, we can make a few observations regarding the computational difficulties of finding the extended MLE, some of which are exemplified in the next result.
From the corollary and equation (9), we can infer that, when the MLE does not exist, maximizing the loglikelihood function using the Newton Rapson method, as well as virtually any other fastest ascent methods, may fail due to numerical instabilities. In fact, the Newton Rapson algorithm proceeds by finding a sequence {θ n } of natural parameters along which x increases most rapidly. At each step of the procedure, the next point in the sequence is determined by the direction of fastest ascent of x , given by the inverse of the Hessian, e.g. by the inverse of I(θ n ). However, for all n large enough, these matrices will be badly conditioned, since, at the optimum, the Fisher information matrix is not invertible (see equation 9). In addition, especially when the observed statistics x belong to the relative interior of a face of small dimension, these singularities can be dramatic, not to mention the fact that, unless x lies on a the relative interior of a facet, there is an infinite number of directions along which the likelihood function increases. It is apparent that these problems are even more accentuated in high-dimensional settings or whenever the data are sparse. From the statistical standpoint, equations (14) and (9) further imply that, when the MLE does not exist, the standard error may be quite large (in the limit, infinite), and that the number of degrees of freedom should be adjusted to reflect the non-estimability of some parameters. As a result, any hypothesis testing or model selection procedure that rely solely on these estimates should be regarded, at the very least, unreliable. Based on these considerations, it is clear that not only is the task of computing the extended MLE particularly daunting, but the statistical interpretation of these quantities is also rather delicate.
We refer the reader to Geyer (2008) and Rinaldo (2006b) for different algorithmic approaches on computing the extended MLE for certain types of exponential models with polyhedral support for which a V representation of P of the form (5) or (7) is either available or easily computable. We remark that, in order to determine the extended MLE, it is necessary not only to have an explicit representation of P but, in addition, to be able to have in closed form the log-partition functions ψ F , for each face F . A class of models for which both conditions are satisfied is the class of the log-linear models. If this information is not available, one may resort to MCMC techniques for computing the MLE or a pseudo-MLE, as for the class of models to be described in the next section. See Geyer and Thompson (1992), and Handcock (2003), Snijders (2002), Wasserman and Robins (2004), Handcock et al. (2006), Hunter et al. (2008) and references therein.
As a final comment, we point out that, while computing the extended MLE is very often a hard problem, deciding whether the MLE exists is typically more feasible, and can be accomplished using linear programming, provided an explicit representation, namely a H or a V representation, of P is available. See Appendix C for details and also Eriksson et al. (2006) for an application to hierarchical log-linear models.

Application to Exponential Random Graph Models
We now apply some of the results from the previous section to the class of exponential random graph models. The motivation for our choice is the attempt to explain certain features of ERG models that have been observed empirically and have been collectively labeled as degeneracy (see, e.g., Handcock, 2003). Our point of view is simply that there is nothing degenerate or unusual about these models, whose behavior can in fact be explained in a direct way using the properties of exponential families with polyhedral support as described in Section 2.3.
Our arguments rely on a thorough analysis of one ERG model, described below in Section 3.2, and on graphical renderings of Corollary 2.7, which we find particularly effective and elucidative of our results. We looked at a variety of other ERG models on 7,8 and 9 nodes, using different choices of the network statistics described below, and arrived to the same kind of conclusions we are about to present.
Finally, we would like to emphasize that, as the log-partition function is not available in closed form, an exact analysis of ERG models on larger graphs is almost impossible. This is due to the need to enumerate all possible graph with a given number of nodes in order to evaluate that function, a task whose computational computationally becomes prohibitive very rapidly as the number nodes grow; see Equation (15)

Introduction to ERG models
There is an extensive literature of ERG models and their use in social network analysis. A partial but representative list of references is: Holland and Leinhardt (1981), Frank and Strauss (1986), Wasserman and Pattison (1996), Wasserman and Robins (2004), Robins et al. (2007a,b) and references therein. Below we briefly describe the settings for ERG models, in order to make explicit the connections with the material in the previous sections.
Consider the set G g of all possible simple, i.e. unweighted, undirected and without loops, graphs on g nodes. Every such graph x can be described by a 0-1 symmetric g × g adjacency matrix, whose (i, j)-th entry is 1 if there exists an edge between the nodes i and j and 0 otherwise. Thus, x can be represented  Table 1: Some information about the complexity of some ERG models on small graphs.
as a g 2 -dimensional 0-1 vector. The cardinality of G g grows super-exponentially in the number of nodes n, namely so that network modeling entails constructing probability distributions over very large discrete spaces (see Table 1). Let T : G g → R k be a vector valued function of network statistics quantifying the key features of interest of a given observed graph. In this article we are mostly concerned with ERG models arising from network statistics that capture rather general and aggregate features of the network. Typical examples of such statistics are (see, e.g., Goodreau, 2007, for more details): 1. the number of edges: i<j x ij 2. the number of triangles: 4. the number of k-stars: i.e. the number of distinct edges that are incident to the same node, where D i (x) is the i-the degree statistic given above; 5. the alternating k-star statistic where λ is a positive parameter.
For all modeling purposes, these network statistics are effectively regarded as sufficient statistics and, by Koopman-Pitman-Darmois theorem, the resulting exponential family of distributions provides a convenient statistical model for G g . Formally, given a set of network statistics in the form of a k-valued function T (·) on G g , the ERG model P ≡ {Q θ , θ ∈ Θ ⊆ R k } is the exponential family of probability distributions over G g with natural sufficient statistics T (x) and base measure µ given by the counting measure on G g . Thus, for θ ∈ Θ, the density of Q θ with respect to µ is x ∈ G g } be the range of T (·) and ν the measure on T induced by µ, namely Then, the distribution of T (X) belongs to the exponential family of distributions on T with base measure ν, natural parameter space Θ and densities p θ (t) = exp{ t, θ − ψ(θ)}, θ ∈ Θ.
Furtheremore, because of the discreteness of the problem, Provided that the network statistics are affinely independent, as it is the case for the examples given above and as it can always be assumed through reduction to minimality, the convex support P = convhull(T ) is a k-dimensional ploytope. Recalling that ν is finite, it is easy to see that the assumptions (A1)-(A4) of Section 2.1 are verified, and the theory developed above applies. Despite its simplicity and interpretability, we need to emphasize that ERG modeling based on simple, low dimensional network statistics such as the ones described above can be rather coarse. In fact, those ERG models are invariant with respect to the relabeling of the nodes and even to changes in the graph topologies, depending on the network statistics themselves. As a result, they do not specify distributions over graphs per se, but rather distributions over large classes of graphs having the same network statistics. Consequently, as we repeatedly observed in our experiments and as elucidates in the example we are about to present, it may very well be the case that many graphs having very different topologies still belong to the same class and, therefore, are considered as equivalent. While this feature may be well suited for defining distributions over large thermodynamic ensambles in statistical physics, its use in other contexts in which the nodes are not interchangeable may be questionable. This is certainly not a common feature of all ERG models: for example, the p 1 model by Holland and Leinhardt (1981) and the Markov graphs by Frank and Strauss (1986) are based on much finer network statistics whose dimension, unlike the aggregate statistics reported above, increases with the size of the network. These more complex models represent explicitly distributions of individual networks rather than of classes on networks: both p 1 and Markov graph models are log-linear models over the probability of edges (see Fienberg and Wasserman, 1981). However, they also present difficulties. In fact, not only is the MLE not likely to exist if the observed network is even moderately sparse, but the asymptotics of these models as g grows remains unknown (see, e.g. Haberman, 1981, for some comments on p 1 models). While the theory developed in the previous sections apply to all ERG models, our analysis below is more directly relevant to models arising from simpler network statistics quantifying macroscopic properties of the network.

Our Running Example
We will be using throughout the example of a ERG model on G 9 with two-dimensional network statistic T (x) = (T 1 (x), T 2 (x)) ∈ N 2 , where T 1 (x) is the number of edges and T 2 (x) is the number of triangles. Note that this model is not hierarchical in the sense of Bishop et al. (1975) and Lee and Nelder (1996), since we do not include the network statistic for the number of 2-stars, which lie intermediate to edges and triangles. The lack of hierarchical model structure affects the interpretation of the exponential family parameters corresponding to T (x) but turns out not to be the cause of the degeneracies we illustrate. We have actually produced similar results for models which are fully hierarchical, but the results are easier to demonstrate in the context of this ERG model with a two-dimensional network statistic.
The number of distinct graphs for this G 9 example is 2 36 , while the number of two-dimensional distinct network statistics is only 9 2 9 3 = 444. The natural parameter space is the entire R 2 . The support of the distribution of T (X) is shown in Figure 1. The convex support for the induced family of distributions of network statistics is a polygon with 6 edges, whose boundary is depicted with the red solid line. Out of the possible 444 points, 29 actually lie on the boundary. The induced base measure ν for this family, i.e. the frequencies of each possible pair of network statistics, is indicated by the color shading of the circles. The maximal value of ν(t) is 1, 876, 664, 161, the median value is 2, 741, 130, while the first and third quartiles are 545, 265 and 79, 674, 084, respectively. Figure 2 shows a plot of the empirical quantile function for ν(t), t ∈ T , which indicates that few network configurations are much more frequent than others.  Figure 1: Support of the distribution of the network statistics for the ERG model on G 9 described in Section 3.2. The color shading indicates the squared root of the relative frequency of each point, namely ν(t) (darker colors correspond to higher-frequency values of t). The solid red line is the boundary of the convex support.

Degeneracy
The notion of degeneracy is central to ERG modeling, and has been investigated in various forms in the more recent literature. See Snijders (2002), Robins et al. (2007b), Robins et al. (2007a) and, in particular, Handcock (2003) and Hunter et al. (2008), just to mention a few. Degeneracy refers quite broadly to a variety of features, typically undesirable and surprising, of ERG models that have been observed empirically. In the literature, degeneracy (or near degeneracy) is used to describe any of the following, often interrelated, phenomena: 1. when a combination of ERG parameters θ implies that only a very small number of distinct graphs have substantial non-zero probabilities; in the most extreme cases, these configurations are the empty graph or the fully connected graph; 2. when, for a certain combination of ERG parameters θ, the density function p θ has multiple, clearly distinct, modes, and there are only very few network configurations that have non-zero probabilities, often radically different from each other; 3. when the MLE of θ is nonexistent or hard to obtain, or the MCMCMLE of θ fails to converge or appears to converge extremely slowly; 4. when the estimate of θ would make the observed network configuration very unlikely.
Each of the situations just described offers strong evidence of misspecifcation or, at the very least, of the inability of the model to describe in a realistic fashion the observed network. To our knowledge, Handcock (2003) is the only attempt to characterize degeneracy in a theoretical way, at least the kind of degeneracy yielding unstable maximum likelihood estimates, with emphasis on MCMCMLE methods.

Degeneracy via Entropy Functions
We based our analysis on a basic observation: a common feature of all the various instances of degenerate ERG models is that the corresponding distributions are highly concentrated on network configurations associated to a small number of network statistics. Therefore, in order to capture the overall degree of concentration of the family P, we turn to Shannon's entropy function, the rationale being that degenerate models have lower entropy. Shannon's entropy function S : Θ → R is defined as where the second summation involves a much smaller number of terms. Notice that, for every θ ∈ Θ, 0 ≤ S(θ) ≤ g 2 log 2, the lower and upper bounds corresponding to a degenerate distribution with point mass at one graph, and to the uniform distribution over G g (which is within the family if ν(t) is constant across T and θ = 0), respectively. Furthermore, as ψ is an analytic function of θ, for every θ ∈ Θ, S(θ) is a smooth function of θ.
Noting that lim x→0 x log x = 0 and using the fact that S(θ) is bounded, by the dominated convergence theorem Corollary 2.6 yields that, for every (θ, {ρ n }, d)-sequence {θ n } with d ∈ ri(N F ), for every face F of P.
On the other hand, because of the correspondence between natural and mean value parameters, the entropy function can be equivalently represented as a function over P. More precisely, we define V : P → R as follows: if µ ∈ relint(P), where µ = ∇ψ(θ), while, for µ F ∈ relint(F ), Equation (16) we obtain that lim where µ F = lim n µ n , with V (µ) a smooth function of µ. Thus, we conclude that S(·) and V (·) have homeomorphic graphs and, therefore, they convey the same information. Below, we use both entropy functions to illustrate the theory developed in Section 2.3 and to provide some characterizations of degeneracy.
We start with Figures 3 and 4. The latter displays the entropy function S(θ) for the ERG model on G 9 with network statistic taking values in N 2 , as described in Section 3.2, and for values of θ in the rectangle  Figures 4 and 3 offer two equivalent views of the exponential family P via the entropy functions S(θ) and V (µ). The mean value view in Figure 3 is straightforward to interpret: the entropy function is a well behaved, strictly concave function that changes smoothly as the mean parameter varies inside the relative interior of P. Distributions with mean value parameters lying well inside the cloud of points describing the support of the family have higher entropy, as their mass is distributed across a larger number of network configurations. In contrast, distributions with mean value parameters that are far removed from that cloud, including points very close to or on the boundary of P, have lower entropy. It is worth pointing out that, for this specific family, the points in the support are closer to the lower boundary of the polygon P, while the side of P determined by the convex hull of points corresponding to the empty and complete graph is significantly distant from the support. This phenomenon becomes more pronounced as g grows, so that this family will include many distributions, whose mean value parameters belong to a region far removed from the support, that would not provide a satisfactory or realistic explanation of any observed network, a feature that is often associated with degeneracy.
In striking contrast, the natural parameter view of Figure 4 does not lend itself to immediate interpretations. In fact, although S(θ) and V (µ) are smooth functions related via the homeomorphism (2), S(θ) displays drastic localized behaviors, including multiple local maxima. In particular, the function S(θ) exhibits sharp changes and high-peaked ridges shooting at infinity along which it remains roughly constant. Furthermore, small variations in the natural parameter values cause big changes in the values of the entropy function, thus making this ERG model rather unstable, in the sense that neighboring parameters specify very different distributions, or at least distributions with different entropies. These features may in fact fall under the general umbrella of degeneracy, as described in Section 3.3. Finally, we remark that the portion of the natural parameter space containing parameter points that produce more realistic distributions with higher entropy values is relatively small, a characteristic that emerged from the inspection of Figure 3 as well. In addition, the entropy function remains relatively high along some rays leaving the origin and shooting to infinity. We remark that Figure 4 matches quite closely analogous plots, not based on Shannon's entropy, for the same ERG model on graphs with 7 nodes by Handcock (2003), although the interpretation of the plots using normal cones, as described below, is missing. Figure 5 shows all the possible MLEs corresponding to the 415 points in the support of E P that are inside P. These points are all the estimates that can be obtained by maximum likelihood procedure, so that, although the family E P contains many other distributions, inference is only restricted to the 415 distributions identified by the MLEs, whose entropies are displayed in the Figure. Part of the seemingly strange behavior of S(θ) can however be explained using the results derived in the previous section. To that end, the convex support of Figure 1, can be expressed either as the convex hull of its vertices, namely P = convhull {(0, 0), (20, 0), (27, 27), (30, 44), (32, 56), (36, 84)} or, equivalently, using the Hp-representation, as the solution set of a system of linear inequalities, i.e.
The rows of A identify the outer normals to the 6 sides of the polygon P and generate the normal cones to the edges of P. The normal cone of a vertex of P is the conic hull of the outer normals to the edges incident to that vertex. For example, the normal cone of the vertex (0, 0) is The convex support P and its outer normals are shown in Figure 6. It is immediate to picture that the normal    , 10] of the natural parameter space with, superimposed, the normal fan of P, centered at the origin, which is the point of maximal entropy. As prescribed by Corollary 2.6, the outer normal to P are precisely the directions along which the closure of the original family E P is realized, by adding the families E F , as F ranges over the proper faces (in this case, edges and vertices) of P. These directions, starting at the origin, match perfectly the ridges of Figure 4, along which the entropy function seems to converge to some fixed value. This is because any sequence {θ n } along the outer normal of some edge F will eventually no longer identifies distributions from the original family E P , but just one distribution in E F supported on F . Consequently, the entropy function does not change because, for all n large enough, θ n specifies almost the same distribution. Figures 8, 9 and 10 offer other two pictorial representations of Corollary 2.6. These plots were obtained using the MATLAB GUI available at http://www.stat.cmu.edu/ ∼ arinaldo/ERG/ (see Section 9 below). The left side of each plot shows the entropy function for the family of Section 3.2 along with the outer normals of P leaving the original. The white circles represent the selected natural parameter. The plots on the right show the support of the family. The red stars indicate the mean parameter values corresponding to the natural parameters indicated by the white circles on the left side of the figure. Points with darker shaded colors correspond to network statistics receiving high probability under the selected natural parameter.
Part (a) of Figure 8 shows a distribution with high entropy, corresponding to a mean value parameter well inside the relative interior of P. In contrast, in parts (b), (c) and (d) the natural parameter is selected as d, with d a point in the relative interior of the 2-dimensional normal cone of the vertex of coordinates (0, 0), which identifies the empty graph. Consequently, the entropy is almost 0, as the associated distribution will put almost all its mass on that vertex of P. Notice that, even though the selected natural parameters from part (b), (c) and (d) are very different from each others, because they are far away from the set of parameters producing nondegenerate distributions and because they all to lye inside the normal cone of the vertex (0, 0), they parametrize essentially the same degenerate distribution on the empty graph. Figure 9 part (a) shows the same phenomenon, but for the different degenerate distribution putting virtually all its mass on the complete graph, which corresponds to the vertex (36, 84). As with Figure 8, notice that the natural parameter is a point inside the normal cone of that vertex and essentially any point in the upper triangular blue part of the entropy plot (which is, effectively, the relative interior of the associated normal cone) would parametrize this distribution. Part (b) and (c) show other degenerate distributions over the vertex of P identified by points inside the interiors of the corresponding normal cones. Figure 10 instead displays similar plots for a selection of natural parameters corresponding to directions lying on the normal cones, i.e. the outer normals, of some of the edges of P.

Discussion
The purpose of this article has been two-fold. First, for the class of discrete linear exponential families with polyhedral convex support, we have characterized the extended family using the normal fan to the convex support. While complete results about closures of general exponential families exist in the literature, our restriction to families with polyhedral support allowed us to obtain a more refined and explicitly geometrical description. In particular, our findings allowed us to gain a better understanding of the geometric and statistical properties of these families, as well as on the theoretical and algorithmic aspects of computing extended maximum likelihood estimates.
Our second goal was to study the behavior and statistical properties of ERG models, that have seen widespread use for the statistical analysis of data for social networks. To that end, we applied the theoretical results derived in the first part of the article to one ERG model on the set of graphs with 9 nodes. Despite our analysis being mostly graphical (due to the lack of a closed-form expression for the log-partition function), it captures a few interesting features of this model, some of which accounts for the seemingly strange behaviors that ERG models have been known to exhibit in practice, and generically termed degeneracy. Our investigation indicated that this type of behavior is, in fact, not unusual, and can be fully explained by the properties of linear discrete exponential families. Furthermore, based on similar experimentations with other ERG models, we believe our conclusions are not just specific to the model we present here but apply more widely to general ERG models.
The application presented here are particularly relevant to ERG models built around network statistics that describes macroscopic features of the networks and whose dimension does not grow with the number of nodes. However, our results apply to more complex models, such as the original p 1 model of Holland and Leinhardt (1981), which has node-specific parameters and whose likelihood is based on an assumption of dyadic independence. For these models with many parameters, degeneracy is typically due to nonexistence of the MLE, which is very likely to occur if the network is even mildly sparse.
Of course, much more needs to be done in order to fully understands the statistical subtleties, features and potential limitations of ERG models and in order to establish whether they are appropriate to model anything else than a large ensamble. Nonetheless, our contributions indicate that perhaps practitioners attribute to ERG models a degree of regularity that they may not possess.

Appendix A: Proofs
Proof of Lemma 2.2. Let θ 1 ∈ Θ F and ζ ∈ lin(N F ) and consider the point θ 2 = θ 1 + ζ. We first show that θ 2 ∈ Θ F and P θ1 = P θ2 . Because ζ ∈ lin(N F ), there exist some scalars c 1 . . . , c m F such that Therefore, almost everywhere ν F , Then, As both ψ F (θ 1 ) and C are finite, it follows that ψ F (θ 2 ) < ∞ and, therefore, θ 2 ∈ Θ F . It is now easy to conclude that P θ1 = P θ2 because, almost everywhere ν F , We now show that if P θ1 = P θ2 and θ 1 = θ 2 , then θ 1 − θ 2 ∈ lin(N F ). By Radon-Nykodin theorem, this occurs if and only if for some constant D, almost everywhere ν F . As ν F has support contained in F and F is defined by (6), the previous equality is equivalent to θ 1 − θ 2 ∈ lin(N F ), thus completing the proof of the Lemma. As for (9), since P is full-dimensional and, almost everywhere ν F , A F x = b F , we have, for any θ ∈ Θ F , 0 = Var θ ( a, X ) = a I F (θ)a if and only if a = lin(N F ). This implies that rank(I F (θ)) = dim lin(N F ) ⊥ = dim(F ).
In the proof we will make use repeatedly of the following decomposition. For any point x 0 ∈ P and proper face F of P, we will write where Notice that, for all n, if x 0 ∈ F , then A >,n (x 0 , F ) = 0, since ν{x : A F (x − x 0 ) > 0} = 0. We will also use the following fact, which stems directly from Lemma 2.2: 1. Party 1.
We will begin by showing sufficiency. First, we consider the case of a generic point x 0 ∈ F . If, d n ∈ ri(N F ), then, by part 1. of Lemma 7.2, d n , x − x 0 = 0 for all x ∈ F , which implies that for al n. On the other hand, for any x ∈ F , since R is a compact subset of ri(N F ) ∩ S k−1 and {d n } ∈ R, we have sup Then, by the dominated convergence theorem, we obtain A <,n (x 0 , F ) 0.

Part 2.
Suppose that, {d n } ⊂ R, where R is a compact subset of N c F . Then, there exists a subsequence {d n k } ⊂ {d n } such that, for all k large enough, d n k belongs to a compact set R * such that either R * ⊂ ri(N F ), for some F = F , or R * ⊂ (N (P)) c . In the latter case, by part 3., proven below, the numbers µ n k − µ F 2 grow unbounded and, therefore, (11) is violated. In the former case, by part 1. of the proof, (11) is verified for F , so it cannot be simultaneously verified for F as well. Indeed, p θn cannot converge pointwise to both p F θ F and p F θ F , which identify different probability distributions with different supports.
We will show that, if {d n } ⊂ R for some compact subset of (N (P)) c , then, This implies that µ n 2 → ∞. Let x 0 ∈ P. As P is full-dimensional and d n ∈ N (P), by Lemma 7.2, part 3., the set S n = {x ∈ P : d n , x − x 0 > 0} is non-empty, for each n. Furthermore, since, by assumption, the set S = lim inf n S n is non-empty as well. We now claim that ν(S) > 0. In fact, arguing by contradiction, suppose that ν(S) = 0. Then, there exists a subsequence {d n k } ⊂ {d n } such that no point x ∈ supp(ν) can satisfy lim k d n k , x − x 0 > 0. However, since, by assumption (A3), P = convhull(supp(ν)), this implies that P ⊆ {y : lim k d n k , y − x 0 ≤ 0}, which in turn implies that lim k d n k ∈ N (P), violating the condition that {d n } is bounded away from N (P). Thus, ν(S) > 0, from which we can conclude that lim inf n ν(S n ) ≥ ν(S) > 0. Then, by the monotone convergence theorem, Proof of Corollary 2.6. Any direction d ∈ R k is either in N (P), in which case, it must belong to ri(N F ) for one face F of P or in (N (P)) c . The results then follow directly from Theorem 2.4.
Proof of Corollary 2.8. If x ∈ ri(P), then the MLE exists, is unique and is given by the vector θ ∈ Θ such that ∇ψ( θ) = x. Equivalently, since in this case N P = {0}, invoking Corollary 2.6, part 1., − x has no direction of recession. Thus consider the case of x ∈ rb(P) and let F be the unique face such that x ∈ ri(F ). If d ∈ ri(N F ), then by Corollary 2.6 part 1., so (13) holds. Suppose now that d ∈ rb(N F ). Let F x = {F : x ∈ rb(F )}, with F being a face of P. By Lemma 7.2, part 4., so, if d ∈ rb(N F ), then d ∈ ri(N F ), for some F ∈ F x . By Corollary 2.6, part 1., almost everywhere ν F , Since x ∈ F , we have ν F (x) > 0, which implies, p F θ F (x) > 0 and, consequently, (21). Thus d is also a direction of recession and we have shown that any point in N F is a direction of recession for − x , It remains to be shown that Equation (13) is not verified if d ∈ N F . If d ∈ N (P) Corollary 2.6 part 2. yields lim ρ→∞ p θ+ρd (x) = 0, hence − x (θ) → ∞, so d is not a direction of recession. If instead d ∈ N (P) ∩ N c F , then it must be the case that d ∈ ri(N * F ), for some face F * such that F ∩ F * = ∅, otherwise N F * ⊂ rb(N F ) (see, e.g., Lemma 7.2, part 4.). Thus, by Corollary 2.6, part 1., As a result, (21) does not hold, so that d does not satisfy (13) and is not a direction of recession.
Proof of Corollary 2.9. The only interesting case is when x ∈ ri(F ), for some proper face F , otherwise N (P) = {0}, and − x has no directions of recession, as the MLE exists. For every θ ∈ Θ, let {θ n } be a (θ, {ρ n }, d)-sequence. By Corollary 2.8, we need to consider only the case d ∈ N F . If d ∈ ri(N F ), by Lemma 7.1, p θn (x) p F θ F (x). Now suppose that d ∈ rb(N F ). Then, d ∈ ri(N F * ) for some face F * such that F ⊂ F * . Another application of Lemma 7.1, yields p θn (x) p F * θ F * (x). However, , since only the points θ ∈ θ F satisfy the first order optimality conditions (3). The result follows.
Proof of Corollary 2.10. Part i) and ii) follows from Lemma 2.2 and results of Matúš (2003, 2005). Part iii) is a direct consequence of part i).

Appendix B
The following lemma in needed in the proof of Corollary 2.9 Lemma 7.1. Under the conditions of Corollary 2.6, p θn p F θ F , a.e. ν F , if and only if d ∈ relint(N F ). Proof. The claim follows from Equation (18) in the proof of Theorem 2.4, which holds for all x ∈ F , thus almost everywhere ν F . Below, we collect some basic facts about the normal fan and normal cones needed in our proofs. With some slight abuse of notation, we say that a vector d is normal to the hyperplane H if d, x − y = 0 for all x, y ∈ H.
Lemma 7.2. Let P be full-dimensional and let F be a face of P.
1. For any x 0 ∈ F , a F , x − x 0 = 0 for all x ∈ F and a F , x − x 0 < 0 for all x ∈ F if and only if a F ∈ relint(N F ).

2.
For any x 0 ∈ F , a F , x − x 0 > 0 for all x ∈ F and a F , x − x 0 ≤ 0 for all x ∈ F if and only if a F ∈ relint(N F ).
5. N F = cone (a 1 , . . . , a m F ), where a i denotes the transpose of the i-th row of the submatrix A F given in (6), i = 1 . . . , m F .
Proof. Recall that, since P is full-dimensional, there is no vector d = 0 such that d, x − x 0 = 0 for all pairs x, x 0 ∈ P.
1. First we show sufficiency. If a F ∈ relint(N F ), then a F is a conic combination of all the rows of A F with positive coefficients. Therefore, a F , x − x 0 = 0 for all x ∈ F , by the definition of F , and a F , x − x 0 < 0 for all x ∈ F , since, in this case, a, x − x 0 < 0 for some row a of A F . As for necessity, if a F ∈ N F , then a F , x − x 0 < 0 for all x ∈ ri(P). However, if a F ∈ rb(N F ), then a F , x − x 0 = 0 for all x ∈ F , where F is the face of P such that a F ∈ ri(N F ). But then, since F ⊂ F , there exists a x ∈ F for which a F , x − x 0 = 0, which would produce a contradiction. Thus a F ∈ rb(N F ).
2. The proof is analogous to the previous case and is omitted.
3. Since d is not normal to any supporting hyperplane, the hyperplane H = {x : d, x − x 0 = 0} intersects P is in its relative interior, and P must have non-empty intersections with both the halfspaces {x ∈ R k : d, x − x 0 > 0} and {x ∈ R k : d, x − x 0 < 0} cut out by H.
4. The claim follows directly from the definition of N F and the fact that N (P) is a polyhedral complex (see, e.g., Sturmfels, 1995), thus the relative boundary of N F is the disjoint union of the relative interiors of all its faces.
5. Let c ∈ cone (a 1 , . . . , a m F ), so that c = A F λ, where λ ∈ R k has nonnegative coordinates. Then, for all x ∈ F and y ∈ P ∩ F c , c, x = λ, A F x = λ, b F ≥ λ, A F y = A F λ, y = c, y since A F x = b F and A F y < b F . Thus, c ∈ N F and we have shown that cone (a 1 , . . . , a m F ) ⊆ N F . Conversely, assume that c is a nonzero vector in N F but c ∈ cone (a 1 , . . . , a m F ). Then, c is not normal to any supporting hyperplane of F , which implies that there exists a x ∈ F and y ∈ P ∩ F c such that c, x − y < 0, producing a contradiction. Thus, it must be the case that c ∈ cone (a 1 , . . . , a m F ) as well, yielding N F ⊆ cone (a 1 , . . . , a m F ).
8 Appendix C: Checking for the existence of the MLE via Linear Programming.
Deciding whether the MLE exists, that is, whether the vector of observed sufficient statistics x is such that x ∈ ri(P) is particularly simple if one has access to a H representation of P as in (5), as indicated in the next result, of immediate verification.
Lemma 8.1. The MLE exists if and only if the system Ax ≤ b is satisfied with strict inequalities.
Unfortunately, this type of representation is typically not available or prohibitively hard to compute, even when k is small, since P may have a number of faces that grow super-exponentially in k (see, for example, Ziegler, 2001).
If instead only a V representation (7) is available or computable, the existence of the MLE can be established using linear programming, as outlined below. Let B be a matrix whose columns contain the vertices and extreme rays of P, namely the vectors in Q and C from Equation (7). Then x ∈ ri(P) if and only if x can be obtained as a linear combinations of the vectors in Q and C with strictly positive coefficients. where z i denotes the i-th coordinate of z and s is a scalar. If (s * , z * ) is the optimum, then the MLE exists if and only if s * > 0. An alternative linear program, which may be computationally preferable, can be formulated based on Theorem 8.3, whose proof can be found in Schrijver (1998), as follows: max 1, y s.t.
If y * is the optimum, the MLE does not exist if and only if 1, y * > 0.

Theorem 8.3 (Gordan's Theorem of Alternatives).
Given a matrix B, the following are alternatives: 1. Bx > 0 has a solution x.