Dynamic regimes of random fuzzy logic networks

Random multistate networks, generalizations of the Boolean Kauffman networks, are generic models for complex systems of interacting agents. Depending on their mean connectivity, these networks exhibit ordered as well as chaotic behavior with a critical boundary separating both regimes. Typically, the nodes of these networks are assigned single discrete states. Here, we describe nodes by fuzzy numbers, i.e. vectors of degree-of-membership (DOM) functions specifying the degree to which the nodes are in each of their discrete states. This allows our models to deal with imprecision and uncertainties. Compatible update rules are constructed by expressing the update rules of the multistate network in terms of Boolean operators and generalizing them to fuzzy logic (FL) operators. The standard choice for these generalizations is the Gödel FL, where AND and OR are replaced by the minimum and maximum of two DOMs, respectively. In mean-field approximations we are able to analytically describe the percolation and asymptotic distribution of DOMs in random Gödel FL networks. This allows us to characterize the different dynamic regimes of random multistate networks in terms of FL. In a low-dimensional example, we provide explicit computations and validate our mean-field results by showing that they agree well with network simulations.


Introduction
Multistate models (MMs) are a class of discrete dynamical systems. The model's variables take values in discrete, finite sets and develop in discrete time steps. At each time point, the value of a variable is determined by an update rule that deterministically depends upon the values of some of the other variables at the previous time point. MMs are frequently used to model molecular networks in theoretical biology [1]- [4]. In these applications, the discrete states of a variable are interpreted as, e.g., 'low'-'medium'-'high' or 'active'-'inactive', and the update rules are typically specified by propositional formulae (Boolean expressions), which often allow for an interpretation in terms of interacting regulatory mechanisms. For this reason, MMs are often referred to as logical models. (Here, we call them crisp logical models when confusion with the fuzzy logic (FL) models introduced below needs to be prevented.) Despite being a crude simplification of biological reality, logical modeling has become a popular tool in theoretical biology and has been substantiated from a biophysical [5] as well as a philosophical point of view [6].
In 1969, Kauffman proposed random MMs-the so-called Kauffman networks (KNs)as generic models for large-scale gene regulatory networks [7]. Kauffman himself provided computational results in the special case of Boolean networks, where each variable assumes values either 'off' or 'on'. He showed that these networks exhibit surprisingly ordered structures and are able to give insights into biological phenomena such as cell replication or lineage differentiation. Interest in KNs was rekindled as their close relation to classical models from statistical mechanics was realized. In a number of studies [8]- [10], the self-organizing capacity of KNs was analyzed. It was shown that depending on their connectivity, KNs exhibit ordered as well as chaotic behavior with a critical boundary separating both regimes. The ordered 3 regime is characterized by small stable attractors, whereas in the chaotic regime long-periodic orbits frequently occur. These properties render both regimes unfavorable for the evolution of living organisms. Consequently, Kauffman promoted the idea of 'living at the edge of chaos' [11]. Interestingly, the critical connectivity of KNs is 2, which agrees well with the average connectivities of gene regulatory networks, e.g. in Escherichia coli, Saccharomyces cerevisiae and Bacillus subtilis [12]. These results about Boolean KNs can be extended straightforwardly to general random MMs [13,14].
A major problem with MMs of molecular networks is the imprecision and subjectivity of categories such as 'high'-'low', 'active'-'inactive', etc. For this reason, FLs [15] are becoming an increasingly popular extension of logical models in theoretical biology. In an FL model, a variable is described by a fuzzy number, i.e. a vector of degree-of-membership (DOM) functions specifying the degree to which the variable is in each of its discrete states. Biological applications of FLs range from gene regulatory networks [16] over signal transduction and metabolic pathways [17]- [20] to ecological systems [21].
FL models are natural and biologically relevant generalizations of crisp logical models. However, they have not yet been combined with random KNs. This is what we address in this paper. We begin by recalling the phase transition criterion of multistate Kauffman networks (MKN) in section 2. In section 3, we then study FL versions of these networks. We briefly review the general concept of FLs and explain that FL models are, indeed, generalizations of crisp logical models. Subsequently, in section 4, we restrict ourselves to the Gödel FL, which can be distinguished from all other FLs as it can be derived from first principles. The dynamics of KNs with Gödel FL are described, and the observations are explained within a mean-field theory. Thus, a characterization of the three dynamic regimes of MKNs in terms of FL is obtained. We visualize our results by explicit computations in a low-dimensional example and further corroborate our findings by simulations of FL-KNs.

Phase transitions in random multistate networks
In this section, we review MKNs and briefly recall some results about critical phenomena; for a detailed account, see [14]. In the following, G = (V, E) will always denote a directed graph of order N with nodes V = {1, 2, . . . , N } and edges (i → j) ∈ E ⊂ V × V . The ancestors (inputs) of a node i are denoted by i 1 < i 2 < · · · < i K i , where K i is the node's in-degree.
An MM is a triple (G, S, F) , which consists of a directed graph G, of numbers of states defining the range i := {0, 1, . . . , S i − 1} of node i = 1, 2, . . . , N , and a vector of discrete functions Each MM (G, S, F) gives rise to a time-discrete dynamical system: with each node i we associate a time-dependent discrete variable where the ith component is given by In this paper, we only consider the above synchronous updating, for different update policies, see e.g. [22,23]. 4 We now define MKNs generalizing the definition of KNs as given e.g. in [24]. An MKN is an MM where (K1) the K i are chosen randomly from a probability distribution P in (K ), K = 1, 2, . . . , K max , K max N ; (K2) the K i inputs i 1 , i 2 , . . . , i K i of i are chosen randomly with uniform probability from among the network's nodes 1, 2, . . . , N ; (K3) the numbers of states S i are chosen randomly from a probability distribution P nos (S), S = 2, 3, . . . , S max ; and (K4) the values of f i are chosen randomly from a probability distribution P S i (s), s ∈ i .
Note that in (K4) the distribution P S i does not depend on the node i but only on its number of states S i . In particular, in the Boolean case the update rules evaluate to 0 with a certain probability w and to 1 with probability 1 − w.
From (K4) it follows that the probability p S i for the function f i to yield two different values for two different arguments depends only on P S i and is given by We define the first momentp = S max S=2 P nos (S) p S as well as the mean connectivityK = In [14] it is shown that MKNs exhibit the following phase transition: pK    < 1 ordered regime, = 1 critical boundary, > 1 chaotic regime. (2)

Fuzzy logic Kauffman networks
In this section, we consider again MKNs as introduced in section 2. However, now we use FLs to evaluate the network. For the sake of clarity, we denote the variables of the FL model byx i (t), i = 1, 2, . . . , N . The variablex i (t) is no longer assigned exactly one value from i . Rather its state at time t is given by a vector . . 1] indicating the degree to whichx i (t) has states s ∈ i . Note that we do not require any normalization ofx i (t).
In crisp logic, update rule (1) can be written as 5 s ∈ i . The disjunction (OR-gate) on the right-hand side runs over all arguments of f i with output value s. For each argument, the inner conjunction (AND-gate) is true if and only if the argument agrees with the input values at time t. Now that variables are assigned fuzzy numbers, the Boolean equations x i (t + 1) = s and x i k (t) = ξ k from this update rule no longer assume crisp true-false values, but their degrees of truth are given by the DOMsx s i (t + 1) ∈ [0, 1] and x ξ k i k (t) ∈ [0, 1], respectively. We therefore need to generalize the Boolean operators AND, OR and NOT to the unit interval.

A short primer on fuzzy logic
Let us now briefly introduce these generalizations of the Boolean operators; for details, see [25]. The generalized NOT operator is a strong negation, i.e. a function ¬ : For the generalization of AND and OR, one typically uses the concepts of t-norms and t-conorms. AND is replaced by a t-norm, i.e. by a function Analogously, an OR is replaced by a t-conorm, i.e. by a function : Clearly, for non-fuzzy DOMs, i.e. DOMs either 0 or 1, the above definitions reduce to their standard versions. We remark that, for fixed ¬, t-norms and t-conorms are dual concepts under a generalized DeMorgan's law: with every t-norm we can associate a dual t-conorm ⊥ via , ¬(y))) . (T1) allows us to inductively define the t-norm and t-conorm of more than two arguments.
It is impossible for a dual pair of t-norm and t-conorm to preserve all laws from Boolean algebra. In fact, there is a trade-off between the classical Aristotelian laws of thought-the law of the excluded middle and the law of non-contradiction-on the one hand, and the distributive law, the law of absorption and idempotency on the other hand. It can be shown that any law of the latter group is respected only by the Gödel t-norm and t-conorm (x, y) = min(x, y) and ⊥ (x, y) = max(x, y), which are dual under the negation ¬(x) = 1 − x, cf appendix A. These choices of t-norm and t-conorm, in turn, violate both the law of the excluded middle and the law of non-contradiction. These properties allow us to deduce the Gödel pair from first principles [26] and make it the standard choice.

General fuzzy logic Kauffman networks
We generalize update rule (3) to the FL update rulẽ s ∈ i . These update rules define a functionF mapping a stateX (t) Let us conclude this section by showing that (4) is, indeed, a generalization of update rule (3). To this end, consider non-fuzzy states, i.e. statesX (t) where for all i there exists s i such There is a canonical bijection φ between the states X (t) and the non-fuzzy statesX (t). Clearly, φ is compatible with the mappings F andF in the sense that for a non-fuzzy stateX (t) alsoX (t + 1) =F(X (t)) is non-fuzzy and the diagram

Kauffman networks with Gödel fuzzy logic
In this section, we treat the special case of the Gödel t-norm and t-conorm. We concretize update rule (4) accordingly and obtaiñ Let us see how this update rule works in a simple example.
Then, following (5), we compute the state of node i at time t + 1 as

Bounds for degrees-of-membership
First we turn our attention to the decreasing range of the DOMs. More precisely, we show that the DOMs become bounded by the smallest maximal DOM per node at time t = 0, which we denote by m. Formally, If N is large and the initial conditions are sampled uniformly, we may expect m to be small.
Clearly, for random initial DOMs, exactly one node is bounded at t = 0; we denote it byī. Now let us consider an FL-KN with general P in , P nos and P S in the thermodynamic limit N → ∞. To understand the long-term behavior of the quantity we set up an iteration b(t + 1) = H (b(t)) in a mean-field approximation. To this end, consider some node i at time t + 1. We show that node i will be bounded at t + 1 if and only if any of its inputs is bounded at time t. First, suppose that i has an input, w.l.o.g. i 1 , which is bounded at t. Then m is an upper bound on the DOMsx s i (t + 1). Moreover, for (ξ 1 , ξ 2 , . . . , Conversely, assume that none of i's inputs are bounded at t and let (ξ 1 , ξ 2 , . . . , ξ K i ) again be defined as above. Now this implies min K i k=1x ξ k i k (t) > m and, hence,x f i ( ξ 1 ,ξ 2 ,...,ξ K i ) i (t + 1) > m. Because, in the thermodynamic limit, the inputs of i can be assumed to be independent [27], we may write Note that this is the mean probability that at least one input of a node is bounded at t. In the two fixed points b = 1 and b = 0 of this iteration, we have H (b ) = P in (1) and H (b ) =K , respectively. FromK > 1 ⇐⇒ P in (1) < 1 it follows that, ifK > 1, the fixed point b = 1 is stable and the fixed point b = 0 is unstable. Hence, forK > 1, the network will ultimately reach a state where all nodes are bounded. Visually speaking, ifK > 1, property (8) percolates through the network. Emanating from nodeī, at each time step all nodes satisfying this property bequeath it to their descendants.

Distributions of degrees-of-membership
Let us now explain the observed differences in the steady-state distributions between the three dynamic regimes. For this, we restrict ourselves to the special case P nos (S) = δ S,S and study the behavior forK > 1 in more detail, so letK > 1. We may then assume that each DOMx s i (t) m and that at least one DOM per node is equal to m after some time t 0 . We study the distribution Z (t) = (Z z (t))S z=0 , Z z (t) := 1 N · #{i|x i (t) has z DOMs equal to m}, z = 0, . . . ,S, of DOMs m per node in the network. In this section, we will consider only times t t 0 . Hence, we have Z 0 (t) = 0 and may thus omit this component.
Our goal now is to set up an iteration Z (t + 1) = E(Z (t)) for this distribution in a meanfield approximation. To this end, let us consider some node i at time t + 1. We describe its inputs by a vector (κ 1 , κ 2 , . . . , κS) indicating the number of inputs with 1, 2, . . . ,S DOMs m at time t. Clearly, κ 1 + κ 2 + · · · + κS = K i . In the thermodynamic limit, we may assume the inputs to be independent [27] and the probability for the configuration (κ ζ )S ζ =1 is given by The number of tuples (ξ 1 , ξ 2 , . . . , (t + 1) belonging to these tuples are the DOMs ofx i (t + 1) which are equal to m.

10
To compute their number, we let (ζ, z) denote the probability that ζ fields, which are randomly filled with numbers 0, 1, . . . ,S − 1 according to PS, contain exactly z different elements. For notational convenience, we let ( S ζ =1 ζ κ ζ , z) = ((κ ζ ), z). The (ζ, z) are difficult to compute analytically and one has to resort to exhaustive enumeration. We can, however, make use of the following relations. (2, 1) · · · S , 1 with equalities if and only if PS is a degenerate delta-distribution.
We are now able to set up an iteration for the probabilities Z (t) in a mean-field approximation: where the zth component E z (Z (t)) of the right-hand side is given by It is intuitive and can also easily be shown that Z = (1, 0, . . . , 0) t is a fixed point of this iteration. We now investigate its stability. The distributions Z (t) live on the affine hyperplane of RS defined by S z=1 Z z (t) = 1. In order to take derivatives, we think of it as a manifold and choose as a global chart the projection on the firstS − 1 coordinates. We remark that the choice of charts, in general, does affect the Jacobian, but not its eigenvalues, as a change of charts merely means a change of basis in the tangent space. Hence, this choice is not crucial for stability analyses. The Jacobian of E at Z can be written in global coordinates as  (8) does not percolate through the entire network. K > 1 andpK < 1 : Eventually, all nodes satisfy (8). The state where only one DOM per node is equal to m is stable. K > 1 andpK = 1 : Critical boundary. K > 1 andpK > 1 : Eventually, all nodes satisfy (8). The state where only one DOM per node is equal to m is unstable.
Iteration (10) is exact in the thermodynamic limit from time t 0 on. Recall that this is the time after which each DOMx s i (t) m and at least one DOM per node is equal to m. The crucial point is whether Z (t 0 ) = Z or not. If yes, it follows that Z (t) = Z for all t > t 0 as Z is a fixed point. In this case its stability is irrelevant. If not, the differences between the dynamic regimes come into play. In the frozen regime, the fixed point Z is attractive and we may expect Z (t) = Z at some point in time. In the chaotic regime, Z is repellent.
Let us now see which case occurs if we draw the initial DOMs randomly from a uniform probability distribution on [0, 1]. To this end, consider the update of a node i with input nodē i, w.l.o.g.ī = i 1 . As the initial DOMs are uniformly distributed, each input i k , k = 2, 3, . . . , K i , has expectedS(1 − m) DOMs bigger than m at t = 0. Moreover, for sufficiently large N , we have m 1, and consequentlyS(1 − m) > 1. Hence, we expect at least (S(1 − m) Consequently, also the expected number of DOMsx s i (1) that are set to m will be greater than 1. Inductively, this shows that Z (t 0 ) = Z . We have thus found an explanation for the observed differences between the dynamic regimes in section 4.1. In the following, these differences are analyzed in more detail.

Example and network simulations
We finish by detailing the behavior of the distributions Z (t) in a specific example, more precisely in the caseS = 3. The distribution P 3 will be assumed to be non-degenerate, i.e. P 3 (s) = 1 for s = 0, 1, 2. In particular, we address the following questions: (i) Are the global dynamics in the frozen regime governed by the stable fixed point Z ? (ii) What is the asymptotic behavior in the chaotic regime?

Conclusions
In this paper, we studied FL-KNs. These are random MMs, where nodes are described by fuzzy numbers, i.e. vectors of DOM functions specifying the degree to which the nodes are in each of their discrete states. The update rules are constructed by replacing the Boolean operators AND and OR by continuous generalizations, the so-called t-norms and t-conorms.
We first observed that, for any choice of t-norm and t-conorm, an FL model is a true generalization of the underlying crisp model in the sense that for non-fuzzy initial conditions the behavior of the latter is reproduced. Subsequently, Gödel FL-KNs were studied in more detail. We found that, unless the mean connectivity equals one, the DOM m from (7) (the smallest of the maximal DOMs per node) percolates through the network, i.e. m becomes a global upper bound for all DOMs in the network.
One can interpret m as the maximal amount of uncertainty present in the network. Nodē i (the node whose maximal DOM is equal to m at t = 0) is the node that we have the greatest 14 difficult assigning a discrete state to. ForK > 1 this maximal uncertainty m becomes a global bound on the certainties with which we assign to nodes their discrete states.
How many DOMs per node are equal to m depends on the dynamic regime of the MKN. More precisely, we could analytically show in a mean-field approximation that the state where only one DOM per node is equal to m is stable in frozen networks and unstable in chaotic networks. In a low-dimensional example, we explicitly computed the distribution of DOMs m and could show that the results from our mean-field approximations well agree with simulations of FL-KNs.
The differences in the behavior of the DOMs in the different regimes have significant dynamical consequences. Let us first consider a frozen MKN and assume that at time t only one of the DOMsx i (t) of node i is equal to m for all i = 1, 2, . . . , N . We define s i ∈ i such thatx s i i (t) = m and initialize the crisp MKN by setting x i (t) equal to s i , i = 1, 2, . . . , N . After evolving the crisp MKN for one time step, x i (t + 1) indicates the state of node i whose DOM is equal to m at time t + 1 in the FL-KN, i.e.x x i (t+1) i (t + 1) = m. Hence, the dynamics of the FL-KN are strongly related to the dynamics of the crisp MKN. This is no longer the case for chaotic MKNs. Here, the FL-KN ultimately reaches a state where a large fraction of DOMs are equal to m. Consequently, the dynamics of the FL-KN contains no information about the dynamics of the crisp MKN.
Future work could address the behavior of other FLs, such as the probabilistic or Łukasiewicz FL. From a more general point of view, FL models are an intermediate between discrete and continuous models. They are fully specified by a discrete model but allow (discrete-)time evolutions of a continuous state space. It would be interesting to compare this to other continuous extensions of logical models as proposed e.g. in [28,29].

Appendix D
To determine the relation between the nominator and the denominator in (12) and (13) Recall that (2, 2) =p. To obtain the claim, consider that according to lemma 4.1 the denominators in (12) and (13) are positive and negative, respectively.