A detailed balanced reaction network is sufficient but not necessary for its Markov chain to be detailed balanced

Certain chemical reaction networks (CRNs) when modeled as a deterministic dynamical system taken with mass-action kinetics have the property of reaction network detailed balance (RNDB) which is achieved by imposing network-related constraints on the reaction rate constants. Markov chains (whether arising as models of CRNs or otherwise) have their own notion of detailed balance, imposed by the network structure of the graph of the transition matrix of the Markov chain. When considering Markov chains arising from chemical reaction networks with mass-action kinetics, we will refer to this property as Markov chain detailed balance (MCDB). Finally, we refer to the stochastic analog of RNDB as Whittle stochastic detailed balance (WSDB). It is known that RNDB and WSDB are equivalent. We prove that WSDB and MCDB are also intimately related but are not equivalent. While RNDB implies MCDB, the converse is not true. The conditions on rate constants that result in networks with MCDB but without RNDB are stringent, and thus examples of this phenomenon are rare, a notable exception is a network whose Markov chain is a birth and death process. We give a new algorithm to find conditions on the rate constants that are required for MCDB.


Introduction
The concept of detailed balance (see [2,3,7,13]) arose early in the history of chemical reaction network theory (see for instance [9,11,13]). As an example, consider the reversible chemical reaction network (CRN) in figure 1. A CRN is prescribed a kinetic scheme (such as massaction kinetics) in order to specify the dynamics of variables corresponding to the species concentrations, represented through a system of nonlinear ODEs . For a reversible reaction network (a reaction network is reversible if every reaction is reversible), the kinetic scheme assigns a unique function to each reaction pair. For instance, mass-action kinetics assigns the 2A respectively. An equilibrium of the dynamical system is said to be detailed balanced if at the steady state concentration, the binomial corresponding to each reaction pair vanishes. Thus, if c * A and c * B represent the equilibrium concentrations of the species A and B respectively, then one of the conditions required for the equilibrium to be detailed balanced is that k 1 c * 2 A = k −1 c * A c * B . The full set of conditions arising from the five reaction pairs in the network in figure 1 is as follows: Assuming that a detailed balanced equilibrium exists for this network, the set of conditions (1) can be simultaneously satisfied only if the reaction rate constants are appropriately constrained. Some algebra then leads us to identify the constraints as follows Note carefully that the second constraint (3) does not require the precise knowledge of the species constituents of the complexes, but only the fact that the triplet of complexes {C 4 := A + C, C 5 := D + E, C 6 := B} forms a cycle as is evident in the depiction on the right of figure 1. On the other hand, (2) cannot be obtained without knowing that C 1 = 2A, C 2 = A + B and C 3 = 2B. The set of constraints on the rate constants, such as the one in (2) and (3), is not only necessary but also sufficient in the sense that if the constraints are satisfied for a reversible CRN with mass-action kinetics, then every equilibrium of the CRN is detailed balanced. Thus we say that the CRN is detailed balanced when its equilibria are detailed balanced or equivalently when the rate constants satisfy the appropriate constraints, called circuit conditions in [7]. If a CRN has this form of detailed balance, then we will say that the CRN satisfies Reaction Network Detailed Balance (RNDB). An unrelated notion of detailed balance arose in the literature on continuous-time Markov chains, also in its early history [6,17]. The equilibrium of a Markov chain (MC), if it exists, is a probability distribution called the stationary distribution. A Markov chain is defined via the transition rates ρ(x, y) between states x and y. A stationary distribution π (which is a non-negative, normalized function on the state space) is detailed balanced if every transition is reversible (ρ(y, x) > 0 if ρ(x, y) > 0 for all states x and y) and π(x)ρ(x, y) = π(y)ρ(y, x) for all states x and y. A Markov chain has a detailed balanced stationary distribution π if and only if the Markov chain satisfies the Kolmogorov cycle condition, which states that the product of the transition rates over every cycle in the graph of the Markov chain is independent of the direction in which the cycle is traversed. Thus a Markov chain which satisfies the Kolmogorov cycle condition over every cycle can be considered a detailed balanced Markov chain.
Despite the fact that the two notions of detailed balance discussed in the previous paragraphs are unrelated, and the two arise independently in distinct contexts, there is a natural reason that they possess the same name. A CRN has a natural graph structure associated with it (see figure 1) and so does a Markov chain. In the case of CRNs, the nodes are chemical complexes (which can be thought of as multisets of species, for instance 2A, A + B, 2B, A + C, D + E, B are complexes in 1). The directed edges are the reactions, where mass-action kinetics provides a positive valued labeling to the edges called the reaction rate constant. For instance, mass-action kinetics associates the rate constant k 2 and the reaction rate k 2 c A c B with the directed edge A + B k 2 → 2B in figure 1. Detailed balance here refers to the property that at equilibrium (which is the fixed point of the dynamical system), the reaction rate for each reaction is balanced by the reaction rate for the corresponding reverse reaction. A Markov chain (in any setting, not necessarily that of CRN) has a natural graph structure associated with it as well. The nodes are the states and the directed edges are the transitions that occur at a positive rate. If at equilibrium (which is the stationary distribution of the Markov chain), the probability flow π(a)p(a, b) from state a to state b is balanced by the reverse flow π(b)p(b, a) for every pair of states {a, b} then we say the Markov chain is detailed balanced. Thus it is appropriate to consider the two kinds of detailed balance discussed here are homonyms, which acquire their precise definition from the context of the graphical structure on which they are defined.
When we consider a stochastic model of a chemical reaction network (which is a continuoustime Markov chain), both graphical structures co-exist. Specifically, the graph structure of the CRN itself, such as the one depicted in figure 1, is present in the background. The graph structure of the Markov chain is defined as follows: the nodes are elements of Z s ≥0 where the i-th component of each node represents the number of molecules of the i-th species. A directed edge (a → b) represents a positive probability of transition between nodes a and b, which in turn exists if there is a reaction in the CRN that allows this transition. Thus the graph structure in the foreground of the stochastic model is the one associated with the Markov chain and it inherits some structural elements from the graph of the CRN. When considering a stochastic model of a CRN with mass-action kinetics, if the corresponding Markov chain has the property of detailed balance, we say that the CRN satisfies Markov chain detailed balance (MCDB).
There are, however, some critical distinctions between the two graphs in the stochastic model of a CRN. Firstly, the nodes in the CRN graph are chemical complexes, while the nodes in the corresponding Markov chain represent the number of molecules of the species. Secondly, the graph of a CRN is finite, while the graph of the corresponding Markov chain usually has an infinite state space. Thirdly, some cycles in the Markov chain are inherited from the cycles in the CRN, but certain cycles arise in the Markov chain from a set of reactions that do not themselves form a cycle in the CRN. For instance, the cycle on the left in figure 2 is clearly inherited from the reaction cycle {A + C ↔ D + E ↔ B ↔ A + C} in figure 1, but there is no equivalent reaction cycle in figure 1 of the cycle on the right in figure 2.
The question then emerges: does detailed balance in a Markov chain model of a CRN with mass-action kinetics have any relation with detailed balance in the ODE model of a CRN with mass-action kinetics? Having considered the distinctions mentioned in the previous paragraph, it seems rather surprising that RNDB and MCDB are even related. However, it appears to be a "folk theorem" that the conditions for RNDB and MCDB are equivalent. The primary purpose of this article is to lay to rest this false notion and establish that RNDB and MCDB are not equivalent. A possible reason for the misunderstanding arising in the first place, may be that a concept of stochastic detailed balance specific to CRNs (which is not the usual Markov chain detailed balance) was introduced by Whittle [21], as an analogue of RNDB. In order to disambiguate, we refer to this notion as Whittle Stochastic Detailed Balance (WSDB). Unfortunately, WSDB and MCDB are often confused with each other (see Theorem 4.5 in [1]). We emphasize that WSDB is not equivalent to MCDB for general CRNs, furthermore, the content of the latter is sensible for all Markov chains, while the former is specific to CRNs. The main result of this article may be summarized as: for reversible reaction networks, We show that the conditions on the rate constants required for RNDB imply MCDB, however the converse is not true in general. Strikingly however, in a vast majority of CRNs, the conditions for RNDB do coincide with those for MCDB, which is perhaps another reason that the two types of detailed balance are often confused. Despite this, it does not take much effort to construct an example of a CRN where RNDB and MCDB do not require identical conditions on the rate constants. Consider, for instance, the CRN 2A ↔ A + B ↔ 2B, which satisfies MCDB independent of rate constants, however does not satisfy RNDB for almost any set of randomly chosen rate constants. We study this CRN and two of its variants in section 2.
This article is organized as follows: section 2 presents certain examples of networks that motivate the main results of the paper; section 3 provides an overview of chemical reaction network theory and mass-action kinetics in the context of deterministic and stochastic models of chemical reaction networks; section 4 describes detailed balance as a collection of ideas and then in specific contexts of deterministic and stochastic models of CRN; section 5 elucidates the relation between reaction network detailed balance and Markov chain detailed balance, and states the main theorem that RNDB implies MCDB; section 6 presents a new algorithm to find constraints on the rate constants that result in MCDB and gives an explicit formulation of the stationary distribution of the stochastic model when the network has reaction network detailed balance.

Motivational examples
We present some examples that will motivate the main results in this article. The first three networks are very similar to each other, however, the relation between RNDB and MCDB is quite different for the three networks.

Network 1
Consider the following chemical reaction network We wish to calculate conditions on the reaction rate constants which will ensure reaction network detailed balance (RNDB) (see section 4.2 for definition) under the assumption of massaction kinetics (see section 3.2 for definition). RNDB is ensured if each reaction is reversible and if at steady state concentration the reaction rate of each forward reaction is equal to the reaction rate of the corresponding backward reaction. Applying this condition to (4), we get the following set of constraints on the reaction rate constants: No conditions RNDB =⇒ MCDB be satisfied simultaneously if and only if the following relations hold among the reaction rate constants: An alternate way to write these relations is as follows where the first relation is a spanning forest condition and the second relation is a circuit condition in the nomenclature of [7]. We now wish to compare the results of the previous calculation with the conditions on the reaction rate constants which ensure Markov chain detailed balance (MCDB) (see section 4.3  indicates the rate of transition while going counterclockwise and indicates the rate of transition while going clockwise. For instance k 1 is the transition rate from (a, b) to (a + 1, b) and k −1 (a + 1) is the transition rate from (a + 1, b) to (a, b). To apply the Kolmogorov cycle condition, we simply set the product of the reaction rates on the path going clockwise equal to the product of the reaction rates on the path going counterclockwise . For this example, this results in [ for definition). Given a chemical reaction network with mass-action kinetics, and assuming a stochastic framework, the state of the system at time t is an element of Z ≥0 representing the number of molecules of each chemical species at time t. Each instance of a chemical reaction changes the population numbers along the reaction vector corresponding to that reaction. This process is naturally modeled as a Markov chain which has a graph structure associated with it, where each directed edge corresponds to a transition that occurs with non-zero probability. MCDB is ensured if the Kolmogorov cycle condition (KCC) holds. KCC is said to hold if each cycle in the graph of the Markov chain can be traversed in either direction, and for each cycle C the product of the transition rates when traversing C clockwise is equal to the product of transition rates when traversing C counterclockwise. For the network in (4), let (a, b), which represents the number of molecules of species A and species B respectively, be the state of the system. For the cycle whose node set is {(a, b), (a + 1, b), (a, b + 1)}, applying KCC results in the following equation which is valid for all a ≥ 1 and b ≥ 1 These conditions hold for all a ≥ 1 and all b ≥ 1 if and only if the following relations apply to the reaction rate constants: a set of relations that is identical to (6). There are other cycle types in the graph of the Markov chain which we have not considered, for instance the cycle that is obtained by considering the transitions resulting from the reactions {0 → A, 0 → B, A → 0, B → 0} (see also figure 4). However, it is straightforward to show that this cycle, or any other cycle, does not result in any new relations among the rate constants. Thus for the network in (4) taken with massaction kinetics, the minimal set of relations among the rate constants that guarantees RNDB is identical to the set of relations that guarantees MCDB. In other words, one may say that RNDB is equivalent to MCDB for the network in (4).

Network 2
Now consider the following chemical reaction network This network is similar to the network in (4), except that the flow reactions {0 ↔ A, 0 ↔ B} are missing. We calculate the conditions on the rate constants that ensure RNDB and the conditions on the rate constants that ensure MCDB. Using the same notation as in (4), we find that for RNDB we need: which implies for the rate constants Figure 5 depicts the graph of the Markov chain for the network in (10). We note that there are no cycles in the graph, which means that the Kolmogorov cycle condition is vacuously satisfied which implies that MCDB holds irrespective of the values of the reaction rate constants, or indeed, MCDB holds independent of the type of kinetics. Note that given an initial population (a 0 , b 0 ), the population (a(t), b(t)) at any future time t is restricted to the line a(t) + b(t) = a 0 + b 0 , thus the dynamics is restricted to one dimension. Since there are no cycles in the graph of the Markov chain, KCC holds vacuously implying that MCDB holds for the network irrespective of the values of the reaction rate constants.

Network 3
On the other hand, consider the following network, which is halfway between networks considered in (4) and (10): While the flow reactions for species A are present, the flow reactions for species B are absent. For this network, the RNDB conditions are still the same as for the network (10), k 2 k −2 = k 3 k −3 and it turns out that the MCDB conditions are identical to the RNDB conditions. Thus including the flow reactions 0 ↔ A does not result in any new conditions on the rate constants for RNDB. However, including the flow reactions 0 ↔ A is sufficient to introduce cycles into the graph of the Markov chains, which introduces a new condition for MCDB, a condition that was already required for RNDB in the network (10) without flow reactions. Thus the introduction of the flow reactions for species A is sufficient to make the conditions for RNDB and MCDB equivalent.
This example might lead one to conjecture that the presence of cycles in the graph of a Markov chain is sufficient to guarantee that MCDB and RNDB are equivalent. However, the next example provides a counter to this claim since the graph of the Markov chain has cycles and the MCDB conditions are a nonempty, proper subset of the conditions for RNDB.

Network 4
RNDB requires that . Thus the set of conditions required for MCDB is a nonempty, proper subset of the conditions required for RNDB. These examples illustrate the main result stated in Theorem 5.9, that reaction network detailed balance implies Markov chain detailed balance but the converse is not true in general. The results obtained in this section are summarized in Table 1. 3 Chemical reaction network theory

Introduction and basic definitions
We begin with a review of the notation and basic definitions related to chemical reaction networks. An example of a chemical reaction is the following: The X i are called chemical species, and X 1 + 2X 2 and X 2 + X 3 are called chemical complexes.
For the reaction in (15), y := X 1 + 2X 2 is called the reactant complex and y := X 2 + X 3 is called the product complex, so we may rewrite the reaction as y → y . We will find it convenient to think of the complexes as vectors, for instance, we may assign the reactant complex X 1 +2X 2 to the vector (1, 2, 0) and the product complex X 2 + X 3 to the vector (0, 1, 1). In other words, we are identifying the species X i with the canonical basis vector whose i-th component is 1 and the other components are 0. We let s denote the total number of species in the network under consideration and we let r represent the number of reactions, each reaction written as y i → y i , for i ∈ {1, 2, . . . , r}, and y i , y i ∈ Z s ≥0 , with y i = y i . We index the entries of a complex vector y i by writing y i = (y i1 , y i2 , . . . , y is ) ∈ Z s ≥0 , and we will call y ij the stoichiometric coefficient of species j in complex y i . For ease of notation, when there is no need for enumeration we typically will drop the subscript i from the notation for the complexes and reactions. We will reserve boldface fonts for vectors (usually of dimension s -the components of which refer to different species). In particular, we may sometimes need to use subscripts to denote a finite sequence of vectors, and so (u 1 , . . . , u c ) will denote a sequence of vectors while (u 1 , . . . , u s ) will denote components of a vector u. The disambiguating notation is only out of abundance of caution, as the context will usually make the type of the object clear.
The basic definitions and notations in this paper follow those in Joshi and Shiu [15,16] and Joshi [14]; we start by defining chemical reaction networks. 1. for each complex y ∈ C, there exists a reaction in R for which y is the reactant complex or y is the product complex, and 2. for each species X i ∈ S, there exists a complex y ∈ C that contains X i .
For a chemical reaction network (S, C, R), unless otherwise specified, we will denote the number of species by s := |S|, the number of complexes by n := |C| and the number of reactions by r := |R|.
We will say u is a reaction vector if u is the reaction vector of some reaction in R. Let V (R) be the set of all reaction vectors, V (R) := {y − y|y → y ∈ R}. Let v := |V (R)| be the number of reaction vectors in R. The stoichiometric subspace of a network is the vector space over R spanned by the reaction vectors of the network, S R (R) := span(V (R)). The stoichiometric module of a network is the Z-module generated by the reaction vectors of the network, denoted by S Z (R).
2. Let R(u) := {y → y |y − y = u} represent the set of reactions whose reaction vector is u, so that ∪ u∈V (R) R(u) = R. Let r(u) := |R(u)| denote the cardinality of the set R(u), so that u∈V (R) r(u) = r. Let C(u) be the set of reactant complexes for the reactions in R(u) and let C (u) be the set of product complexes for the reactions in R(u).
Definition 3.3. We say that the reaction (y → y ) ∈ R is reversible if (y → y) ∈ R. We say that a chemical reaction network is reversible or that R is reversible if all reactions in R are reversible.
Definition 3.4. 1. For x ∈ Z ≥0 and y ∈ Z, we define the falling factorial as follows: In particular, a > 0 means that a ∈ Z s >0 and a ≥ 0 means that a ∈ Z s ≥0 .

Mass-action kinetics
For a deterministic chemical reaction network, the state of the system at any time is specified by x = (x 1 , x 2 , . . . , x s ) ∈ R s where x i are the species concentrations and for a stochastic chemical reaction network, the state of the system is specified by a = (a 1 , a 2 , . . . , a s ) ∈ R s where a i are the species numbers. In general, the rate at which a reaction (y → y ) ∈ R occurs is a function of the reactant complex y, the product complex y and either the species concentration vector x in the deterministic case or the species number vector a in the stochastic case. Furthermore, this reaction rate function is parametrized by reaction rate constants.
is the stochastic reaction rate. In each case, k represents the set of reaction rate constants which parametrize the reaction rates.
The choice of a certain class of functions for reaction rates is referred to as specifying kinetics. In this article, we restrict our attention to the common choice of mass-action kinetics. For mass-action kinetics, a single reaction rate constant k y→y > 0 is associated with each reaction y → y ∈ R. For a chemical reaction network (S, C, R) with r = |R|, let the (ordered) set of positive mass-action reaction rate constants be denoted by k = (k 1 , k 2 , . . . , k r ) ∈ R r >0 . For a reaction vector u, let k(u) = (k 1 (u), k 2 (u), . . . , k r(u) (u)) represent the (ordered) set of rate constants for reactions in the set R(u), so that ∪ u∈V (R) k(u) = k (up to ordering). For reversible networks, we will always assume consistent ordering by which we mean that k j (−u) is the rate constant of the backward reaction whose forward reaction has the rate constant k j (u).
When the chemical reaction network is endowed with stochastic mass-action kinetics, the result is a continuous-time Markov chain model. The alternative is to specify deterministic mass-action kinetics which results in a system of ordinary differential equations. Definition 3.6 defines the two types of mass-action kinetics.
1. Let a i ∈ Z ≥0 represent the population of the chemical species X i and let a = (a 1 , . . . , a s ) ∈ Z s ≥0 represent the state of the system. A chemical reaction network is said to be endowed with stochastic mass-action kinetics if for a state a and for all reaction vectors u the transition a → a + u happens at rate: If the system is in state a, the instantaneous rate at which the populations of the species change is given by 2. Let x i represent the concentration of the chemical species X i and let x = (x 1 , . . . , x s ) ∈ R s ≥0 represent the state of the system. A chemical reaction network is said to be endowed with deterministic mass-action kinetics if the time-evolution of x(t) is governed by the following system of ordinary differential equations: The striking resemblance between sto(a) in (18) and det(x) in (19) is not a coincidence. In fact, the time evolution of (19) closely approximates the dynamical evolution of (18) for the special case of large population numbers.
If the initial state for the stochastic mass-action system is a 0 ∈ Z s ≥0 , then for all future times the trajectory is restricted to P a 0 := (a 0 + S Z ) ∩ Z s ≥0 . Thus P a 0 is the set of states that are accessible from a 0 . Similarly in the case of the deterministic system, if the initial position is x 0 ∈ R s ≥0 , then the trajectory is confined to P x 0 := (x 0 + S R ) ∩ R s ≥0 for all positive time. In the chemical reaction network theory literature, P x 0 is referred to as the stoichiometric compatibility class containing x 0 . Definition 3.7.
1. For deterministic mass-action kinetics, we refer to P x 0 := (x 0 + S R ) ∩ R s ≥0 as the stoichiometric compatibility class containing x 0 .
2. For stochastic mass-action kinetics, we refer to P a 0 := (a 0 +S Z )∩Z s ≥0 as the stoichiometric compatibility class containing a 0 .
Remark 3.8. In the stochastic setting, the stoichiometric compatibility class containing a 0 is the set of states that are accessible from a 0 . In a reversible chemical reaction network, for all initial states a 0 ∈ Z s the stochastic stoichiometric compatibility class P a 0 is irreducible. The reason for this is that if a ∈ P a 0 then a is accessible from a 0 , on the other hand reversibility of the underlying chemical reaction network implies that a 0 is accessible from a. For a general CRN (not necessarily reversible), P a 0 is a subset of the basin of attraction of some irreducible subset of the Markov chain.

Topological structure of Markov chain arising from CRN
The graph corresponding to the transition (or adjacency) matrix of the Markov chain induced by a chemical reaction network has a special "lattice structure" which we now describe.
Definition 3.9. u is said to be a reaction vector based at a if there is a (y → y ) ∈ R such that y − y = u and a ≥ y.
Proposition 3.10. Suppose that u is a reaction vector based at a. Then for b ∈ Z s ≥0 , u is a reaction vector based at a + b.
Proof. Since u is a reaction vector based at a, there is at least one reaction (y → y ) ∈ R such that y − y = u and a ≥ y. Clearly, a + b ≥ y and so u is a reaction vector based at a + b. Proposition 3.11. If at least one reaction in R(u) is reversible, and u is a reaction vector based at a, then −u is a reaction vector based at a + u.
Proof. Let y → y + u be reversible, so that (y + u → (y + u) − u) ∈ R, from which the result follows.
These easy propositions indicate that the topological structure of the Markov chain that arises from chemical reaction networks is highly constrained, and in particular possesses the highly symmetrical and repeating structure of a lattice. Many properties of the graph of a Markov chain arising from a CRN are studied in [20]. 1. If u is a reaction vector based at a and −u is a reaction vector based at a + u, then we say that u is a reversible reaction vector based at a. (u 1 , . . . , u c ) is called a reaction cycle if c i=1 u i = 0. By cyclic ordering, we mean that the reaction cycles (u 1 , u 2 , . . . , u c ) and (u 2 , u 3 , . . . , u c , u 1 ) are considered identical.

A cyclically ordered nonempty set of reaction vectors
• A reaction cycle U is trivial if U = (u, −u), otherwise it is nontrivial. Without explicitly mentioning it, we will always assume modular arithmetic ( mod c) for the indices in a reaction cycle of length c. For instance, for the reaction cycle (u 1 , . . . , u c ), the indices satisfy c + 1 = 1, and the vectors u c and u 1 are considering adjacent.
4 What is detailed balance?

Detailed balance as a collection of related ideas
Detailed balance arises in deterministic chemical reaction network theory [7,18,19,22], theory of stochastic CRNs [21], theory of discrete-time and continuous-time Markov chains (see [17] for numerous examples), and electrical networks [4]. When the context is not provided, a detailed balanced network or a detailed balanced equilibrium should be thought of as a meta-concept or as a collection of related ideas each of which has the following properties: 1. there is either an explicit or implicit underlying network structure which is fixed in time, and has the property that for every pair of nodes {a, b}, if (a, b) is an edge then (b, a) is an edge, 2. there is a positive real-valued function defined on the edge set which is fixed in time, called the edge weight, 3. there is a dynamically evolving (either in discrete or continuous-time) function, whose evolution is at least partially dictated by the edge weights, called the state variable, 4. a dynamic equilibrium can be defined and exists for the system, 5. each directed edge has some notion of "flow" associated with it (which is in general, a function of the edge weight, the node where the edge originates, and the state variable) 6. a detailed balanced equilibrium exists if, at this equilibrium, the flow from node a to node b is equal to the flow from node b to node a for every pair of nodes (a, b) in the network.
7. detailed balance is a property of the network taken with the edge weights rather than of any particular equilibrium of the dynamics on the network. This is because the conditions that are required to show the detailed balance property of an equilibrium are shown to be equivalent to conditions that can be written as constraints on the fixed edge weights. Such conditions include circuit conditions in the case of deterministic mass-action CRNs, and Kolmogorov cycle conditions in the theory of Markov chains.
In table 2, we provide the specific instantiations of the above-listed properties in the contexts of (1) chemical reaction network, and (2)   We now proceed to define the two types of detailed balance that are relevant for this article.

Detailed balance in deterministic models of CRNs
Definition 4.1 (Reaction network detailed balance). 1. By steady state of a chemical reaction network, we mean a steady state of the system of differential equations resulting from applying the kinetic scheme to the network. Thus for deterministic mass-action kinetics, x * is a steady state if det(x * ) = 0 in equation (19). If x * ∈ R s >0 (∈ R s ≥0 ), we say that x * is a positive (non-negative) steady state.

A steady state of a reversible chemical reaction network is said to be detailed balanced if
at the steady state, the rate of each forward reaction (y → y ) ∈ R is equal to the rate of the backward reaction y → y.

A reversible chemical reaction network is said to satisfy reaction network detailed balance
if each positive steady state of the network is detailed balanced.

Remark 4.2.
For mass-action kinetics, if one positive steady state of a network is detailed balanced then every positive steady state is detailed balanced. Thus, detailed balance is a property of the network itself rather than any particular steady state of the network, for details see for instance [7].
For mass-action kinetics, reaction network detailed balance can be guaranteed by specifying certain relations between the rate constants. Let u be a reaction vector. Let y i (u) . For a reaction network detailed balanced network, if x * is a positive steady state, then we must have that This condition which relates the detailed balanced steady state coordinate values to the reaction rate constants can be written simply as a relation between rate constants without invoking the steady states. We state the result in Theorem 4.3. The content of Theorem 4.3 can be found in [7]. However, we provide an alternate formulation that is suited to our purposes and does not involve the more technical graph-theoretic jargon of [7]. We also provide a simple proof for the forward direction. 1. For a reaction vector u, 2. For a reaction cycle (u 1 , u 2 , . . . , u c ) and for all q(u i ) ∈ {1, 2, . . . , r( Proof. Suppose first that a chemical reaction network satisfies reaction network detailed balance. For a reaction cycle (u 1 , u 2 , . . . , u c ), c i=1 u i = 0, and so We used (20) in the last step thus proving the second identity (22). The first identity (21) then follows by considering the reaction cycle (u 1 , u 2 ) = (u, −u) and considering q(u 1 ) = q(u 2 ). We will omit proof of the converse and refer the interested reader to [7]. Remark 4.4. As the proof makes it clear, the second condition for RNDB, (22) subsumes the first condition (21). Thus to show RNDB, it is sufficient to show that (22) holds for all reversible reaction cycles. Nevertheless, it is convenient to think of (21) as a condition on each reaction vector, and (22) as a condition on nontrivial reaction cycles.
We now define the notion of complex balancing in chemical reaction networks, first introduced by Horn and Jackson in [13]. This idea will be useful to us when determining the stationary distribution of certain Markov chains arising from chemical reaction networks. More details on complex balancing can be found in [8,12], also see [1,5] for more recent work. Dickenstein and Millán explore the relation between detailed balance and complex balance in reversible networks [5].
Definition 4.5. A chemical reaction network is said to be complex balanced if for each complex y ∈ C, at equilibrium, the sum of reaction rates of reactions with y as the reactant complex is equal to the sum of reaction rates of reactions with y as the product complex, It is easy to see that detailed balance implies complex balance. In the case of mass-action kinetics, the complex balancing condition can be written as: (24)

Detailed balance in Markov chains
The definition of Markov chain detailed balance comes from the theory of Markov chains, we will state the definition in generality since it does not depend on the particular setting of chemical reaction networks.
Definition 4.6 (Markov chain detailed balance). A continuous-time Markov chain with state space Ω and transition rate matrix ρ is said to satisfy detailed balance if there exists a nonnegative valued function π, not identically equal to 0, such that for every pair of states x, y ∈ Ω, we have the relation π(x)ρ(x, y) = π(y)ρ(y, x). A reversible chemical reaction network is said to satisfy Markov chain detailed balance (MCDB) if the continuous-time Markov chain resulting from applying mass-action kinetics satisfies detailed balance. An equivalent condition to detailed balance in a Markov chain is the Kolmogorov cycle condition, see for instance [6,17]. We will make use of this Kolmogorov criterion for establishing Markov chain detailed balance of chemical reaction networks.
We will now translate KCC to reversible chemical reaction networks with stochastic massaction kinetics. Consider a reversible reaction cycle (u 1 , u 2 , . . . , u c ) based at a. Then (x 0 = a, x 1 = a + u 1 , x 2 = a + u 1 + u 2 , . . . , x c−1 = a + u 1 + u 2 + . . . + u c−1 ) is a cycle of states of length c. We replace the transition rates for chemical reaction networks from (17) into (25) to get the following: where we used r(−u) = r(u) which holds for reversible networks. If (26) holds on every cycle (u 1 , u 2 , . . . , u c ) in the graph of the transition matrix of the Markov chain, then the network satisfies Markov chain detailed balance. On the face of it, the conditions on the reaction rates required for reaction network detailed balance- (21) and (22)-look quite dissimilar to the rate condition for Markov chain detailed balance (26). We will show that the two are closely related, and in fact reaction network detailed balance implies Markov chain detailed balance. Furthermore, in a vast majority of applications the two are equivalent.

Detailed balance in stochastic models of CRNs
Whittle [21] defined a notion of detailed balance for chemical reaction networks with stochastic mass-action kinetics, which is distinct from the usual definition of detailed balance in Markov chains. In order to avoid confusion, we will refer to Whittle's notion as Whittle stochastic detailed balance (WSDB).

Preliminaries
In this section, we will flesh out the precise relation between RNDB, WSDB and MCDB. First we state a theorem due to Whittle [21]: See Whittle [21] for a proof. In this section we will show that in general the following holds for reversible chemical reaction networks with mass-action kinetics: Let Z be the set of reversible reaction cycles on R. Let U : Z × Z s → R be defined via where (u 1 , . . . , u c ; a) is a reversible reaction cycle based at a. If for all a ∈ Z s ≥0 and for all reaction cycles U ∈ Z based at a, we have U(U ; a) = 1, then the network satisfies Markov chain detailed balance. In order to find conditions on the rate constants of R for Markov chain detailed balance, the first order of business is to bring (29) into a more workable form. We start with a lemma which facilitates manipulation of the falling factorials.
Lemma 5.2. The following identities hold for x ∈ Z s ≥0 and a, b, u ∈ Z s : In Property 2. For x ≥ a and x + u ≥ 0, In particular, this implies that for x ≥ a, b, we have (x − a) b−a ≥ 1 and (x − b) a−b ≥ 1. So that for 1 ≤ i ≤ s, Then Property 1 follows by taking a product over 1 ≤ i ≤ s.
Since x ≥ a and x + u ≥ 0, (x) a > 0 and (x + u) a+u > 0. For 1 ≤ i ≤ s, The last equality can be checked to hold both for u i ≥ 0 and u i < 0 when x i + u i > 0. Both equations of Property 2 then follow by taking a product over 1 ≤ i ≤ s.
Recall the definition of stochastic mass-action kinetics which gives the rate of transition a → a + u for a reaction vector u: Proof. If y i (u) is a reactant complex then (y i (u) → y i (u) + u) ∈ R(u). This implies that (y i (u) + u → y i (u)) ∈ R(−u). Since R(u) and R(−u) are consistently ordered, it then follows that The result then follows.
Lemma 5.4. If u is a reaction vector based at a, then the rate of transition a → a + u is Proof. By definition of m(u), y i (u) ≥ m(u) for all reaction vectors u. Since u is a reaction vector based at a, there is at least one y i (u) such that a ≥ y i (u). So that where in the last step we used Property 1 of Lemma 5.2.
Lemma 5.5. If u is a reversible reaction vector based at a then Proof. By Lemma 5.3, m(−u) = m(u) + u. Since u is a reaction vector based at a, there is at least one y i (u) such that a ≥ y i (u) ≥ m(u). Further since u is a reversible reaction vector based at a, it must be that −u is a reaction vector based at a + u and so a + u ≥ 0. The result then follows immediately from Property 2 of Lemma 5.2.
Proof. By definition of U, The first line is Definition 29, the second line follows from Lemma 5.4, the third line from Lemma 5.6 and the last line from Lemma 5.3.
We introduce some notation which will simplify the appearance of U ((u 1 , . . . , u c ; a)).
• To each reaction vector u based at a we associate a vector of length r(u) via the following definition: We state some simple but useful properties of F as a lemma. (ii) If r(u) = r(−u) = 1, then F(u, a) = 1 for all a ∈ Z s ≥0 .
Proof. The first result follows easily from Lemma 5.3 and the second result follows from the fact that if r(u) = 1 then y 1 (u) = m(u).
• These definitions result in the following compactified version of (34) Here the symbol " · " denotes the usual scalar product. Thus the condition for MCDB is that for every reversible reaction cycle (u 1 , u 2 , . . . , u c ) based at a, we must have that: where F(u, a) = (a − m(u)) y 1 (u)−m(u) , . . . , (a − m(u)) y r(u) (u)−m(u) . It is clear that in order to show MCDB, it suffices to check conditions (37) for all irreducible, nontrivial, reversible reaction cycles.

RNDB implies MCDB
We now possess the technology required to prove the main result of the article which is that RNDB implies MCDB.
Theorem 5.9. Suppose that a reversible reaction network (S, C, R) satisfies reaction network detailed balance. Then (S, C, R) satisfies Markov chain detailed balance.
Proof. Let (u 1 , u 2 , . . . , u c ) be a reaction cycle based at a. For 1 ≤ j ≤ c, where we used RNDB condition (21) from Theorem 4.3 to get the last equality above.
where the last equality follows from the RNDB condition (22).
Thus reaction network detailed balance in a reversible chemical reaction network endowed with mass-action kinetics implies Markov chain detailed balance. The converse is not true in general. We study some instances where the converse holds, followed by instances where the converse does not hold.

When does MCDB imply RNDB?
Even though MCDB does not imply RNDB in general, the two are in fact equivalent for a vast majority of networks. There is at least one commonly occurring case where the conditions for the equivalence can be checked at a glance. This occurs when there is exactly one reaction corresponding to each reaction vector. We state the result as Theorem 5.10. Proof. Consider a reaction cycle (u 1 , u 2 , . . . , u c ) based at a. Substituting r(u j ) = 1 in (36) and using Lemma 5.8, we find that U ((u 1 , . . . , u c ; a) which is equal to 1 if and only if RNDB is satisfied (by RNDB condition (22)). This shows that RNDB and MCDB are equivalent.

Which networks have MCDB but do not have RNDB?
In a reversible reaction network, if there is only one pair of reversible reaction vectors (u, −u) and there is more than one reaction whose reaction vector is u, MCDB holds independent of any constraints, but RNDB fails to hold in general. Proof. From any state a, the set of states that can be reached in one time step is a subset of {a + u, a − u}, so the process is a reversible birth and death process which is known to satisfy MCDB.
Network 2 in Section 2 provides an example of a network which satisfies the hypotheses of Theorem 5.11. Another example is the network in (41). For birth and death processes, MCDB holds because the absence of cycles in the graph of the Markov chain makes the Kolmogorov cycle conditions hold vacuously. However, as Theorem 5.12 shows, the mere presence of cycles is not sufficient to make MCDB equivalent to RNDB.
Theorem 5.12. Consider a reversible reaction network (S, C, R). Suppose that V (R) is linearly independent and for all u ∈ V (R) and v ∈ V (R)\{u, −u}, F(u, a) = F(u, a+v). Then (S, C, R) satisfies MCDB for all reaction rate constants.
We will need some technical results in order to prove Theorem 5.12.
Lemma 5.13. Consider a reversible reaction network (S, C, R). Suppose that u ∈ V (R) is a reaction vector with the following properties: Then MCDB does not require any constraints on k(u) or k(−u).
Proof. Let u be a reaction vector that satisfies the hypotheses of the theorem. Let U := (u 1 = u, u 2 , . . . , u c ) be an irreducible, nontrivial, reversible reaction cycle based at a. Since u / ∈ span(V (R)\{u, −u}), we must have that −u ∈ U , so that U = (u, u 2 , . . . , u k , −u, u k+2 , . . . , u c ). Since U is irreducible, u and −u are nonconsecutive and v := k i=2 u i is nonzero and in span(V (R) \ {u, −u}). So we can write the reaction cycle U as U = (u, v, −u, −v), where v is a sum of reaction vectors.
In the expression of U(U ; a) the factor involving k(u) and k(−u) is: where the first equality follows from applying Lemma 5.8, and the next equality follows from F(u, a) = F(u, a + v). Since the other factors in U(U ; a) do not involve either k(u) or k(−u), it is clear that any set of conditions for MCDB do not constrain k(u) or k(−u).
Theorem 5.14. Consider a reversible reaction network (S, C, R). Suppose that u ∈ V (R) is a reaction vector with the following properties: (c) r(u) > 1.
Then there exist a set of rate constants for which the reaction network possesses MCDB but does not possess RNDB.
Proof. By Lemma 5.13, k(u) and k(−u) are unconstrained. On the other hand, if RNDB holds then k(−u) is a constant multiple of k(u). Since r(u) > 1, this results in a nontrivial condition on the rate constants, a condition absent from the requirements for MCDB, thus proving the theorem.
In the following example, we look at a network with two reaction vector pairs (±1, 0) and (0, ±1). The conditions of theorem 5.14 hold for (±1, 0) but not for (0, ±1), and so we find that the MCDB constraints form a proper subset of RNDB constraints.
Example 5.15. Consider the following example Network 4 from section 2.

3B
Let u = (1, 0) and v = (0, 1). It is easy to show that F(u, a) = 1, a(a − 1) , so that F(u, a) = F(u, a + v). Clearly, u and v are linearly independent, so u satisfies the hypotheses of Theorem 5.14, and thus MCDB does not require any constraints on k(u) = {k 1 , k 2 } and k(−u) = {k −1 , k −2 }. RNDB does require that k 1 k −1 = k 2 k −2 . Note however that F(v, a) = a, b(b − 1) , and so F(v, a) = F(v, a + u). Thus MCDB requires that k(v) which is also necessary for RNDB. Now we are ready to prove Theorem 5.12.
Proof of Theorem 5.12. Let u ∈ V (R). Both hypotheses of Lemma 5.13 are satisfied for u, thus implying that there are no constraints on k(u) or k(−u). Since this is true for all reaction vectors u ∈ V (R), in fact no conditions are required on the rate constants for MCDB.
The following example illustrates the content of Theorem 5.12 and Theorem 5.14 that having cycles in the graph of the MC is not sufficient to make RNDB and MCDB equivalent.
Example 5.16. Consider a "disjoint union" of two networks: (40) Notice that the network decouples into a subnetwork involving only species A and a subnetwork involving only species B. (The shared 0 does not count as a species.) The network is easily shown to satisfy the hypotheses of Theorem 5.12, thus the network satisfies MCDB for all reaction rate constants.

Birth and death processes
RNDB implies that the deterministic dynamical system has a unique positive steady state in each stoichiometric compatibility class. MCDB implies that the stochastic system has a unique stationary distribution. Since, RNDB and MCDB often co-occur, the stationary distribution has a mode around the deterministic steady state. Thus, RNDB results not only in the existence of a stationary distribution in the Markov chain, but it constrains the shape of the distribution as well. The following example of a birth and death process is a rare example where the network has MCDB without RNDB, and the distribution is bimodal. When V (R) = {u, −u} and r(u) ≥ 2 the network, in general, has MCDB (as shown in Theorem 5.11) but not RNDB. Satisfying MCDB does not require any constraints on the rate parameters, however RNDB involves at least one condition, and therefore at least one less degree of freedom in the available parameter space. Network 2 in Section 2 provides one example. Another is given by the following network: The network in (41) is notable because it is (in a sense described below) the simplest example of a continuous-flow stirred tank reactor network (CFSTR network) with multiple positive massaction stable steady states. See for instance [14,16], where a close relative of (41) -obtained by making the reaction 2A → 3A irreversible -is studied and shown to be the "smallest atom of multistationarity". A CFSTR network is a chemical reaction network where every chemical species is in inflow and outflow, thus for every species A in the network a reaction of the type 0 A is included in the network. An atom of multistationarity is a CFSTR network which has the property of possessing multiple positive steady states (for some positive parameter values) and is a minimal network with respect to this property in the sense that if a single species or a single reaction is removed then the network loses multistationarity. The network (41) is a minimal element within the class of CFSTR networks which admit multiple positive steady states that are stable. We extend the notion of atom of multistationarity to that of atom of multistability, to be a network which admits multiple stable steady states and is minimal with respect to possessing this property.
It is a fairly simple exercise to find reaction rate constants for which (41) does not possess RNDB and has multiple positive stable steady states. For instance, consider the rate constants below:

3A
(42) The network above possesses two stable steady states at a 1 = 5 and a 3 = 100 and an unstable steady state at a 2 = 27. The stationary distribution is easily calculated to be where Γ is the normalization constant. Figure 7 depicts a graph of the equation in (43). The stationary distribution is seen to be bimodal with the two modes located approximately at the stable steady states of the deterministic system.

A new algorithm for determining MCDB conditions
Since the conditions on rate constants for RNDB guarantee MCDB, there exists a set of conditions for MCDB which is a subset of the conditions for RNDB. For a complicated reaction network, there may be a huge number of cycle types in the graph of the corresponding Markov chain, which means that finding the complete set of conditions for MCDB may be nontrivial. However, we can first determine the conditions for RNDB, which is a significantly simpler exercise, and then use these conditions to circumscribe the set of conditions for MCDB.
For a state a in the Markov chain, let Λ a be the set of states that communicate with a. Let a 0 ∈ Λ a be an arbitrary reference state. Then we will show that the stationary distribution π is given by π(a) ∝ 1 a! k compu(a;a 0 ) The stationary distribution can be directly constructed from the reaction network detailed balance property of the chemical reaction network. Alternatively, one can use the following theorem of Anderson et al.   2010)). Let (S, C, R) be a chemical reaction network endowed with mass action kinetics. Suppose the deterministic system is complex balanced with a complexbalanced equilibrium at x * ∈ R s >0 . For a communicating class of states containing the initial state a 0 , the stationary distribution π in the corresponding stochastic model is given by π(a) ∝ (x * ) a a! , a ∈ Λ a 0 .
Thus we find the stationary distribution to be: