University of Birmingham Compositional probabilistic verification through multi-objective model checking

Compositional approaches to veriﬁcation offer a powerful means to address the challenge of scalability. In this paper, we develop techniques for compositional veriﬁcation of probabilistic systems based on the assume-guarantee paradigm. We target systems that exhibit both nondeterministic and stochastic behaviour, modelled as probabilistic automata, and augment these models with costs or rewards to reason about, for example, energy usage or performance metrics. Despite signiﬁcant theoretical advances in compositional reasoning for probabilistic automata, there has been a distinct lack of practical progress regarding automated veriﬁcation. We propose a new assume-guarantee framework based on multi-objective probabilistic model checking which supports compositional veriﬁcation for a range of quantitative properties, including probabilistic ω -regular speciﬁcations and expected total cost or reward measures. We present a wide selection of assume-guarantee proof rules, including asymmetric, circular and asynchronous variants, and also show how to obtain numerical results in a compositional fashion. Given appropriate assumptions to be used in the proof rules, our compositional veriﬁcation methods are, in contrast to previously proposed approaches, eﬃcient and fully automated. Experimental results demonstrate their practical applicability on several large case studies, including instances where conventional probabilistic veriﬁcation is infeasible.


Introduction
Many computerised systems exhibit probabilistic behaviour, for example due to the use of randomisation (e.g., in distributed communication or security protocols), or the presence of failures (e.g., in faulty devices or unreliable communication media). The prevalence of such systems in today's society makes techniques for their formal verification a necessity. This requires models and formalisms that incorporate both probability and nondeterminism. Although efficient algorithms for verifying such models are known [1][2][3] and mature tool support exists [4,5], applying these techniques to large, real-life systems remains challenging, and hence techniques to improve scalability are essential.
In this paper, we focus on compositional verification techniques for probabilistic and nondeterministic models, in which a system comprising multiple interacting components can be verified by analysing each component in isolation, rather than verifying the much larger model of the whole system. In the case of non-probabilistic models, a successful approach is the use of assume-guarantee reasoning [6,7]. This is based on checking queries of the form Ψ A M Ψ G , with the meaning "whenever component M is part of a system satisfying the assumption Ψ A , then the system is guaranteed to satisfy property Ψ G ". Proof rules can then be established to show, for example, that, if a component M 1 satisfies assumption Ψ A and Ψ A M 2 Ψ G holds for a second component M 2 , then the combined system M 1 M 2 satisfies Ψ G .
For probabilistic systems, compositional approaches have also been studied, but a distinct lack of practical progress has been made. In this paper, we present novel assume-guarantee techniques for compositional verification of systems exhibiting both probabilistic and nondeterministic behaviour, and illustrate their applicability and efficiency on several large case studies. This is the first approach that, given appropriate assumptions about components, can perform compositional verification in an efficient and fully-automated manner.
We use probabilistic automata (PAs) [8,9], a well-studied formalism that is naturally suited to modelling multi-component probabilistic systems. We also augment PAs with rewards (or, dually, costs), which can be used to model a variety of quantitative measures of system behaviour, such as execution time or power consumption. We present compositional techniques for verification of a range of quantitative properties, including probabilistic ω-regular properties (which subsume, for example, probabilistic LTL and probabilistic safety properties) and expected total reward/cost properties (which can also encode the expected reward/cost to reach a target and time-bounded reward measures).
Probabilistic automata were developed as a formalism for the modelling and analysis of distributed, randomised systems [8], and a rich underlying theory has been developed, in particular for models in which PAs are combined through parallel composition. A variety of elegant proof techniques have been created and used to manually prove the correctness of large, complex randomised algorithms [10]. Key ingredients of the underlying theory of PAs include probabilistic versions of strong and weak (bi)simulation [9] and trace distribution inclusion [8]. The branching-time preorders (simulation and bisimulation) have been shown to be compositional [9] (i.e., preserved under parallel composition), but are often too fine to give significant practical advantages for compositional verification. Trace distribution inclusion, which is defined in terms of probability distributions over sequences of observable actions, is a natural generalisation of the (non-probabilistic) notion of trace inclusion but is known not to be preserved under parallel composition [11]. Thus, other proposals for compositional verification frameworks based on PAs tend to restrict the forms of parallel composition that are allowed [12,13]. By contrast, the approach we present in this paper does not impose restrictions on the parallel composition permitted between components, allowing greater flexibility to model complex systems.
Our assume-guarantee framework uses multi-objective probabilistic model checking [14,15], which is a technique for verifying multiple, possibly conflicting properties of a probabilistic automaton. Conventional verification techniques for PAs quantify over its adversaries, which represent the various different ways in which nondeterminism in the model can be resolved. A typical property to be verified states, for example, "the probability of a system failure is at most 0.01, for any possible adversary". Multi-objective model checking, on the other hand, permits reasoning about the existence of an adversary satisfying two or more distinct properties, for example, "is there an adversary under which the probability of a system failure is at most 0.005 and the expected battery lifetime remains below 2 hours?".
Our compositional approach to verification is based on queries of the form Ψ A M Ψ G , with the meaning "under any adversary of PA M for which assumption Ψ A is satisfied, Ψ G is guaranteed to hold". The assumptions Ψ A and guarantees Ψ G are quantitative multi-objective properties [15], which are conjunctions of predicates, each of which imposes a bound on either the probability of an ω-regular property or the expected total value of some reward structure. A simple example of an assumption is "with probability 1, component M 1 eventually sends a request, and the expected time before this occurs is at most 5 seconds". We show that checking these assume-guarantee queries can be reduced to existing multi-objective model checking techniques [14,15], which can be implemented efficiently using linear programming.
Building upon this notion of probabilistic assume-guarantee reasoning, we formulate and prove several compositional proof rules, which can be used to decompose the process of verifying a multi-component probabilistic system into several smaller sub-tasks. One important class of such proof rules is those that restrict assumptions and guarantees to be probabilistic safety properties, which impose a bound on the probability of satisfying a regular safety property. These are slightly cheaper to verify than the other properties we consider, but still represent a useful set of system properties. In order to present proof rules for the more general class of quantitative properties (probabilistic ω-regular and expected total reward), we incorporate a notion of fairness, restricting our analysis to cases where each component in a system executes a step infinitely often.
For both of these classes of properties, we present several different assume-guarantee proof rules, including variants that are asymmetric (using assumptions only about one component) and circular (assumptions about all components). We also give proof rules for systems with components that are asynchronous and methods to decompose the analysis of reward-based properties. Finally, we describe how to obtain numerical results from compositional verification, in particular, obtaining lower and upper bounds on the actual probability that a system satisfies a property and constructing Pareto curves to investigate trade-offs between multiple system properties in a compositional fashion.
We have implemented our assume-guarantee verification techniques by extending the PRISM model checker [4], and present experimental results from its application to several large case studies. We demonstrate significant speed-ups over conventional, non-compositional verification, and also successfully verify models that are too large to be analysed without compositional techniques.

Paper structure
The remainder of the paper is structured as follows. Section 2 introduces the necessary background material regarding probabilistic automata, including notions required for compositional methods such as parallel composition and projection. Section 3 summarises the classes of properties of PAs that we use in this paper and techniques for their verification. It also covers the topics of multi-objective model checking and verification of PAs under fairness constraints. Section 4 introduces our assume-guarantee framework, defining the basic underlying ideas and presenting instances of the two main classes of compositional proof rules that we consider. Section 5 gives several additional proof rules and then Section 6 discusses how our techniques can be adapted to produce numerical results. In Section 7, we describe an implementation of our techniques and show results from its application to several large case studies. Section 8 concludes the paper.

Probabilistic automata (PAs)
Probabilistic automata [8,9] are commonly used for modelling systems that exhibit both probabilistic and nondeterministic behaviour. They are a slight generalisation of Markov decision processes. 2 For the purposes of applying standard probabilistic verification techniques, the two models can often be treated identically; however, probabilistic automata are particularly well suited to compositional modelling and analysis of probabilistic systems.
• S is a set of states and s ∈ S is an initial state; • α M is an alphabet of action labels; • δ M ⊆ S × α M × Dist(S) is a probabilistic transition relation; • L : S → 2 AP is a labelling function mapping states to sets of atomic propositions taken from a set AP.
In any state s of a PA M, a transition, denoted s a − → μ, where a is an action label and μ is a discrete probability distribution over states, is available if (s, a, μ) ∈ δ M . In an execution of the model, the choice between the available transitions in each state is nondeterministic; the choice of successor state is then made randomly according to the distribution μ. For presentational convenience elsewhere in the paper, we do not identify a special "silent" action τ , but this can easily be We denote by π(i) the (i + 1)th state s i of π and by IPaths M (FPaths M ) the set of all infinite (finite) paths in M. If π is finite, |π | denotes its length and last(π ) its last state. The trace, tr(π ), of π is the sequence of actions a 0 a 1 . . . and we use tr(π ) α to indicate the projection of such a trace onto an alphabet α ⊆ α M .
To reason about PAs, we use the notion of adversaries (also called schedulers or strategies), which resolve the nondeterministic choices in a model, based on its execution history.

Definition 2 (Adversary).
An adversary of a PA M = (S, s, α M , δ M , L) is given by a function σ : FPaths M → Dist(α M × Dist(S)) such that, for any finite path π of M, σ (π ) only assigns non-zero probabilities to action-distribution pairs (a, μ) for which (last(π ), a, μ) ∈ δ M . Employing standard techniques [26], an adversary σ of a PA M induces a probability measure Pr σ The set of all adversaries for PA M is denoted by Adv M . Amongst these, we distinguish several important classes.
An adversary σ is deterministic if σ (π ) is a point distribution for all π , and randomised otherwise; σ is memoryless if σ (π ) depends only on last(π ), and finite-memory if there are a finite number of memory configurations such that σ (π ) depends only on last(π ) and the current memory configuration, which is updated (possibly stochastically) when an action is performed. We will also sometimes need to distinguish between partial and complete adversaries, which are discussed in the next section. We augment PAs with rewards, which will be used to capture a variety of quantitative properties of the systems that we model. In this paper, we attach rewards to the transitions of a PA, according to the actions that label them.

Definition 3 (Reward structure).
A reward structure for a PA M is a mapping ρ : α ρ → R >0 from some alphabet α ρ ⊆ α M to the positive reals.
For an infinite path π = s 0 the total reward for π over reward structure ρ is given by ρ(π) def = i∈N∧a i ∈α ρ ρ(a i ).

Parallel composition of PAs
To model and analyse probabilistic systems comprising multiple components, we need parallel composition of PAs. We use the definition of [8,9], which is based on multi-way synchronisation over transitions with identical action labels, in the style of the process algebra CSP.

Definition 4 (Parallel composition).
and only if one of the following holds: . The atomic propositions used by the labelling functions (L 1 , L 2 and L) are assumed to be from some global set AP. In practice, the atomic propositions labelling the states of each individual PA are usually disjoint. Example 1. Fig. 1 shows a pair of PAs, M s and M d , which we will use as one of our running examples throughout the paper. We draw each transition of a PA as a group of arrows, joined by an arc and annotated with its action. We label individual arrows with their corresponding probabilities, but omit this information if the probability is 1. A short incoming arrow denotes the initial state of the PA. We model a system with two components, each corresponding to one of the PAs. Component M s represents a sensor which, upon detection of a system failure, issues instructions to other devices causing them to power down. Upon receipt of the detect signal, it first issues the warn signal followed by shutdown; however, with probability 0.2 it will fail to issue the warn signal. M d represents a device which, given the shutdown signal, powers down correctly if it first receives the warn signal and otherwise only powers down correctly 90% of the time. The combined system, i.e. the parallel composition M s M d of the two PAs, is shown in Fig. 2

(a).
For compositional reasoning about PAs, we also require the notion of projections [8], used to decompose models that have been constructed through parallel composition. First, for any state s = (s 1 , s 2 ) of M 1 M 2 , the projection of s onto M i , denoted by s M i , equals s i . We extend this notation to distributions over the state space S 1 × S 2 of M 1 M 2 in the standard manner. Next, for any (finite or infinite) path π of M 1 M 2 , the projection of π onto M i , denoted π M i , is the path obtained from π by projecting each state of π onto M i and removing all the actions not in α M i together with the subsequent states. Notice that the projection of an infinite path may be finite.
To define projections of adversaries, we first need the notion of partial adversaries, which are functions that map finite paths to sub-distributions over available transitions in the final state of the path, rather than distributions. The interpretation is that such an adversary can opt to (with some probability) take none of the available transitions and remain in the current state. This generalises the normal definition of an adversary, which we will sometimes refer to as a complete adversary.
The probability space Pr σ M for a partial adversary σ is defined in a similar manner as for a complete adversary. We use Compositional reasoning about PAs may require partial adversaries since, even if an adversary of M 1 M 2 is complete, its projection onto one component may be partial. Later, in Section 3.5, we will show that, by restricting our attention to fair adversaries, we can ensure that the projection of a complete adversary remains complete. Definition 6 (Adversary projection). Let M 1 and M 2 be PAs and σ an adversary of M 1 M 2 . The projection of σ onto M i , denoted σ M i , is the (partial) adversary on M i where, for any finite path π i of M i and (last(π i ), a, μ i ) ∈ δ M i : For any partial adversary σ of a PA M, a complete adversary σ of M is called a completion of σ if σ (π )(a, μ) σ (π )(a, μ) for all paths π and action-distribution pairs (a, μ) of M. Any partial adversary always has at least one completion.

Quantitative verification of probabilistic automata
In this section, we describe how to specify and verify a variety of quantitative properties of probabilistic automata. We also discuss multi-objective probabilistic model checking and verification under both partial and fair adversaries.

Specifying properties of PAs
There are various different ways of formally specifying properties of PAs for the purposes of verification. In this paper, we focus primarily on linear-time, action-based properties (i.e those defined in terms of the action labels attached to PA transitions). More precisely, we will use probabilistic ω-regular properties (which subsume, for example, probabilistic LTL) and expected total reward properties. For the former, we will make particular use of the subclass of probabilistic safety properties. The latter, as mentioned earlier, also allows a variety of other reward-based properties to be encoded, including the expected reward to reach a target and time-bounded reward measures; see below for details.
In this section, we introduce the required properties; in the following section, we outline the corresponding techniques to perform model checking. Throughout, we will assume a fixed PA M = (S, s, α M , δ M , L). We begin by defining probabilistic predicates and reward predicates.

Definition 8 (Probabilistic predicate).
A probabilistic predicate [φ] ∼p comprises an ω-regular property 3 φ ⊆ (α φ ) ω over some alphabet α φ ⊆ α M , a relational operator ∼ ∈ {<, , >, } and a rational probability bound p. The probability of satisfying φ under adversary σ is: Definition 9 (Reward predicate). A reward predicate [ρ] ∼r comprises a reward structure ρ : α ρ → R >0 over some alphabet α ρ ⊆ α M , a relational operator ∼ ∈ {<, , >, } and a rational reward bound r. The expected total reward for ρ under adversary σ is: It is worth noting that expected total reward properties (i.e. reward predicates) can also be used to specify the expected total reward accumulated until some set T of target states is reached, which is another commonly used class of properties. This is done by modifying the PA M to ensure that no rewards are accumulated after a state in T is reached for the first time. In the worst case, this requires adding a second copy of each state in the PA. Properties of this kind can, in turn, be used to encode other useful classes of properties, such as the expected amount of reward accumulated over a fixed time period (see [28] for details).
Probabilistic predicates and reward predicates will be collectively referred to as quantitative predicates. We typically use [φ] ∼p and [ρ] ∼r to denote probabilistic and reward predicates, respectively, and [ψ] ∼x for an arbitrary quantitative predicate.
In this work, we will often use a particular class of probabilistic predicates called probabilistic safety properties, defined in terms of regular safety properties.

Definition 10 (Regular safety property).
A regular safety property φ ⊆ (α φ ) ω for PA M is a set of infinite words over alphabet α φ ⊆ α M that is characterised by a regular language φ of bad prefixes. These are finite words, any extension of which is not • "the probability of an error occurring is at most 0.01", • "event A occurs before event B with probability at least 0.98", • "the probability of terminating within k time-units is at least 0.75".
The last of these represents an important class of properties for timed probabilistic systems, perhaps not typically considered as safety properties. Using the digital clocks approach of [29], verifying real-time probabilistic systems can often be reduced to analysis of a PA with time steps encoded as a special action type. Such requirements are then naturally encoded as probabilistic safety properties.

Model checking for PAs
As illustrated in the previous section, we typically verify properties of PA M by quantifying over all possible adversaries of M. For example, for a quantitative predicate [ψ] ∼x , we have: A model checking procedure for verifying whether M satisfies a probabilistic predicate [φ] ∼p reduces to the computation of the minimum or maximum probability of satisfying φ: In the remainder of this section, we discuss how these values can be computed. For the probabilistic case, the problem reduces to the calculation of reachability probabilities. For an atomic proposition q from the set AP used to label states of PA M, we define the probability of reaching a q-labelled state under adversary σ , denoted Pr σ M (3 q), as: and, as for other probabilistic properties, we define: Computation of the values Pr min for M can be achieved by solving a linear programming problem of size |M| [30,1], and thus performed in time polynomial in |M|. In practice, other techniques, such as value iteration or policy iteration, are often used [31].
The probabilities Pr min for an ω-regular property φ can be computed by reducing the problem to calculating reachability probabilities in a product PA composed from the PA M and an ω-automaton for the property φ. In the implementation developed for this work, we follow the approach of [3], which is based on the use of deterministic Rabin automata and then determining the maximum probability of reaching a set of accepting end components. The presentation in [3] uses ω-regular properties over atomic propositions (on states) rather than the action labels (on transitions) used here, but the procedure is essentially the same. Full details for the action-based case can be found in [28]. The total time complexity for computing probabilities is polynomial in the sizes of both the PA M and the deterministic ω-automaton for φ.
We now describe in more detail the computation of probabilities for regular safety properties, since these are used in various places throughout the paper. Although a regular safety property φ is an instance of an ω-regular property, it is more efficient to bypass the use of ω-automata and end component identification, instead computing reachability probabilities directly from the product of the PA M and the error automaton for φ.
where δ is defined such that we have (s, q) a − → μ × η q if and only if one of the following holds:  Using Lemma 1, the (minimum) probability of satisfying regular safety property φ A relates to (maximum) reachability probabilities in the product as follows: and therefore: Example 3. We return to the PA M s from Example 1 (see Fig. 1(a)) and consider a probabilistic safety property [φ A ] 0.8 where φ A is a regular safety property over alphabet α A = {warn, shutdown} with the meaning "warn occurs before Fig. 3(b). States labelled with atomic proposition err A , i.e. those corresponding to the accepting state a 2 of the DFA are marked with grey shading. We have Pr min Finally, we mention the computation required for expected total reward properties, i.e. to calculate ExpTot min ExpTot max M (ρ) for a reward structure ρ. In fact, for this, we can use very similar techniques to those for reachability probabilities, such as value iteration or linear programming. We refer the reader to, for example, [3] for details.

Multi-objective model checking for PAs
In addition to conventional quantitative verification techniques for probabilistic automata, the approach presented in this paper requires the use of multi-objective model checking [14,15]. The conventional approach, described in the previous section, allows us to check whether a single quantitative (probabilistic or reward) predicate holds for all adversaries of a PA (or, dually, for at least one adversary). Multi-objective queries allow us to reason about whether adversaries satisfy multiple properties of this form.

Definition 13 (Qmo-property).
A quantitative multi-objective property (qmo-property) for PA M is a finite conjunction 4 of quan- We first observe that verifying whether a qmo-property Ψ P = [ψ 1 ] ∼ 1 x 1 ∧ · · · ∧ [ψ k ] ∼ k x k is satisfied for all adversaries of a PA, denoted M | Ψ P , reduces simply to k separate checks: However, a typical multi-objective query (often called an achievability query) asks if there exists an adversary of M satisfying all k predicates, i.e. whether: For the case where a qmo-property comprises only probabilistic ω-regular properties, an algorithm for checking the existence of such an adversary is given in [14]. This is based on a reduction to a linear programming (LP) problem, yielding a time complexity polynomial in the sizes of the PA M and the ω-automata representing the k ω-regular properties ψ i . Whenever such an adversary exists, the solution of the LP problem yields a randomised, finite-memory adversary σ of M satisfying all k predicates. In [15], this LP-based approach to multi-objective model checking is extended to include expected total reward properties, i.e. for the class of qmo-properties described in Definition 13. In [33], an alternative approach to model checking qmo-properties is presented, based on value iteration. This has higher time complexity (exponential in the size of the model, in the worst case) but tends to perform better in practice.
Another useful class of multi-objective properties, also treated in [15,33], is that of numerical queries. These yield the minimum or maximum achievable value for the probability of an ω-regular property φ or the expected total value of a reward structure ρ, whilst still satisfying some qmo-property Ψ P . For example, in the maximum case: The techniques for numerical queries in [33] are further extended to the class of Pareto queries, which can be used for a more detailed analysis of the trade-off between two or more properties. For example, given two ω-regular properties φ 1 and φ 2 , we can consider the Pareto curve of points (p 1 , ] p 2 is achievable but any increase in either p 1 or p 2 would necessitate a decrease in the other. We discuss the use of numerical and Pareto queries for compositional verification in Section 6.

Model checking PAs under partial adversaries
In this section, we explain how to perform model checking of PAs over the class of partial (rather than complete) adversaries, which will be needed later in the paper. We cover two cases. Firstly, we show that, for model checking of probabilistic safety properties, these two classes of adversaries are equivalent. Secondly, we show any other kind of property can be handled using a simple transformation of the PA. In both cases, we then have a reduction from model checking of PAs over partial adversaries to the problem of model checking over complete adversaries, as described in the preceding sections.
The following proposition states that satisfaction of probabilistic safety properties is equivalent whether quantifying over complete adversaries or over the larger class of partial adversaries. This is because checking probabilistic safety properties reduces to the computation of maximum reachability probabilities.
Proof. Consider any PA M and probabilistic safety property [φ A ] p A . By Definition 8, we have: and from Lemma 1 there is a bijection f M,A between the adversaries of PA M and the PA-DFA product M ⊗ A err For the ⇒ direction, we assume M | [φ A ] p and consider any partial adversary σ of M. Now, letting σ be a completion of σ , it follows that f M,A (σ ) is a completion of f M,A (σ ) and, by the definition of reachability probabilities, we have: Next, using Lemma 1, we have: Therefore, since σ was an arbitrary partial adversary of M, this half of the proof is complete.
The next proposition shows that satisfaction of qmo-properties in a PA M using partial adversaries is equivalent to satisfaction under complete adversaries after performing a simple transformation of M which adds a probability 1 self-loop to every state. We formalise this transformation as a special case of alphabet extension (see Definition 7) using a fresh action b.  For the direction ⇒, the proof follows by showing that, for any partial adversary σ of M, we can construct a complete adversary σ of M[{b}] which satisfies precisely the same qmo-properties that do not include b in their alphabet. The construction proceeds as follows. For any finite path π ∈ FPaths M [{b}] we have the following two cases to consider.
• If π ∈ FPaths M , then for any action distribution pair (a , μ ) From Definition 7, in the 'otherwise' case above, it follows that π is a path of length 0, that is, a state of M. Now, considering any complete adversary σ of M[{b}], we construct the following partial adversary σ of M. For any finite path π ∈ FPaths M and action distribution pair (a, μ) ∈ Dist(α M × Dist(S)): The remainder of the proof follows from the construction of the probability measures for the adversaries σ and σ . 2

Model checking PAs under fairness
We conclude our discussion of model checking techniques for PAs with the topic of fairness. When verifying systems where multiple PAs are composed in parallel and can execute asynchronously, it is often necessary to impose conditions that ensure the PAs are scheduled in a fair manner. These fairness conditions, which typically correspond to reasonable assumptions about the system being modelled, may be essential in order to verify even trivial properties.
In this paper, we use a simple but effective notion of fairness called unconditional fairness, in which it is required that each component makes a transition infinitely often. For probabilistic automata, a natural approach to incorporating fairness (as taken in, e.g., [34,35]) is to restrict analysis of the system to a class of adversaries in which fair behaviour occurs with probability 1.
If M = M 1 · · · M n is a PA comprising n components, then an (unconditionally) fair path of M is an infinite path π ∈ IPaths M in which, for each component M i , there exists an action a ∈ α M i that appears infinitely often. A fair adversary σ of M is an adversary for which Pr σ M {π ∈ IPaths M | π is fair} = 1. We let Adv fair M denote the set of fair adversaries of M.
An important consequence of our definition of fairness is that the projection of a fair adversary onto its component PAs results in a complete, rather than partial, adversaries. Proof. Consider any PAs M 1 and M 2 and adversary σ ∈ Adv fair M 1 M 2 . Suppose for a contradiction that σ M i is a partial adversary for some i = 1, 2. Since σ M i is partial, by definition there exists some finite path π fin i of M i such that: Now, by Definition 6, it follows that: By the definition of a fair path, we have that the projection of a fair path is always infinite and, combining this fact with the above, yields: which contradicts the fact that σ is a fair adversary as required. Model checking algorithms for verifying PAs under fairness are presented in [35]. These techniques apply to more general notions of strong and weak fairness, of which unconditional fairness is a special case. The algorithms work by restricting the set of end components of the PA under consideration. A related problem is realisability: determining whether a PA has any (unconditionally) fair adversaries. Efficient techniques for this problem, based on an analysis of the underlying graph of a PA, can be found in [36]. We can also pose multi-objective queries, as discussed in the previous section, in the context of fairness, i.e., ask whether there exists a fair adversary satisfying all conjuncts of a qmo-property Ψ P = [ w ∈ (α M ) ω some action a ∈ α M i appears in w infinitely often .
Thus, we can check for a fair adversary satisfying Ψ P by checking for an arbitrary adversary that satisfies Ψ P ∧ [φ fair ] 1 .

Assume-guarantee verification for probabilistic automata
We now present our compositional verification framework for probabilistic automata. The approach is based on the well-known assume-guarantee paradigm and builds on the multi-objective model checking techniques summarised in Section 3.3. In this section, we define the basic underlying ideas and then introduce our two main classes of assume-guarantee proof rules, for safety properties and general quantitative properties. In Section 5, we will consider a number of further assume-guarantee proof rules.

Assume-guarantee triples
The key ingredient of classical assume-guarantee reasoning is the assume-guarantee triple, comprising a component, an assumption and a guarantee. To enable compositional verification of probabilistic systems, we introduce probabilistic assumeguarantee triples. These triples take the form Ψ A M Ψ G , where M is a PA, representing a component of a probabilistic system, and Ψ A and Ψ G are qmo-properties, representing an assumption and a guarantee, respectively. Triples are also parameterised by a class of adversaries for M, denoted by the symbol ∈ {part, comp, fair}, indicating the set of partial, complete or fair adversaries, respectively. The triple Ψ A M Ψ G asserts that any adversary for M (of type ) which satisfies the assumption Ψ A also satisfies the guarantee Ψ G . Formally, we have the following definition. 5 Definition 14 (Probabilistic assume-guarantee triple). Let M be a PA, Ψ A and Ψ G be qmo-properties such that α G ⊆ α A ∪ α M and ∈ {part, comp, fair} be a class of adversaries. Then Ψ A M Ψ G is a probabilistic assume-guarantee triple with the following semantics: Notice that Definition 14 quantifies over adversaries of the alphabet extension M[α A ] of M (see Definition 7), rather than M itself. This is because, when we come to use these triples to formulate assume-guarantee proof rules, Ψ G will be a property of the overall system being verified, of which M is just one component. Property Ψ G may therefore be defined over a superset of the actions in the alphabet α M of M and, as a result, the assumption Ψ A may also include additional actions not in α M .
Because probabilistic assume-guarantee triples are stated in terms of quantification over adversaries, checking whether or not a triple is true can be reduced to a multi-objective model checking problem of the form described in the previous section. More precisely, we ascertain whether Ψ A M Ψ G is not true by checking for the existence of an adversary that satisfies the assumption Ψ A but does not satisfy the property Ψ G .

Proposition 3.
Let PA M, qmo-properties Ψ A , Ψ G and ∈ {part, comp, fair} be as given in Definition 14. Furthermore, let Ψ G be of the form [ψ G ] ∼x G , i.e., a single quantitative predicate. 6 Then, checking whether the assume-guarantee triple Ψ A M Ψ G holds reduces to multi-objective model checking as follows: This can be done using the techniques from [15], described in Section 3.3, with time complexity polynomial in the sizes of the PA M and the (deterministic Rabin) automata representing the probabilistic predicates that occur in Ψ A and Ψ G .

Proof. The result follows directly from Definition 14. 2
Moreover, for the special case of assume guarantee triples restricted to probabilistic safety properties and partial adversaries, checking whether the triple holds reduces to the simpler multi-objective model checking algorithm of [14].
Now, using Definition 14, we have: 5 Our definition of assume-guarantee triple generalises the one we originally presented in [17], where the parameter is omitted. In that work, the set of partial adversaries is always used, i.e. = part. 6 For clarity of presentation, we restrict our attention in Proposition 3 to the case where Ψ G is a single predicate and then later generalise to arbitrary triples in Theorem 3.
rearranging which completes the proof. 2

Assume-guarantee verification for safety properties
We now present, using the definitions above, asymmetric assume-guarantee proof rules for compositional verification of probabilistic automata, that is, those which use only a single assumption about one system component. Experience in the non-probabilistic setting [7] indicates that, despite their simplicity, rules of this form are widely applicable. First, in this section, we discuss an approach in which both assumptions and guarantees are probabilistic safety properties. Then, in Section 4.3, we describe how to handle a wider class of quantitative properties, including ω-regular and reward properties, by incorporating fairness. To simplify the presentation, here and in Section 4.3 we restrict our attention to guarantees consisting of a single predicate, however in Section 4.4 we extend the approach to allow general guarantees.
Our first proof rule, (Asym-Safety), can be stated as follows Theorem 1 means that, given an appropriate assumption Ψ safe A = [φ 1 ] p 1 ∧ · · · ∧ [φ n ] p n , we can check the correctness of a probabilistic safety property [φ G ] p G on a two-component system, M 1 M 2 , without constructing and model checking the full system. Instead, we perform one instance of (standard) model checking on M 1 , to check the first premise of rule (Asym-Safety), and one instance of multi-objective model checking on M 2 [α A ], to check the assume-guarantee triple in the second premise (recall, from Definition 14, that a triple for M 2 is defined in terms of M 2 [α A ]). If the error automata A err φ i for the regular safety properties φ i in the assumption are much smaller than the PA for component M 1 , we can expect significant gains in terms of the overall performance for verification. Our experimental results, described later in Section 7, show that this can indeed be the case in practice.
Before proving Theorem 1, we give an example of its usage.

Example 4.
We illustrate the application of rule (Asym-Safety) using the PAs M s and M d from Example 1 (see Fig. 1 does not hold for any value of p G > 1 − 0.02 = 0.98. 7 This rule is equivalent to the rule (Asym) we originally presented in [17]. In that work, the first premise and conclusion quantify over partial adversaries, rather than all adversaries. However, as Proposition 1 shows, these are equivalent. We now prove Theorem 1. For this, and for the other results in this section, we require the following lemma, adapted from [8]. (a) Pr σ If ρ and ρ i are reward structures over α i and α i , then: If Ψ i and Ψ i are qmo-properties over α i and α i , then: Since σ was an arbitrary adversary of M 1 M 2 , it follows that M 1 M 2 | Ψ safe G , as required. 2

Assume-guarantee verification for quantitative properties
Next, we extend our assume-guarantee framework with a proof rule in which the properties to be proved and the assumptions used to do so can comprise arbitrary quantitative predicates, rather than just probabilistic safety properties.
More precisely, we use a qmo-property Ψ A for an assumption and a single quantitative predicate Ψ G = [ψ G ] ∼ G x G for the property to be proved. Recall that quantitative predicates include probabilistic ω-regular properties (which subsume both probabilistic safety properties and probabilistic LTL) and expected total reward properties.
To As in the previous section, we initially restrict our attention to an asymmetric assume-guarantee proof rule. Formally, we state this as follows.
Then the following proof rule holds: Theorem 2 provides the means to verify an ω-regular or reward property Ψ G on a composed system M 1 M 2 . Like (Asym-Safety) in Theorem 1, we can do this without constructing M 1 M 2 by decomposing into two-sub problems: one instance of normal verification (premise 1) and one instance of multi-objective model checking (premise 2). Again, these two steps have the potential to be significantly cheaper than verifying the combined system M 1 M 2 .
The inclusion of fairness in the two premises of (Asym-Quant) is to permit recursive applications of the rule, in order to compositionally verify systems comprising more than two components. If M 1 is just a single PA, the stronger (but easier  PA M c models a specific controller for the machine, which instructs it which way to execute jobs, when composed in parallel with M m . With probability 0.5, the controller sends an initial instruction slow. Subsequently, it only sends the instruction fast.
Our aim is to verify that the expected total execution time for the controlled machine is at most 19 6 . Since PAs do not , which we see to be true from inspection. Thus, we can conclude that . Proof (of Theorem 2). Suppose M 1 and M 2 are PAs, Ψ A is a qmo-property and Ψ G = [ψ G ] ∼ G x G is a single quantitative predicate, such that both M 1 | fair Ψ A and Ψ A M 2 fair Ψ G hold. Consider an arbitrary fair adversary σ of M 1 M 2 . Now, from Lemma 2, we know that σ M 1 is a complete adversary of M 1 . Furthermore, we have that M 1 , σ M 1 is also fair, and hence by definition of | fair : Since σ was an arbitrary fair adversary of M 1 M 2 , we have M 1 M 2 | fair Ψ G , as required. 2

Extensions
Next, we discuss two ways in which the proof rules presented so far can be extended. Subsequently, in Section 5, we will present several additional rules.

Conjunctions of predicates.
Firstly, we remark that it is straightforward to extend our approach to verify properties that are conjunctions of predicates, rather than single predicates. For presentational simplicity, we so far assumed (in Propositions 3 and 4, and Theorems 1 and 2) that the qmo-property Ψ G used on the right hand side of a triple Ψ A M Ψ G comprised a single predicate. However, checking a triple in which Ψ G is a conjunction of k predicates can be decomposed into k separate checks, i.e, k applications of Proposition 3 or Proposition 4.
Proof. The result follows directly from the definition of assume-guarantee triples (see Definition 14). 2 Theorem 3 provides a way to check assume-guarantee triples containing arbitrary qmo-properties. Furthermore, observe that Theorems 1 and 2 can easily be generalised to the case where the property Ψ safe G or Ψ G being proved on the system M 1 M 2 comprises more than one predicate, thus yielding proof rules for arbitrary qmo-properties.
Notice that, in Theorem 3, unlike the previous theorems, we omitted explicit statements about the restrictions imposed on the alphabets of the PAs and properties. There, and from this point on, we will implicitly assume that, if a rule contains an occurrence of the triple Ψ A M Ψ G , then α G ⊆ α A ∪ α M ; similarly, for a premise that checks Ψ A against M, we assume that α A ⊆ α M .
Multiple components. Secondly, we observe that, simply through repeated application of either (Asym-Safety) or (Asym-Quant), we obtain proof rules for systems consisting of n components.
Proof. The result follows by iteratively applying rules (Asym-Safety) and (Conj-Safety) or (Asym-Quant) and (Conj-Quant), respectively, to the parallel composition of M 1 · · · M i−1 and M i , for i = 2, . . . ,n. 2 Elsewhere [20], we have already successfully applied rule (Asym-Safety-N), showing that it can be significantly more efficient to use an N-way, rather than 2-way, decomposition for assume-guarantee verification. It also worth noting at this point that the parallel composition operator for PAs is both associative and commutative, which allows for flexibility in the way that multi-component rules are applied, and the way that several separate proof rules are combined.

Further proof rules
In this section, we consider three additional classes of assume-guarantee proof rules for compositional verification of probabilistic automata.

Circular proof rules
One potential limitation of the various proof rules considered so far is that they are all asymmetric, i.e., they may analyse, for example, a component M 2 using an assumption Ψ A about another component M 1 , but checking whether M 1 satisfies Ψ A cannot make any assumptions about M 2 . Below, we give proof rules that we call circular, which do allow the use of additional assumptions in this way.
Proof. We give the proof of (Circ-Quant); the proof for (Circ-Safety) follows similarly using Proposition 1 as opposed to Lemma 2 (see the proof of Theorem 1).
Suppose M 1 and M 2 are PAs, Ψ A 1 , Ψ A 2 and Ψ G are qmo-properties, such that M 2 | fair Ψ A 2 , Ψ A 2 M 1 fair Ψ A 1 and Ψ A 1 M 2 fair Ψ G hold. Now, consider an arbitrary fair adversary σ of M 1 M 2 . Since M 2 | fair Ψ A 2 , using Lemma 2, we have M 2 , σ M 2 | Ψ A 2 and therefore, since α A 2 ⊆ α M 2 and applying Lemma 3(e), we have: Therefore, since σ was an arbitrary fair adversary of M 1 M 2 , we have M 1 M 2 | fair Ψ G as required. 2

Asynchronous proof rules
Our next class of rules is motivated by the fact that, often, part of a system comprises several asynchronous components, that is, components with disjoint alphabets. In such cases, it can be difficult to establish useful probability bounds on the combined system if the fact that the components act independently is ignored. For example, consider the case of n independent coin flips; in isolation, we have that the probability of each coin not returning a tail is 1/2. Ignoring the independence of the coins, all we can say is that the probability of any of them not returning a tail is at least 1/2. However, using their independence, we have that this probability is at least 1 − 1/2 n . In the context of our assume-guarantee proof rules, we can reason about systems with asynchronous components as follows.
, the components are asynchronous), then the following proof rule holds: Note that the union of two regular safety properties is also a regular safety property.
In addition, if ∼ ∈ {<, , , >} and, for i = 1, 2, we have that Ψ A i is a qmo-property and = ∅, then the following proof rule holds: Note that ω-regular properties are closed under union (and also intersection and complementation).

Decomposition of reward-based properties
The third class of proof rules that we present in this section concerns the verification of reward properties. We show that, to prove properties of reward structures over a complete system, we can divide the reward structure into sub-structures over the system's components, prove properties for each sub-structure and then combine the results to prove properties of the original reward structure over the complete system.
Using this definition we have the following result.
Proof. Suppose M 1 and M 2 are PAs, ∼ ∈ {<, , , >} and, for i = 1, 2, we have that Ψ A i is a qmo-property and Although the rule (Sum-Reward) is defined in terms of assume-guarantee triples, it also allows us to prove the satisfaction of reward properties. In particular, assuming α ρ i ⊆ α M i for i = 1, 2, then as a special case of the rule we have:

Numerical assume-guarantee queries
Practical experience with probabilistic verification suggests that it is often more instructive to analyse models using numerical, rather than Boolean, queries. For example, instead of checking the correctness of a (lower-bounded) probabilistic predicate [φ] p against a PA M for some bound p, it may be preferable to just directly compute the minimum probability In this section, we discuss how to formulate such numerical queries in the context of assume-guarantee reasoning.
Consider, first, an assume-guarantee triple which can be determined with essentially the same procedure that checks whether assume-guarantee triple Ψ A M Ψ G is true for a fixed p G (i.e., as described in Proposition 3).
We call expressions such as the one above numerical assume-guarantee queries. The example above is equivalent to the maximum value of p G for which Ψ A M Ψ G holds: Optimal bounds. In the rest of this section, we will illustrate how to use numerical assume-guarantee queries, such as the one above, in the context of our compositional proof rules. For simplicity, we will focus on proof rule (Asym-Safety) (see Theorem 1) An appropriate numerical interpretation of this rule would therefore be to instead determine the maximal lower bound p G , say p G , for which [φ G ] p G can be shown to hold using the (Asym-Safety) rule (assuming the use of property φ A in the assumption). This can be achieved as follows. First, we note that, from Definition 14, it is clear that the highest value of p G for which Ψ A M 2 Ψ G holds will be obtained by using the maximum possible value of p A , which we will call p A . For rule (Asym-Safety) to be applicable, p A is equal to Pr min M 1 (φ A ), since any higher value of p A will result in the first premise failing to hold. Now, to find p G , we use a numerical assume-guarantee query that computes the maximum value of p G for which Ψ A M 2 Ψ G holds, assuming that the bound p A in Ψ A is taken to be p A . Reasoning as above, this can be obtained through the multi-objective numerical query Pr min In summary, we have a numerical interpretation of the rule (Asym-Safety) which reduces to a two-step procedure, with one instance of (standard) model checking and one of multi-objective model checking: Through similar reasoning, we can determine either maximal lower bounds, or minimal upper bounds, for many of the other rules presented in this paper, where the property being proved is a single quantitative predicate. any further would necessitate also increasing p A to ensure that Ψ A M Ψ G is still true and, conversely, decreasing p A would require a lower value of p G . In a similar fashion to the numerical query used to compute p A in the section above, we actually use a query that searches for adversaries that make the assume-guarantee triple false, namely those for which not true. More precisely, we use a Pareto query which maximises the probability Pr σ verification techniques. Currently, all assumptions are constructed manually, based on user knowledge of the models, their structure and the properties that are being checked. An important direction of future work in this area is the automatic generation of such counterexamples. Some progress has already been made on this topic, using algorithmic learning to generate probabilistic safety properties to be used as assumptions [19,20]. The next two sections describe in more detail how we verified the two case studies compositionally; in the following section, we summarise the experimental results.

Aspnes and Herlihy's randomised consensus algorithm
The first case study we consider is the randomised consensus algorithm of Aspnes and Herlihy [39]. The algorithm allows N processes in a distributed network to reach a consensus and employs, in each round, a shared coin protocol parameterised by a constant K and the number of processes N. Each shared coin protocol is based on a random walk where the boundaries of the walk are derived from the values of K and N.
The model of the algorithm used here is based on [41] and comprises one PA for each process and one for the shared coin protocol used in each round. Although the number of rounds of the protocol can be unbounded, the properties that we check here relate only to the behaviour within a finite number of rounds R. The combined PA is: where P i is the PA corresponding to process i, SCP j is the PA corresponding to the shared coin protocol used in round j and we write ||| to denote parallel composition between PAs with disjoint alphabets. Although each shared chain protocol SCP j does interact with the N processes, there is no communication between the protocols themselves, i.e., the shared coin protocols for each round are independent. This lets us use the asynchronous proof rules introduced in Section 5.2.
We analyse the following two properties: (i) the minimum probability that the processes decide by round R; (ii) the maximum expected number of steps required in the first R rounds.
We adopt the numerical approach described in Section 6, computing lower/upper bounds on these values, rather than proving that they are above or below a given threshold. Property (i) can be represented by a probabilistic safety property. We use the rule (Asym-Safety), in conjunction with the rule (Async-Safety), to establish the required assumptions. More precisely, compositional verification is achieved by: • calculating the minimum probability that the coin protocols in earlier rounds return the same coin value for all processes, and then combining these results using rule (Async-Safety) to prove a probabilistic safety property satisfied by the (asynchronous) composition of the coin protocols; • using this probabilistic safety property as the assumption for an application of the (Asym-Safety) rule, yielding the final property of interest on the combined system, namely, the minimum probability that agreement is reached by round R.
Compositional verification of property (ii) is performed by: • splitting the reward structure for the property into several parts (one for the N processes and one for each coin protocol) and then using rule (Async-Quant) to determine an upper bound on the maximum total expected value for each one; • combining these results, using the (Sum-Reward) rule, to compute an upper bound on the maximum total reward.

The Zeroconf protocol
The second case study is the Zeroconf protocol [42], for configuring IP addresses in a local network. We use the model from [29], composed of two PAs: one representing a new host joining the network and the second representing the environment, i.e. the existing network. The model is parameterised by K , the number of messages ("probes") that the new host sends before using its chosen IP address. We consider the following four properties: (i) the minimum probability that the new host employs a fresh IP address; (ii) the minimum probability that the new host is configured by time T ; (iii) the minimum probability that the protocol terminates; (iv) the minimum and maximum expected time for the protocol to terminate. Properties (i) and (ii) are probabilistic safety properties, which are verified compositionally by: • applying the rule (Circ-Safety), where the first and second assumptions concern the new host and environment, respectively, and the first assumption is a qualitative (probability 1) safety property. For properties (iii) and (iv), verification is performed by: • applying the rule (Circ-Quant), where the first assumption is a either a qualitative safety or qualitative liveness property, about the new host, and the second assumption is a probabilistic ω-regular property, concerning the environment.

Experimental results
In Tables 1 and 2, we summarise the experimental results for the randomised consensus and Zeroconf case studies, respectively. Experiments were run on a 2.8 GHz PC with 8 GB RAM. Any run exceeding a time-limit of 6 hours was disregarded. The tables show the time taken for verification, performed both compositionally and non-compositionally. For the former, we proceed as described as above, using PRISM to check each individual model checking problem. For the latter, we also use PRISM, selecting its fastest available model checking engine. To give an indication of the scale of the verification tasks performed, the tables also show, for the non-compositional case, the size (number of states) of the PA for the composed system and, for the compositional case, the size (number of variables) of the largest LP problem solved for multi-objective model checking. Finally, the table includes the (numerical) results obtained from verification. For probabilistic safety properties, we show the maximum probabilities of violating the property, so the actual values are these subtracted from 1.
One the whole, compositional verification performs very well. For the randomised consensus models (Table 1), on all but the smallest examples, it is faster than the non-compositional case, often significantly so, and is able to scale up to larger models. In particular, this allows the verification of several models for which it is infeasible with conventional techniques (those marked with "mem-out" in the table). For the Zeroconf example (Table 2), we again see improved times for the compositional approach on the first property. For the other properties, the times for the two approaches are closer, but non-compositional verification is slightly faster. Encouragingly, the times for compositional verification tend to grow more slowly with model size. Since the time required for this is usually dominated by the time to solve LP problems, we anticipate better performance through enhancements to the underlying LP solver and optimisations for our implementation that will reduce LP problem sizes. In fact, LP solution represents the limiting factor with respect to the sizes of models that our techniques can be applied to, so such improvements will also help improve the scalability of compositional verification. Lastly we also comment on the results obtained from assume-guarantee verification. As mentioned earlier, we use the approach discussed in Section 6 to obtain numerical values for each property of interest, representing a lower or upper bound on the actual value of the property. We observe that the bounds obtained using our assume-guarantee approach are generally quite precise. In fact, for several properties, the bounds are tight, matching the actual values exactly.

Conclusions
We have presented new techniques for compositional verification of probabilistic automata, based on the assumeguarantee paradigm. Our techniques can be used to verify probabilistic ω-regular properties (including the special case of probabilistic safety properties) and expected total reward properties. The key novelty of our approach is the use of multiobjective model checking, for which efficient techniques exist. We have presented a variety of assume-guarantee proof rules and also discussed how to formulate and evaluate numerical verification queries in a compositional manner. In contrast to existing work in this area, our techniques can be implemented efficiently and we demonstrate successful results on several large case studies.
There are several interesting directions for future work. For the fragment of our framework that uses probabilistic safety properties, algorithmic learning techniques have already been developed to automatically produce the assumptions required for compositional reasoning [19,20]. This work adapts the L* algorithm for learning regular languages to the problem of synthesising assumptions expressed as probabilistic safety properties. It would be interesting to extend these methods to the more general class of properties considered in this paper, perhaps using techniques for learning of ω-automata [43,44].
Another topic to investigate is that of completeness. Some assume-guarantee frameworks are complete in the sense that, if a property of a composed system is true, then there must exist an assumption that can be used to verify it compositionally. For example, this is trivially true for frameworks in which assumptions are expressed in the same manner as the models themselves: then, completeness can be shown by using the component itself as the assumption that represents it. In our approach, the formalisms for components (probabilistic automata) and assumptions (qmo-properties) are distinct, so this argument is not applicable. We would like to investigate for which classes of models and properties our framework can be shown to be complete.
Finally, we would also like to consider assume-guarantee techniques for richer classes of models such as probabilistic timed automata and continuous-time variants of probabilistic automata, such as interactive Markov chains [45] or Markov automata [46]. In order to adapt the multi-objective model checking approach used in this paper, the first step will be to develop efficient multi-objective techniques for timed properties of such models.