Local abstraction reﬁnement for probabilistic timed programs

We consider models of programs that incorporate probability, dense real-time and data. We present a new abstraction reﬁnement method for computing minimum and maximum reachability probabilities for such models. Our approach uses strictly local reﬁnement steps to reduce both the size of abstractions generated and the complexity of operations needed, in comparison to previous approaches of this kind. We implement the techniques and evaluate them on a selection of large case studies, including some inﬁnite-state probabilistic real-time models, demonstrating improvements over existing tools in several cases. © 2013


Introduction
Abstraction refinement is a highly successful approach to the verification of complex infinite-state systems.The basic idea is to construct a sequence of increasingly precise abstractions of the system to be verified, with each abstraction typically over-approximating its behaviour.Successive abstractions are constructed through a process of refinement which terminates once the abstraction is precise enough to verify the desired property of the system under analysis.Abstraction refinement techniques have also been used to verify probabilistic systems [6,11,13,7], including those with real-time characteristics [16,17,8] and continuous variables [25].Frequently, though, practical implementations of these techniques are hindered by the high complexity of both the abstractions involved and the operations needed to construct and refine them.
In this paper, we target the verification of programs whose behaviour incorporates both probabilistic and real-time aspects, and which include the manipulation of (potentially infinite) data variables.We analyse systems modelled as probabilistic timed programs (PTPs) [17], whose semantics are defined as infinite-state Markov decision processes (MDPs).We introduce an abstraction refinement procedure for computing minimum and maximum reachability probabilities in PTPs.As in [6,11], we use an MDP-based abstraction.This provides outer bounds on reachability probabilities (i.e., a lower bound on the minimum probability or an upper bound on the maximum).In addition, we compute dual, inner bounds, based on a stepwise concretisation of adversaries of this abstract MDP, yielding upper and lower bounds on minimum and maximum probabilities, respectively.Concretisation is also used, for example, in [11], for untimed models.The key difference in our work is that we aim to keep the abstraction small by using local refinement and simplification operations, so as to reduce the need for expensive operations such as Craig interpolation.
At the core of our approach is a refinement loop that repeatedly attempts to construct a concrete adversary of the PTP.This is based on the exploration of the part of the state space on which the current abstract adversary can be concretised.In each exploration step, we may encounter an inconsistency, in which case we derive a refinement operation and restart.Otherwise, we numerically solve the constructed adversary, giving inner bounds on the desired probability values.The refinement loop terminates once the difference between upper and lower bounds is smaller than a specified threshold ε.
We implement our abstraction refinement approach, deploy it on various large case studies, and compare to the probabilistic verification tools PRISM [18], PASS [9] and FORTUNA [3], illustrating improved performance in many cases.We are also able to verify probabilistic timed programs containing both real-time behaviour and infinite data variables, which these tools cannot handle.

Related work
Abstraction refinement for MDPs and related models is an active research field.In [6], techniques were proposed for abstracting MDPs using the notion of probabilistic simulation.Building on the same approach to abstraction, [11] developed a probabilistic version of the classic counterexample-guided abstraction refinement (CEGAR) method, which was then implemented in the tool PASS [9].This verifies a probability-bounded reachability property using a refinement scheme based on probabilistic counterexamples and Craig interpolation.In contrast to the implementation of [6], PASS uses predicate abstraction, allowing it to analyse infinite-state models.More recent work [15] proposes an alternative probabilistic CEGAR technique using stochastic tree counterexamples; this applies to finite-state MDPs, on which properties are specified using simulation rather than reachability.However, all three methods [6,11,15] were applied to discrete-time models (MDPs), whereas our approach generalises to models with real-time behaviour.We provide an experimental comparison with PASS, for MDP models, in Section 4.
In [13], a quantitative abstraction refinement technique for MDPs was proposed, using a different form of abstraction based on stochastic games.This computes lower and upper bounds for reachability probabilities, the difference between which determines if further refinement is needed.The framework of [13] was subsequently applied to verification of C programs with probabilistic behaviour [12].Later extensions to PASS also use game-based abstraction refinement [24].In recent work [7], the abstraction frameworks of [13,24] were adapted to handle arbitrary abstract domains, illustrating cases where this can outperform predicate abstraction.As for [6,11,15] above, though, these methods [13,12,24,7] all focus on models with a discrete notion of time.
Probabilistic timed automata (PTAs) are a subclass of the probabilistic timed programs (PTPs) that we target in this paper, since only the latter allows arbitrary (infinite) data variables.For PTAs, several verification techniques exist.Most relevant here is [16], which extends the abstraction refinement framework of [13] mentioned above, to PTAs, by using zones to represent abstract states.Other possibilities include the digital clocks discretisation [19] and backwards reachability [21].The probabilistic model checker PRISM [18] supports verification of PTAs, using either [16] or [19].In [16], abstraction refinement was shown to outperform the other available techniques.Subsequently, an optimised version of backwards reachability, implemented in the tool FORTUNA [3], was shown to exhibit superior performance on various examples.We compare the performance of our approach to both PRISM and FORTUNA in Section 4.
Several PTA verification tools do support PTAs with data variables, but they are required to be finite.This includes PRISM, discussed above, mcpta [10], which translates the modelling language Modest to PRISM using [19], and UPPAAL PRO, which computes maximum reachability probabilities for PTAs by progressively partitioning the state space into sets of zones.The closest approaches to the one presented here are [17] and [8].In [17], an extension of the game-based abstraction refinement framework of [13] is defined for PTPs, but not implemented.This defines abstractions as stochastic games, rather than MDPs as in our approach.In recent work [8], PTPs (there called variable-decorated PTAs) are verified using a combination of discretisation via digital clocks [19] and predicate abstraction methods from PASS [24].Our approach avoids the use of discretisation by using zones and aims to improve efficiency by using local refinement and simplification operations to reduce the size of abstractions.The implementation of [8] is not currently available; we give a brief, indirect comparison of results in Section 4.

Preliminaries
We assume a set V of variables, ranging over a domain D defined by a theory T (linear integer arithmetic in our examples).We require satisfiability of quantifier-free formulae in T to be decidable.The set of assertions over V, i.e., (conjunctive) quantifier-free formulae in T , is denoted by Asrt(V), and Val(V) is the set of valuations of V, i.e., functions u : V → D. We use Assn(V) for the set of assignments over V, given by a term r x for each x ∈ V.The result of applying assignment f to a valuation u is f (u), given for each x ∈ V by f (u)(x) = u(r x ).Given an assignment f and an assertion ϕ, the composition For a set S, we use P(S) to denote the set of subsets of S and D(S) for the set of discrete probability distributions over S, i.e. finite-support functions : S → [0, 1] such that s∈S (s) = 1.A distribution ∈ D(S) with support {s 1 , . . ., s n } and (s j ) = λ j will also be written λ 1 s 1 + • • • + λ n s n .

Clocks
We use a set X of clock variables to represent the time elapsed since the occurrence of various events.The set of clock valuations is R X 0 = {v : X → R 0 }.For any clock valuation v and δ ∈ R 0 , the delayed valuation v + δ is defined by The valuation 0 has all clocks set to 0.
A clock difference constraint over X is an upper or lower bound on either a clock or the difference between two clocks.It is convenient to extend X with a dedicated zero clock x 0 which is always 0, so that all clock difference constraints have the form x − y b with x, y ∈ X 0 := X ∪ {x 0 }, ∈ {<, } and b ∈ Z ∪ {±∞}.We define the complement c of a clock difference constraint c as: A (convex) zone is the set of clock valuations satisfying a number of clock difference constraints, and the set of all zones is Zones(X ).We use several standard operations on zones: • future: the set of clock valuations reachable from ρ by letting time pass; • past: We will use one additional operation on clocks, for which we first require some standard operations on pairs (b, ) ∈ R 0 × {<, } (see [2]).The set {<, } is ordered by < , and the set of pairs (b, ) by the lexicographic combination ≺ of < R and , i.e. (b (ii) the conjunction of all constraints in C is unsatisfiable, and (iii) C is minimal in the sense that no proper subset C C satisfies (ii).
An unsatisfiable core always exists, since the zones are given by finite sets of constraints whose union U is unsatisfiable (otherwise the zones would have a non-empty intersection), and the subsets of U satisfy the descending chain condition.We can compute an unsatisfiable core with the following straightforward generalisation of the Craig interpolation procedure for difference logic from [5].
For all i, j, let (b ij , ij ) be the least pair for which c ij ≡ x i − x j ij b ij is implied by some ρ k .Note that the conjunction of the c ij is unsatisfiable.This means that, if we label the edges of the complete directed graph on X 0 with the pairs (b ij , ij ), then there is a cycle with a negative label sum [2]; using the Floyd-Warshall algorithm, we can find a shortest such cycle (i 1 , . . ., i k ).
The constraints along this cycle are x i j−1 − x i j i j−1 i j b i j−1 i j for j = 1, . . .,k, where i 0 = i k ; the label sum (b, ) being negative means that summing the constraints results in an unsatisfiable implied constraint 0 b.On the other hand, removing one of the constraints (w.l.o.g., we can assume it is the first one) gives a path whose associated constraints are satisfied by any clock valuation t of the form v(x i j ) = a + b i 1 i 2 + • • • + b i j−1 i j for some large enough a.So the constraints along the cycle form an unsatisfiable core.

Markov decision processes (MDPs)
The underlying semantics for the models studied in this paper is defined in terms of Markov decision processes (MDPs),

Probabilistic timed programs (PTPs)
The systems we will verify are probabilistic timed programs (PTPs) [17], which can be thought of as MDPs extended with state variables and real-valued clocks (or as probabilistic timed automata with state variables).For simplicity, we assume that a PTP contains an initial location which must be left immediately and never re-entered, and an error location which cannot be left.We also make the common assumption that models are structurally non-Zeno [23].

Definition 2 (PTP).
A PTP is a tuple P = (L, V, X , l i , u i , l e , I, T ) where: • L is a finite set of locations and l i , l e ∈ L are initial and error locations; • V is a finite set of state variables and u i ∈ Val(V) is the initial valuation; • X is a finite set of clocks and I : L → Zones(X ) is the invariant condition, where we assume I(l i ) = {0}; • T : L → P(Trans(L, V, X )) is the probabilistic transition function, where Trans(L, V, X The set of all states is denoted Q c (P ), or Q c if P is clear from the context.The initial state is q i = (l i , u i , 0), and the set of error states is Q e = {(l e , u, v) | v ∈ I(l e )}.A step of the PTP from state (l, u, v) consists of some delay δ 0 followed by a transition (G, E, ) ∈ T (l).The transition comprises a guard G, enabling condition E and probability distribution = λ triples containing an update f j ∈ Assn(V), clock resets r j ⊆ X and target location l j ∈ L.
The delay δ must be chosen such that the invariant I(l) remains continuously satisfied; since I(l) is a (convex) zone, this is equivalent to requiring that both v and v + δ satisfy I(l).The chosen transition must be enabled, i.e., the guard G and the enabling condition E must be satisfied by u and v + δ, respectively.An assignment, set of clocks to reset and successor location are then selected at random, according to the distribution .
Our focus in this paper is determining the minimum or maximum probability of reaching an error state in PTP P , denoted p min P and p max P , respectively.These are defined as the values p min JP K and p max JP K for its MDP semantics JP K. which differ by at most ε.

Abstraction refinement for PTPs
We now introduce our abstraction refinement approach for PTPs, which we call local abstraction refinement.Our abstractions are based on MDPs and yield both lower and upper bounds on the desired probability.The outer bound (p lb,min P or p ub,max P ) is obtained from an adversary σ a on the abstract MDP (which overapproximates the choices available to concrete adversaries), while the inner bound (p

ub,min P or p lb,max P
) is based on a partial concretisation of σ a .
In the next section, we describe the basic ideas underlying our abstractions; in subsequent sections, we describe in more detail how to generate suitably precise abstractions using a refinement loop.Throughout, we will assume a fixed PTP

MDP abstractions
The abstraction for a PTP P in our approach is an MDP, which, for convenience, we augment with concretisation information.

Definition 3 (MDP abstraction).
For a probabilistic timed program P , an abstract state (or node) is a triple n = (l, ϕ, ρ) ∈ L ×Asrt(V)×Zones(X ).We use Q a (P ) to denote the set of all abstract states.An MDP abstraction is a tuple A = (N, n i , N e , T ), where: • N ⊆ Q a (P ) is a finite set of nodes, n i ∈ N is the initial node, and N e ⊆ N the set of error nodes; The set of dead nodes from which N e is not reachable is denoted by N d .
In order to formalise the relationship between an MDP abstraction A and its corresponding PTP P , we introduce the notion of reflections.Recall from the definition of JP K that any adversary σ is induced by functions δ σ : A reflection captures the idea that every concrete adversary can be simulated in an abstraction; if, in addition, every transition in the abstraction represents a special case of a concrete transition, we call this abstraction sound.

Definition 4 (Reflection).
Let A = (N, n i , N e , T ) be an MDP abstraction for P .A reflection of adversaries for A is a map ∇ : Q + c × R 0 → N with the following properties: • for any adversary σ of JP K, let ∇ σ : Q + c → N be the reflection of σ , defined as ∇ σ (w) = ∇(w, δ σ (w)).
Then for any path w The need to have a delay as an extra argument arises from the behaviour of refinement with respect to clock constraints (see Section 3.4).The delay is effectively a prophecy variable [1] representing the next decision of the adversary.This dependency means, in particular, that we do not have a straightforward simulation relation between concrete and abstract states; the reflection allows us, however, to construct a simulation for each concrete adversary (see Theorem 1).

Definition 6 (Sound abstraction
).An MDP abstraction A for PTP P is sound if it has a reflection of its adversaries and concretisable transitions.
All our abstractions will be sound and have the following additional property.

Definition 7 (Tight abstraction). An MDP abstraction
where n = (l, ϕ, ρ) and n j = (l j , ϕ j , ρ j ) for all j, we have E ⊆ ρ, and G, E ensure validity of all successors, i.e.G ⇒ ϕ j • f j and E ⊆ [r j := 0]ρ j for all j.
The significance of these properties is as follows.Soundness allows us to obtain correct bounds, while tightness ensures progress.Specifically, a reflection of adversaries ensures that the reachability probabilities in the MDP underlying A yield correct outer bounds p lb,min P := p min A and p ub,max P := p max A (see Theorem 1) while concretisability of transitions means that we obtain correct inner bounds by partially concretising an abstract adversary (see Theorem 3).Tightness ensures that each refinement step is a proper refinement, in the sense that each node obtained by splitting a node n either satisfies stronger constraints than n or has stronger enabledness conditions on its outgoing transitions (see Theorems 4 and 5).Proof.Let A = (N, n i , n e , T ) be an MDP abstraction for P with a reflection of adversaries ∇ :

Theorem 1. If A is a sound MDP abstraction for PTP P , then p min
) with the singleton transition sets all w.This is essentially the DTMC induced by JP K and σ , and satisfies p min . By the assumption that l i cannot be revisited, all reachable states of M lie in To build abstractions for PTPs, we start with an initial abstraction, which is defined as follows.
Definition 8 (Initial abstraction).For a location l of PTP P , let n l denote the abstract state (l, true, I(l)).The initial abstraction for P is the MDP abstraction A i (P ) := ({n l | l ∈ L}, n l i , {n l e }, T ) where: Theorem 2. The initial abstraction A i (P ) for PTP P is sound and tight.
Proof.Let A i (P ) = ({n l | l ∈ L}, n l i , {n l e }, T ).We define ∇ : Clearly, ∇ maps (wq, δ) to n l i iff l = l i , and to n l e iff q ∈ Q e .Let σ be an adversary of JP K induced by the delays δ σ (q) and transitions τ σ (q) as defined in Definition 4.Then, for each w ∈ Q * c and q = (l, v, t), ∇ σ (wq) = n l and, by the definition of A i (P ), T (n l ) contains the abstract transition α l (τ σ (wq Altogether, this means that ∇ is a reflection of adversaries and, since concretisability of transitions is ensured by the obvious mapping of α l (τ ) to τ , the initial abstraction A i (P ) is sound.Tightness is achieved by using the strengthened enabledness condition E in α l (τ ) (since all state assertions are true, G does not need to be changed).2 To extract the complementary (inner) bound p ub,min P or p lb,max P from an MDP abstraction A, we will need the notion of a (memoryless) abstract adversary, which selects an outgoing transition from each node n of A. In practice, we obtain such an adversary when computing the extremal reachability probabilities p min A,n or p max A,n for each node n in A.

Definition 9 (Abstract adversary
).An abstract adversary for an MDP abstraction Example 2. Fig. 1 (right) shows the initial MDP abstraction A i (P ) for PTP P from Example 1 (Fig. 1, left), for which we want to determine the maximum error probability.The top of each box shows the abstract state (node); underneath is (to the left) a node id and (to the right) the computed maximum probability of reaching error node n e .A corresponding adversary σ a is indicated in bold.
An abstract adversary resolves the nondeterminism in the MDP, giving a discrete-time Markov chain which will allow us to compute p ub,min P or p lb,max P .More precisely, we build a (partial) concretisation, based on a forward exploration through the model.We explore the Markov chain induced by σ a using discrete states s = (n, u) consisting of a node n = (l, ϕ, ρ) of A and a valuation u with u ϕ.Interleaved with these forward exploration steps, we perform backwards propagation of time constraints, starting with the invariants of the newly expanded successor states, which are then iteratively strengthened.We formalise this as follows.
Definition 10 (Concretisation).A (partial) concretisation of an abstract adversary σ a is a tuple C = (S, O , E), consisting of: open states whose successors still have to be explored; • a set E of edges e = (s, λ, r, s ), representing edges of the adversary transitions, with the associated probability λ and resets r.For a state s = (n, u), E s ⊆ E is the set of edges with source s.We have: - We denote by S = {s ∈ S | E s = ∅} the set of expanded states in C .
• Let the zones η(s), η(e) for states s and edges e be the greatest solutions to the fixpoint equations: -for all s = (n, u) The idea of this construction is that a partial concretisation whose associated time constraints are satisfiable represents a portion of the model on which the abstract adversary's chosen transitions can be consistently followed by a concrete adversary; π 0 (n i , u i ) and π 1 (n i , u i ) then give a lower bound and an upper bound on p σ P for any adversary σ doing so.We can therefore use the probability bounds π b from the concretisation of a maximising or minimising abstract adversary to determine inner bounds, i.e., we take p lb,max P In order to formalise this, we first define some auxiliary sets capturing the relationship between concrete and discrete states.For s = (n, u) ∈ S, with n = (l, ϕ, ρ), we let We then say that a concrete adversary σ for P follows the partial concretisation C = (S, O , E) if there is a map {(s(w), λ j , r j , s(wq j )) | 1 j k} for some r 1 , . . ., r k ⊆ X .
Theorem 3. Let C = (S, O , E) be a partial concretisation for adversary σ a of A, such that all zones η(s) for s ∈ S and η(e) for e ∈ E are non-empty.Then: 1. there exists a concrete adversary σ following C ; 2. for any such σ , we have π 0 with w m = (l, u, v).Let s(w) be (n, u) ∈ S w m with n = (l, ϕ, ρ) (chosen arbitrarily if |w| = 1) and let the abstract adversary's chosen transition be σ a . Therefore, there exists some δ ∈ R 0 such that v + δ ∈ ρ ∩ e∈E s(w) η(e) (and v + ∈ ρ for all 0 δ, since ρ is convex).By concretisability of transitions, there is a corresponding concrete transition τ with G ⇒ G and E ⊆ E .Due to the above, a concrete adversary can choose this concrete transition after a suitable delay δ, getting the distribution λ 1 q 1 + • • • + λ k q k , where q j = (l j , f j (u), v j ) with v j = (v + δ)[r j := 0].The edges in E s are e j = (s(w), λ j , r j , s j ), where s j = (n j , f j (u)); since δ was chosen such that v + δ ∈ η(e j ) = E ∩ [r j := 0]η(s j ) for all j, we get v j ∈ η(s j ), and thus s j ∈ S q j , so we can choose s(wq j ) = s j .Iterating this argument, we get a concrete adversary following C .
2. Let σ be a concrete adversary following C , w = w 1 . . .w m be any path within Q S in the Markov chain induced by P and σ , and s 1 , . . ., s m be the corresponding path in the Markov chain (S, T S ) represented by C (with transitions By the above correspondence, the paths have equal probabilities.
From the soundness of the abstraction, we get a correspondence regarding reachability of the error: . N e is not reachable from n, then Q e is likewise not reachable from q m ; • if s m ∈ O , then the error may be reachable from q m with any probability in [0, 1] (we made no assumption on the behaviour of σ outside Q S ).(n 3 , c = 1).The outgoing transition contains a time constraint x < 1, which gets propagated backwards.One successor is error state (n e , c = 1), leading to an increase in the lower bounds.

The refinement loop
Our approach to computing reachability probabilities for PTPs is based on an iterative abstraction refinement loop, which generates increasingly precise MDP abstractions.At each iteration, we first compute minimum or maximum reachability probabilities for the MDP, yielding an (outer) bound p lb,min or p ub,max and an abstract adversary.Then, we partially concretise this adversary, based on a forward exploration through the model, yielding after each step a complementary (inner) probability bound p ub,min = π 1 (n i , u i ) or p lb,max = π 0 (n i , u i ).If the difference between the lower and upper bounds for the initial state of the model falls below a pre-specified tolerance ε, then abstraction refinement terminates.Otherwise, concretisation continues until an inconsistency is identified, which will be used to refine the abstraction.
There are two classes of inconsistencies which may occur during concretisation: state-based inconsistencies, which, due to tightness, will always manifest themselves as the failure of a valuation to satisfy the guard of the adversary's chosen transition, and time-based inconsistencies, which occur when there is no consistent choice of delay, and manifest themselves as the occurrence of an empty zone η(_) = ∅ in the concretisation.Accordingly, there are two separate refinement operations for an MDP abstraction: state refinement, which splits a node with a predicate ϕ ∈ Asrt(V); and time refinement, which splits based on an inconsistent set of clock difference constraints c 1 , . . ., c k .The former is relatively standard, for abstraction refinement techniques; the latter is a novel method that we have developed for the PTP model.In the next two sections, we describe each of these in more detail.The main refinement loop is sketched in Fig. 3. Pseudocode and descriptions of auxiliary functions addState(), expand() and backpropagate() are shown in Figs. 4 and 5. Details of the refinement functions stateRefine() and timeRefine() are given in the next sections.The algorithm maintains an MDP abstraction A = (N, n i , N e , T ), along with: • an abstract adversary σ a and resulting outer bounds p min A,n or p max A,n ; • a partial concretisation (S, O , E), along with the associated inner bounds π b (s) and time constraints η(s) and η(e) for all s ∈ S and e ∈ E; • a set bp ⊆ S of states whose time constraints still need to be strengthened to satisfy the required fixpoint equations (see p. 43).
The outcome of the algorithm, if it terminates, is a set containing an inner and an outer bound {π b (n i , u i ), p dir A,n i }.Note that termination cannot be guaranteed in general, since the class of PTPs is Turing complete (it contains counter automata as a subclass).If the system is a PTA, then in the worst case the refinement procedure constructs the region graph, since in each iteration we nontrivially split a clock zone; see the proof of Theorem 5.

State refinement
State refinement is triggered when forward exploration encounters a state (n, u) such that u does not satisfy the guard G of the transition σ a (n).Since G is a conjunctive quantifier-free formula, at least one of its atomic constraints is violated by u.We use the first failed constraint g to split n, introducing a case distinction.This makes the refinement local in the (in which case a refinement step is triggered and we return true) or bp is empty (in which case a fixpoint was reached, and we return false).Since in each iteration one of the finitely many zones η(s) shrinks, and zones satisfy the descending chain condition, the procedure always terminates.
sense that we only use information directly related to state (n, u), rather than having to take the entire concretisation into account.The result of this split is a pair of new nodes n + = (l, ϕ ∧ g, ρ) and n − = (l, ϕ ∧ ¬g, ρ).We then modify the sets of transitions: • first, n + and n − both inherit the outgoing transitions in T (n); -find the indices I := { j | n j = n} of edges which need to be redirected; -for each possible redirection ν : with n j = ν( j) for j ∈ I , n j = n j otherwise, and G ν obtained from G by adding the corresponding preconditions g • f j or ¬g • f j for j ∈ I ; -for each n = (l , ϕ , ρ ) with τ ∈ T (n ), replace τ by those τ ν for which ϕ ∧ G ν is satisfiable.
return S; Fig. 8. tsplit(τ , n, N ) is an auxiliary function for the refinement procedures.Given that n is split into a set N of nodes, it computes all possible variants of the given transition τ where each edge leading to n is redirected to some node in N instead.
Satisfiability checks needed in the last step of the above are performed using an SMT solver.Pseudocode for the refinement procedure stateRefine(), along with auxiliary functions split() and tsplit() are shown in Figs. 6, 7, and 8.
Note that the strengthening of G to G ensures the refined abstraction is again tight, and this is the step which actually introduces new predicates into the abstraction.Furthermore, the refined abstraction remains sound.
Theorem 4. Let A = (N, n i , N e , T ) be an MDP abstraction of PTP P , and let A = (N , n i , N e , T ) be obtained from A by a state refinement.If A is sound for P , then so is A .If A is tight, then so is A .Furthermore, A is a proper refinement of A in the sense that the state assertions ϕ ∧ g and ϕ ∧ ¬g in the new nodes are both satisfiable.
Proof.Let ∇ be a reflection of adversaries for A, and let A be obtained from A by splitting n into n + , n − using the constraint g.We can then adapt ∇ by defining for each wq ∈ Q + c with q = (l, u, v) and each δ 0: Since n i and error nodes n ∈ N e are never split (the former because it only occurs in (n i , u i ), and inconsistency with u i means that σ (n i ) cannot be taken and can be discarded; the latter because there are no outgoing transitions), we still have ∇ (wq, δ) = n i iff q = (l i , _, _) and ∇ (wq, δ) ∈ N e iff q ∈ Q e .Let w ∈ Q + c , and σ be a (concrete) adversary for ).Let I be the set { j | ∇ σ (wq j ) = n} and let the function ν : I → {n + , n − } be given by ν( j) = n + iff q j g.The refinement creates (among others) the new abstract transition τ with G obtained from G by adding the respective preconditions.
Existence of the concrete transition σ (w) implies that ∇ σ (w) satisfies these preconditions, and τ ν occurs in T (∇ σ (w)) after the split.So ∇ is a reflection of adversaries.As for concretisability of transitions, let τ  λ k ( f k , r k , n k )) be a transition in A .Then τ was obtained from a transition τ = (G, E, In each case, the associated locations are the same, so any concrete transition witnessing concretisability of τ also does so for τ , and we are done. In order to preserve tightness, all transitions have their guard G strengthened by the preconditions of the newly introduced constraints as needed; note that time constraints are copied from n to n + , n − and remain unchanged otherwise, so that E does not need to be changed.Finally, ϕ ∧ g is satisfiable since otherwise the transition σ a (n) = (G, E, ) would not have been in T (n), and ϕ ∧ ¬g is satisfiable since the valuation u giving rise to the split satisfies both. 2 Example 4. We return to the partial concretisation shown in Fig. 2 (right).The next step of forward exploration, from (n 3 , c = 0), fails since the guard constraint c > 0 is violated.Thus, node n 3 is split using the predicate c > 0. Fig. 9 shows the resulting MDP abstraction, where node n 3 has been split into n 4 and n 5 , representing the states with c > 0 and c 0, respectively.This leads to a corresponding split of transitions.Note the strengthening of guards with preconditions to ensure tightness.Fig. 10 shows two successive partial concretisations for the new abstraction.The first step encounters the clock constraint x < 1 on the transition from n 5 and propagates it backwards.In the second step, we expand (n 4 , c = 0), encountering the constraints x < 1, y > 3. Backpropagation then obtains the constraints: / * where i 0 = i m , and ordered such that only i 0 can be 0 * / if i 0 > 0 then for j = 1, . . .,m do amounts to intersecting them with the corresponding source invariants.We distinguish two cases, depending on whether c contains single-clock bounds; see the proof of Theorem 5 for details.

Time refinement
The refinement step for timing inconsistencies is more complex, for several reasons.Firstly, because we treat time symbolically, a failure of concretisability cannot always be localised as it could for the state predicate case.In general, for some discrete state s, we will be dealing with a contradiction between the constraints for some incoming edge (s 0 , λ 0 , r 0 , s) and those for the outgoing edges (s, λ j , r j , s j ), j = 1, . . .,k.In particular, we will need to use more than one difference constraint for splitting and obtain more than two new nodes.
Secondly, because of the implicit passing of time between transitions, we must be careful when splitting with difference constraints: suppose one of the constraints is a lower bound like x 5. Naively splitting n = (l, ϕ, ρ) into (l, ϕ, ρ ∩ (x < 5)) and (l, ϕ, ρ ∩ (x 5)) wrongly eliminates any path which enters n while x < 5, lets time pass until x 5, and then leaves n.
We avoid this problem by not adding the lower bound x 5 to the invariant of the second node; instead, we add it to the enabledness conditions of its outgoing transitions, so that it only has to hold when the node is left.One consequence of this is that the abstraction will not represent a partition of states, in the sense that exactly one abstract state is associated to each concrete state.Instead, we have a partition (captured by the notion of reflection) of pairs (q, δ) of a concrete state q and a delay δ 0, in the sense that exactly one abstract state is associated to each such pair.
A time refinement step is triggered whenever, during the backpropagation of timing constraints, we find s = (n, u) such that strengthening η(s) with the constraints η(e) on the outgoing edges e = (s, λ, r, s ) ∈ E s , and successively strengthening the constraints η(e ) on the incoming edges with η(s), would encounter an empty zone.Given such a contradiction involving n = (l, ϕ, ρ), the backpropagation algorithm computes an unsatisfiable core C = {c 1 , . . ., c m } (see Section 2.1), and calls timeRefine, which, for each j = 1, . . .,m: • defines a set of constraints ψ j = {c 1 , . . ., c j−1 , c j } -note that since C is unsatisfiable, every clock valuation satisfies at least one of the complements c i , so that ψ 1 , . . ., ψ m define a partition of R X 0 into zones; , where ψ j is given by the constraints in ψ j not of the form x c; • adds to T (n * j ) all transitions from T (n), strengthening their enabledness conditions by the constraints in ψ j to take care of lower bounds and ensure that again E ⊆ ρ.
Finally, the transitions in A are split like in the state-based case.For each τ we determine the set I := { j | n j = n}.Then, for each ν : where n j = ν( j) for j ∈ I , n j = n j otherwise, and E ν is the intersection of E with the target node preconditions.Lastly, for each n = (l , ϕ , ρ ) with τ ∈ T (n ), we replace τ by those τ ν for which E ν is non-empty.Pseudocode for the time refinement procedure timeRefine() is given in Fig. 11.Theorem 5. Let A = (N, n i , N e , T ) be an MDP abstraction for PTP P and A = (N , n i , N e , T ) be obtained from A by a time refinement step.If A is sound for P , then so is A .If A is tight, then so is A .Furthermore, A is a proper refinement of A in the sense that, for each of the new nodes n * j , either its invariant or the enabledness condition on its outgoing transitions is stronger.
Proof.Let ∇ be a reflection of adversaries for A and A be obtained from A by splitting n = (l, ϕ, ρ) into n * 1 , . . .,n * m using an unsatisfiable core c 1 , . . ., c m .Like in the proof of Theorem 3, we can refine ∇ by mapping all w, δ with ∇(w, δ) = n to a suitable n * j instead.We make one simplifying assumption on the ordering of c 1 , . . ., c m .Recall that any variable occurs in exactly 2 or 0 constraints of an unsatisfiable core; in particular, there is at most one lower bound x b and one upper bound x b .We assume that, if these do occur, they are c 1 and c m , respectively.We first treat the case in which each c j is a proper difference constraint, i.e. not an upper/lower bound.In this case, the new nodes form a partition n * j = (l, ϕ, ρ ∩ η j ), where the η j are pairwise disjoint and closed under time elapse.This means that, for each q = (l, u, v) ∈ Q c , there is an index j q such that v + δ ∈ ρ ∩ η j q for all delays δ 0 with v + δ ∈ ρ.We thus define: This again ensures ∇ (wq, δ) , where E is obtained by intersecting E with the invariant of ∇ σ (w) and the preconditions of the zones ∇ σ (wq j ).The existence of the concrete transition σ (w) shows that E is non-empty, and therefore τ a will be included in T (∇ σ (w)).
In case we do have lower and upper bounds , the split of n involves a fragment n * 1 representing all "early" steps, i.e. those taken while c 1 is false, all others requiring the outgoing transitions to occur after it becomes true.We reflect this in ∇ as follows, for wq ∈ Q + c with q = (l, u, v): where again j q is the unique index with v + δ ∈ ρ ∩ η j q .Otherwise, the argument works exactly as in the first case.
As in the state refinement case, we again observe that any transition τ ∈ T (n ) is obtained from a transition in A whose source node belongs to the same location as n , each of whose target nodes belongs to the same location as the corresponding target in τ , and whose enabledness conditions are implied by those in τ , so that A inherits concretisability of transitions from A.
Tightness is preserved by strengthening the enabledness condition E in each transition with the same constraints as the invariant in its source node (in addition to lower bounds which are only added to E), and also with the pre-images [r := 0]ρ of the invariants in their new target nodes (done in tsplit()).State assertions are copied from n to the n * j and remain unchanged otherwise, so that G does not need to be strengthened.
To show that A is a proper refinement, it is enough to show that, for each constraint c j in the unsatisfiable core, c j is not implied by ρ, i.e. c j is satisfied by some t ∈ ρ.There are two cases, depending on where in the backpropagation the inconsistency giving rise to the split occurred.
If it happened when trying to strengthen η(s) for s = (n, u), then: and the split is performed with an unsatisfiable core C for ρ, {η(e) | e ∈ E s }.By tightness, for each e ∈ E s , η(e) ⊆ E ⊆ ρ, and η(e) = ∅ since otherwise a refinement would have been triggered earlier.So each constraint in C is implied by a non-empty subset of ρ, and therefore satisfied by some v ∈ ρ.
If the inconsistency occurred when strengthening η(e) for e = (s , λ, r, s) with s = (n, u) and σ a (n) = (G, E, ), then: and the split is performed with an unsatisfiable core C for E[r := 0], η(s).

Experiments
We have built a prototype implementation of our local abstraction refinement approach on top of the probabilistic model checker PRISM [18].We use an extended version of PRISM's existing DBM library for manipulating zones and the MathSAT 4 SMT solver [4] for satisfiability queries.Experiments were run on a 64-bit PC with an Intel Xeon CPU X5660 2.80 GHz and 32 GB RAM.Our implementation is parameterised by the strategy used to search the state space during concretisation.
For models with unbounded data types (see below), we use breadth-first search and set ε = 1 × 10 −6 for termination of refinement; for other models we use depth-first search and set ε to zero.
PRISM and its modelling language already have native support for PTAs, and these can incorporate data variables using PRISM's built-in datatypes (bounded integers and Booleans) so we use this as a basis for modelling PTPs.We evaluated our implementation on 11 case studies.Firstly, we used 6 existing PTA benchmarks [26]: csma full, csma abst (two models of the CSMA/CD protocol); nrp malicious, nrp honest (two variants of Markowitch & Roggeman's non-repudiation protocol); and firewire impl, firewire abst (two models of the FireWire root contention protocol).Secondly, we used real-time versions of two more complex models: the Zeroconf model of [13] (zeroconf full), and the sliding window model of [11] (sliding window real-time), which uses infinite data types (an unbounded integer storing the round counter).Finally, we tested our implementation on three discrete-time (MDP) models with infinite data types: the (original, discrete-time) sliding window protocol (sliding window discr.)and bounded retransmission protocol (brp) models from [11]; and the discrete-time model of Zeroconf (zeroconf discr.)from [13], modified so that the counter of "probe" messages sent is an unbounded integer.This change does not affect the value of the property we check.For brp, we fix the parameter TIMEOUT = 16.In order to support infinite data types, we slightly extended the PRISM modelling language in the style of PASS [9].All models and properties, and our prototype implementation, are available at [27].
We compare our approach to other available tools.For PTPs with only bounded datatypes, we run PRISM [18] (v.4.0.1), using its game-based abstraction refinement [13] engine (which outperforms digital clocks [19] on all but the zeroconf full case study).We also compare to FORTUNA [3] (v.0.2), which verifies (priced) PTAs, but we are only able to run a subset of the benchmarks since models are hard-coded into the tool.FORTUNA only handles maximum probabilities, but can compute minimum probabilities for some models through hard-coded model translations.For the three MDP models, we compare to the predicate abstraction tool PASS [9].None of these tools are applicable for the sliding window real-time model, which uses both clocks and unbounded data types.Recent work [8] describes an extension of PASS for real-time models.Presently, we only have access to the experimental results in [8], rather than the tool; we make a brief comparison below.
For the csma full benchmark, we pre-process models to reduce the degree of probabilistic branching and avoid transition explosion during refinement (details at [27]).PRISM performs better on the original so we use that for comparison.

Results
Experimental results for the real-time models are summarised in Tables 1 and 2, covering maximum and minimum probability properties, respectively.The results for discrete-time (MDP) models are in Table 3.For each abstraction refinement method (PRISM, PASS and our local abstraction refinement), we show the number of abstract states (both the peak and final number), the number of iterations of refinement and the total time.For FORTUNA, we just show states and time, since no refinement is performed.
Our prototype consistently outperformed PRISM's game-based abstraction for PTAs except on the firewire impl model with T = 10 000.We use a significantly higher number of refinement steps, but these are small, local refinements and, crucially, the size of the abstraction constructed is typically much smaller.It should be noted, though, that the zeroconf full The comparison with FORTUNA is more varied.Our prototype runs slightly faster on the larger nrp models, and does not fall far behind FORTUNA on the csma abst and firewire abst models.But FORTUNA performs much better on the csma full examples.This is because abstraction refinement for these models needs a relatively high number of refinement steps, each of which requires numerical computation to be performed.For PTAs, it is possible to avoid abstraction refinement entirely, as done by FORTUNA.On more complex models with many data variables, the benefits of abstraction should be more visible.In particular, for those with unbounded data variables, our abstraction refinement method works where PRISM and FORTUNA do not.
PASS has several different refinement strategies, from which we select the best performing one on each model.The results show that our prototype needs significantly less runtime on the sliding window discr.and zeroconf discr.case studies (and scales to larger models on the latter).Like for the comparison with PRISM, we use more iterations of refinement, but each one is cheaper; our abstractions also have significantly fewer states, For the brp example, PASS performs slightly better: it benefits from a small number of states, while our implementation suffers from heavy numerical analysis, especially when p1 and p2 are close to one, which makes value iteration converge very slowly.On this example, we used a simple heuristic that gives significantly improved performance: we discover all conflict states during the symbolic execution of an adversary and choose the state with the maximum depth for splitting.Interestingly, this heuristic does not improve performance for other case studies, which is worth further investigation.A different numerical solution method (not value iteration) might also help in this case.Further investigation on a wider set of untimed models is required to make a more thorough comparison between local abstraction refinement and the techniques implemented in PASS.At the present time, our focus is primarily on models incorporating both real-time and data aspects.Finally, we make a brief comparison with the tool PASS-PTA, described in the recent paper [8].Since this is not available for testing, we give an indirect comparison based on the results in this paper and in [8].The only model in common is csma (property P1 in [8]), on which PASS-PTA is slower than PRISM and local abstraction refinement is faster.For zeroconf our results are from a more complex variant of the model than in [8].We intend to perform a more in-depth comparison when PASS-PTA becomes available.

Overview
In summary, our approach tends to perform well, generating comparatively small abstract models using a relatively large number of (small, local) refinement steps.When the number of refinements becomes particularly high (e.g., csma with bmax = 4, K = 16), performance is degraded but, in our experiments, the tool still outperformed the game-based abstraction of PRISM.This gain derives from the use of simple operations on smaller abstractions and a reduction in the amount of numerical computation that needs to be performed.
Perhaps unsurprisingly, our approach performs better on real-time models, than on the versions that have been discretised.Compare, for example, the zeroconf models in Tables 1 and 3.These models are equivalent (and indeed, the results match), but abstraction is much more efficient when applied to the real-time version.For the sliding window model, a direct comparison is not possible since the real-time version of the model contains an additional clock because of limitations in the modelling language.This model does, though, illustrate the main benefit of our approach: the ability to verify models where abstraction of both the timed and data aspects is required, in particular when both real-time clocks and unbounded data types are present.

Conclusions
We have presented a novel abstraction refinement approach for the verification of probabilistic, real-time systems with potentially infinite data variables.Our approach uses local refinement steps, which results in more compact abstractions than alternative abstraction refinement techniques.
Future work includes extending it to support probabilistic hybrid automata (where operations to build and refine abstractions are even more costly) and applying it to mainstream programming languages such as C, Java or SystemC.
the set of clock valuations from which ρ can be reached by letting time pass;• clock reset: if Y ⊆ X , then ρ[Y := 0] = {v[Y := 0] | v ∈ ρ}contains the valuations obtained from ρ by setting the values of all y ∈ Y to 0; • inverse reset: if Y ⊆ X , then [Y := 0]ρ = {v | v[Y := 0] ∈ ρ} contains the valuations which end up in ρ if the values of all y ∈ Y are set to 0.
and let σ be an arbitrary adversary of JP K. Consider the MDP M

Fig. 2 . 2 Example 3 . 2 (
Fig. 2. Two partial concretisations for the MDP abstraction of Fig. 1.This implies that the reachability probabilities of S e = {(n, u) ∈ S | n ∈ N e } and of S e ∪ O are a lower and an upper bound on p σ P , and (see e.g.Thm.1.3.2in[22]) they are exactly the least solutions of:• p(s) = 1 if s ∈ S e (resp.s ∈ S e ∪ O ), and • p(s) = T (s)(s )p(s ) otherwise, making them equal to π 0 and π 1 .2 Example 3. We return to the MDP abstraction and abstract adversary σ a of Example 2, shown in Fig. 1 (right).Fig. 2 (left) shows a partial concretisation.The top of each box shows the discrete state; the bottom shows the lower probability bound π 0 (since we are computing maximum probabilities) and time constraint η(s).Unexpanded states are drawn with dashed lines.For clarity, edge details are omitted.Fig. 2 (right) shows a subsequent partial concretisation, expanding state

Fig. 4 .Fig. 5 .
Fig.4.Procedure addState(s), left, adds a new state s to the partial concretisation, initialises its time constraints η(s), and schedules it for expansion (puts it in O ) if necessary.Procedure expand(s), right, determines the successors s 1 , . . ., s k of s, adds the edges from s to s j , and adds s to bp if its time constraints now need to be strengthened.

Fig. 6 .
Fig. 6. stateRefine(n,u) splits node n into new nodes n + , n − based on a guard constraint violated by u.

Fig. 11 .
Fig. 11.timeRefine(n, C ) splits a node with time constraints obtained from an inconsistency.Note the strengthening of the enabledness conditions E, which
standard model for systems with both probability and nondeterminism.An MDP is a tuple (S, s i , S e , T ), where S is a (possibly infinite) set of states, s i ∈ S is the initial state, S e ⊆ S is a set of error states and T : ({s 1 s 2 . . .| s 1 = s and s j ∈ is an adversary of M}.Probabilities from the initial state s i of the MDP are denoted by omitting the subscript s i , i.e. σ R is a probabilistic simulation, and we get p min By tightness, both are subsets of ρ.E[r := 0] is non-empty since E is non-empty, and η(s) is non-empty since otherwise a refinement would have been triggered earlier.Again, each constraint in C is implied by a non-empty subset of ρ, and therefore satisfied by some v ∈ ρ. 2 For presentational simplicity, we illustrate time refinement on a separate example.Fig.12(topleft)shows part of an MDP abstraction in which we want to split n 1 with c = {x > 2, x < y, y < 1}.From c, we first obtain a partition {x 2, x > 2 ∧ x y, x > 2 ∧ x < y ∧ y 1}.The straightforward analogue to the state-based case, using these zones as invariants in three copies of n 1 , falsely makes it impossible for time to progress beyond 2 in l 1 .Adding lower bounds such asx > 2 to enabledness conditions instead of node invariants (as on the transition n 5 → n 2 in the corrected version) fixes this.

Table 1
Experimental results for real-time models (maximum probabilities).

Table 2
Experimental results for real-time models (minimum probabilities).be expensive for PRISM since the current implementation expands parallel composition in a PTA before constructing an abstraction from it, resulting in a blow-up in the (intermediate) state space.It should be possible to adapt PRISM to bypass this step and avoid the blow-up.

Table 3
Experimental results for discrete-time models.