An investigation of safe and near-optimal strategies for prevention of Covid-19 exposure using stochastic hybrid models and machine learning

In this work investigate the use of stochastic hybrid models, statistical model checking and machine learning to analyze, predict and control the rapid spreading of Covid-19. During the pandemic numerous studies using stochastic models have been produced. Most of these studies are used to predict the effect of some restrictions. In contrast, in this paper we focus on the synthesis of strategies which prevent Covid-19 spreading. The computed strategies provide valuable information which can be used by the authorities to design new and more specific restrictions. We consider two large case studies that develop in the Copenhagen area in Denmark. Our experiments show that the computed strategies significantly prevent Covid-19 spreading, and thus provide valuable information e.g. expected social distance to minimize Covid-19 spreading. On the technical side, we demonstrate the applicability of analytical methods for preventing the spreading of Covid-19 in large scenarios.


Introduction
The spread of Covid-19 had a tremendous impact on societies around the world: people were restricted from traveling, people were working from home, and wearing face masks was customary todayfew years ago face masks was a rare sight in most western societies -and shops were closed. All of these restrictions were implemented by government to limit the spread of Covid-19 but before such drastic measures were implemented their effect should be assessed. Classic techniques for this is modeling the evolution of a pandemic using compartmental models (using ordinary differential equations) [1,2] and making informed guesses on how different measures affect the parameters of the ordinary differential equation. However, those models lose behavior of individual agents and assume everyone interacts with everyone -which is clearly not true. In contrast, agent-based models, model each individual agent and models its precise movements and interactions. Computer science has a long tradition for making models of computer systems individual processes (its agents) and synthesizing control strategies for them. Our efforts to model cyber-physical systems have also pushed us in the direction of stochastic hybrid games. Thus the modeling language and analyses employed by our tools should be ideal for evaluating different pandemic control mechanisms.
The crisis caused by the spread of Covid-19 has pushed the community to develop numerous models to predict the spreading of Covid-19 or to evaluate the impact of given restrictions e.g. use of masks.
(near-optimal), strategies which guarantee that certain locations are periodically visited (safe), and strategies that guarantee visiting locations while preventing Covid-19 exposure (near-optimal and safe).
For both our case studies, we perform an extensive evaluation of the strategies that we compute. We run experiments involving thousands of simulations in hundreds of CPUs, such computation took several days. Our results show that implementing the computed strategies can greatly prevent Covid-19 spreading. In addition, our experiments confirm that it is possible to compute strategies effectively for such real world based case studies involving thousands of agents. In particular for the Lakes case study we are able to compute strategies in real-time, for this we use techniques from [5] to periodically compute online strategies online for a short horizon.

Related work
Statistical methods and tools are playing a major role in the study of Covid-19 developments. As such a there exist numerous studies and tools. In [6] a decision framework to help software startups to increase their chance of success during the Covid-19 pandemic is proposed. The overall goal of that work is thus different than ours. However, their proposed framework which uses the Nominal Group Technique methodology, could profit from using the techniques we propose to synthesize strategies which could aid software start ups. The work in [7] studies a plan for minimizing the costs of distributing Covid-19 vaccines in the European Union (EU). The plan is then obtained by solving a mix integer program. While that work focuses on minimization of costs, our work focuses on minimization of Covid-19 spreading. The techniques that we present are complementary, because the vaccine distribution problem could be represented using stochastic hybrid models which will allow a more refined description of the vaccine distribution process. Then one could compute safe and nearoptimal strategies for minimization of costs and ensuring guarantees on the distribution of Covid-19 vaccines. In [8] A multi criteria evaluation methodology for assessing the impact of Covid-19 in EU countries is presented. This work is orthogonal to our. However, we can consider their results to construct more accurate models, or to calibrate our models. The study in [9] considers the impacts of Covid-19 in the behavioral change of passengers. In particular it studies the attitudes of passengers towards preventive measures. The results from that work could be included in our work to refine our models. For example, the study measures the compliance of passengers with respect to social distancing. We could use their measures to calibrate our models. In [10] the authors evaluate the effect of restriction policies e.g. school closures in Australia. Similarly [11] uses agents to evaluate Covid-19 development in Iran. In [12] compartmental model is used to study the spread of Covid-19 in Wuhan. Other works focus on the development of dedicated tools for performing simulations e.g. Covasim [13]. The key difference between our work and studies similar to the ones mentioned above, is that we synthesize strategies where as the other approaches evaluate different scenarios. Our work builds directly on previous Covid-19 models from [4] and techniques for computing near-optimal strategies from [5]. In [14] a predictive analytics model for Covid-19 pandemic using artificial neural networks is presented. The work is focused on prediction where as in our work we focus on optimization. In our second case study we implement the SEIHR (Susceptible Exposed Infected Hospitalized Recovered) epidemiological model. During the Covid-19 pandemic, the mathematical epidemiology community have produced a number of other relevant models. The work in [15] presents generalizations of the SIR model e.g. the SIR-PH model. In [16] the authors present the SEIHQRD model which adds the critically ill class whereas the work in [17] SEIAHR includes the asymptomatic class. Note that these works are complementary, since we could use them as the underlying epidemiological model.
Structure of the paper. The structure is as follows, In Section 2 we give an overview of the underlying models and tools which we use in our work. Section 3 we present case study related to computing nearoptimal strategies for walks around the Lakes. In Section 3.1 we give a detailed description of the scenario. In Section 3.2 we describe the tools and preliminaries required for this case study. Sections Section 3.3 and Section 3.4 describe the models and the controllers that we use in this case study. Section 3.5 presents our experimental results for this case study. Our second case study involving the family itineraries in Copenhagen is presented in Section 4. Section 4.1 describes in detail the case study. Section 4.2 describes the tools and methodology used in this case study. Later Section 4.4 presents our experimental results for this case study. Finally Section 5 presents our conclusion.

Computing Strategies with Uppaal
Uppaal [18,19] is a family of software tools used for the design, analysis and verification of complex systems with wide application in industrial scenarios. Uppaal Stratego [20] is a branch of Uppaal which can synthesize strategies for complex systems and has been successfully used in several industrial case studies [21][22][23], and in this work to minimize the spreading of Covid- 19.
The models that we use in this work are large, complex, and optimized for scalability. Therefore, in order to introduce the models and functionality of Uppaal we present a small example. Consider Fig. 1 which is a Uppaal Stratego model implementing a Stochastic Hybrid Game [5,24] between a controller and the environment. A Strategy indicates the controller which actions to take in order to minimize a certain function. In the rest of this work we use the Greek letter to refer to strategies.
The model represents an agent (the controller -solid arrows) which needs to commute from Home to Work. The agent can go by Bus or by Train. If the agent chooses to go by bus there is a probability weight of 5 that the bus is Full and a probability weight of 1 that the bus is almost Empty. Note that taking the bus takes up to 20 min indicated by the invariant T<=20. Additionally, there is a risk for the agent to become infected. The expression risk'==4 * beta is the derivative of the variable risk with respect to time. Thus if the bus is full, the likelihood for the agent to become infected is 4 times higher than when the bus is empty. Similarly if the agent decides to take the Train there is probability weight of 2 for the agent to miss the train and to wait for T <= 10 time units. On the contrary the agent takes the train and reaches work in at most T<=15 time units. Note that there is a risk for the agent to get infected with derivative risk'==4 * beta.
There are a number of interesting questions that we would like to ask the model. For example, Would it be possible to go with certainty from Home to Work within 50 time units?. This could be answered by computing a safe strategy. Another question could be, Would taking the bus or the train minimize the risk of infections?. This could be answered by computing a near-optimal strategy. In fact approximated expected value using 1000 simulations for variable risk under a nearoptimal strategy is 16.75 ± 0.5, in contrast the approximated expected value is 24.8 ± 1.06. Finally, it would be interesting to minimize the risk of infection while guaranteeing that the agent reaches work within a time bound. This can be answered by computing a safe and nearoptimal strategy. In Section 3 we compute near optimal strategies, and in Section 4 we compute safe and near-optimal strategies to prevent Covid-19 spreading.

SUMO
SUMO [25] is an open source tool which allows to model and simulate traffic and pedestrian dynamics at a microscopic level. It network transformation, waiting time calculations, traffic light performance, etc. SUMO provides features for modeling a vast number of scenarios and  possibilities to inter-operate with other tools. There is also a wide active community which offer support.
In this work we mainly use the following SUMO components. Netconvert to import the Copenhagen Lakes from open street maps. Road networks which allow to model the relevant part of the map, roads lanes and intersections of the Lakes. Traci a software interface that gives access to objects in the running simulation, and enables communication with Uppaal Stratego.

SEIHR compartmental model
Compartmental models is a modeling technique applied in epidemiology. A simple compartmental model is the SIR [15] susceptible ( ) being those that can affected by the disease, infectious ( ) being those that have the disease and can infect others, recovered/removed ( ). In both of our case studies we use a more refined compartmental model i.e. the SEIHR compartmental fluid model, where the population is divided in susceptible, exposed ( ) being those that have the disease but not infectious, infectious, recovered/removed ( ), and hospitalized ( ). The rate diagram is given in Fig. 2. The arrow to with a rate multiplied by the number of elements. Similarly for the other arrows. A major challenge consist in calibrating the model using appropriate rates. We use the rates as presented in previous work [4] which where provided by the Novo Nordisk Fonden project that fitted Danish data to a SEIHR model and estimated the parameters for Danish conditions.

Near-optimal strategies for walks around the lakes
In this case study we consider the one direction walking restriction in the Lakes (Søerne) in Copenhagen c.f. Fig. 3 The restriction was given by Statens Serum Institut (SSI 1 ). We construct an agent based model which considers the geographical location and the stochastic behavior of pedestrians. Our model can be seen as an stochastic hybrid game in which the controllable modes include the choice of allowing an agent to enter the lakes. Our goal is to compute a near-optimal strategy which minimizes the number of Covid-19 exposures by maintaining social distancing.

Scenario description
The Copenhagen lakes are a popular location for pedestrians in Copenhagen, and therefore also a potential hot spot for Covid-19 exposure. Recently, Copenhagen municipality introduced a one-way walk restriction around the lakes to help minimize Covid-19 exposure. In addition to this restriction we make the assumption that when a pedestrian wish to enter the Lakes for a walk she will request access to the Lakes, and she will not enter until the request has been granted. In reality, this assumption could be implemented by using pedestrian traffic lights at different points around the Lakes. Once a request has been issued, a controller is responsible to grant or deny access. For this scenario we implement several controllers described in Section 3.4.

Tools and methodology
To model the scenario as close to reality as possible we use the microscopic simulation tool SUMO [3,25] c.f. 2.2 where we consider a number of parameters such as, initial number of infected pedestrians, coughing probabilities, wind direction, different speeds for pedestrians, etc.
The architecture of our system is given in Fig. 4. The SUMO model can be seen as an Stochastic Hybrid Game where the controllable modes correspond to allowing an agent to enter the Lakes, and the uncontrollable modes correspond to the environment which includes the stochastic movement of the agents and their coughing probabilities. Ideally,  we would like to compute an Strategy for a long horizon (which could be a full day or week). Unfortunately, computing such an strategy is unfeasible given the that the number of choices grow exponentially with the horizon. To overcome this problem we apply the following methodology.

Online strategy synthesis
For this case study our goal is to compute a strategy (controller) to prevent Covid-19 infections for a long horizon . As the number of choices for the controller grows exponentially in the horizon, computing the strategy for a long horizon is intractable. To overcome this problem we resort to the Online Strategy Synthesis [5] methodology, where we periodically compute a near-optimal strategy ℎ for a short horizon ℎ < . The strategy ℎ is less accurate than the strategy but it can be computed effectively and periodically. The methodology has successfully been applied to multiple case studies involving cyberphysical systems such as, intelligent traffic lights [23], floor heating systems [5], rerouting [26] etc. Fig. 4 depicts the architecture of our system. Whenever an agent request to enter the Lakes, the status of the SUMO simulation is passed to a controller, the controller after some computation, returns a strategy ℎ which SUMO implements to continue with the simulation. For this case study we implement and compare several controllers, in particular we implement a Uppaal Stratego controller.

Model description
The lakes can be seen from a birds-eye view in Fig. 3(a). The lakes are split into multiple smaller lakes with walking areas around and between all lakes. A SUMO recreation of the lakes can be seen in Fig. 3(b). All vehicular simulation has been omitted to simplify the scenario.
The full Copenhagen lakes are modeled in SUMO. However, simulating the complete network may include tens of thousands of pedestrians and cause computational problems. As such, as a proof of concept we focus only on the southernmost lake, the SUMO network recreation is seen in Fig. 3(c), in which we also see a large number of pedestrians walking around the lake.
We assume there to be some controller allowing or disallowing pedestrians to enter the network e.g. a pedestrian traffic light, such that any pedestrian (agent), who wants to enter the network sends a request to enter the network at some entry point to the controller. The controller then answers by either allowing the agent to enter or not.
Furthermore, we assume some of the agents to be infected with Covid-19 and cough under a given distribution. If an infected agent coughs, given the wind conditions, the viral load is spread in a cone. If the cough overlaps with another agent the viral load of the agent is increased and thus its chance to be infected as well. This scenario can be seen in Fig. 5 in which the cough is modeled as a triangle from the agent.  Table 1 describes the main parameters that we use for computing SUMO simulations. The wind is a two dimensional vector which influences the length of the triangle which can infect other agents, the width of the triangle is given by the constant triangle degree. Every agent has a variable accumulating the viral load to which the agent has been exposed, if the viral load exceeds the constant infection threshold, the agent becomes infected. At every simulation step, agents have a cough probability. The SUMO model for pedestrians accepts a number of parameters, we use the default parameters but we use only three different walking speeds.

Controllers
In our scenario, we have several entry points to the lakes. For every entry point, pedestrians make request to enter the Lakes. It is responsibility of the controller to allow or disallow an agent to enter. We make use of three different controllers, note that these controllers do not have access to SUMO variables such as, walking speeds, or wind speed. Algorithm 1 gives a high level idea of the interactions taking place within the SUMO model and the different controllers. Line 4 of Algorithm 1 uses one of the controllers described below to decide if an agent is allowed or not to enter.

Default controller
This controller models the usual behavior of pedestrians wishing to enter the lakes i.e. the controller allows agents issuing enter requests to enter immediately. This serves as a baseline for simulating the scenario with no external input.

Naive controller
This controller is used to model social distancing, an agent prior to enter the Lakes, looks a number of meters around her. If there is implement strategy ℎ in SUMO (allow agent to enter or not) enough room, the controller allows the agent to enter. Different social distancing values will result in different controllers. The social distance used for our simulations is 5 meters.

Uppaal stratego controller
The controller which implements the online strategy synthesis methodology explained in Section 3.2.1. The controller computes online near-optimal strategies for whether or not to allow an agent to enter the network c.f. Section 2.1.
The corresponding Uppaal Stratego model is given in Fig. 6. Note that the Uppaal Stratego model is an abstraction of the SUMO model, and will be used to perform reinforced learning. Following Algorithm 1, line 4 will create an instance of this model using the current observations from the SUMO model e.g. positions of the walking agents. Then Uppaal Stratego is going to be called to compute a strategy for each agent.
In the following we give a detailed description of the Uppaal Stratego model. The controller is initialized with the known information of the world such as pedestrian locations, current requests, etc. The initialization is performed by an external C++ library called by functions restart _ world() and initvars(). The Uppaal Stratego functionality to load libraries is recent and allows easy implementation of tasks such as parsing files, etc. In this case the library parses a JSON file with thousands of lines with the observable state of the SUMO simulation. The observable states includes current positions of agents, but it does not include their speeds, or if they are infected.
After initialization, at location SelectAction the action of the agent is chosen as either entering the network or not entering the network. If the controller chooses to not enter a penalty is increased depending on how many agents are waiting to enter. In this way, the penalty can be used to force agents into the Lakes. However, if the penalty is too high, the number of Covid-19 exposures might increase.
At the next location there are three choices with probability weights 3,6,3. These choices are used to coarsely approximate the unknown walking speeds of the agents in SUMO.
The next transition set the chosen values for the agent in the C++ library. Finally the function do _ sim _ step calls the C++ library to simulate the choice for some time horizon and calculates the penalty of the choice. In this step we use the C++ library to compute the posterior of the abstract world where e.g. agents are represented as circles moving at the chosen speeds. Since we have hundreds of agents and the learning methods of Uppaal Stratego are going to call this function hundreds of times, this step needs to be implemented as efficient as possible. Therefore, having an optimized compiled library is quite convenient in comparison to an implementation in Uppaal Stratego which would be interpreted at running time.
Given this model, every time there is an agent request to enter the lakes, Uppaal Stratego executes hundreds of simulations to compute a near-optimal strategy which minimizes the variable penalty, and thereby the possible Covid-19 exposure.

Experiments
For our experiments we conduct 30 simulations for each scenario for every controller. Every simulation is run using the same seed for each controller, meaning each controller is given the exact same simulation configuration. A scenario consists of a number of persons that wish to enter the network in a given time spawn where every person is infected by a given probability inf Pr. We conduct an experiment for each combination of persons in 100, 600, 1100, 1600 and 2100, the time spawn include 100,1000 and 2000 seconds, and inf Pr. includes 0.01, 0.05 and 0.10.
The results can be seen in Table 2. The column Scenario corresponds to the combinations of persons, spawn, and inf Pr. Since we run 30 simulations for each scenario, the column Controller shows for We see that both for the Naive controller and the Uppaal Stratego controller the computed mean results in much less infected compared to the Default controller. We also see that when the scenarios become larger the Uppaal Stratego controller generally minimizes Covid-19 infections by ∼ 50% compared the Naive controller and even more compared to the Default controller. However, the trade off seems to be the simulation steps expended on completing the simulations. That is in order to minimize the exposures Uppaal Stratego distributes pedestrians conservatively at the cost of time. As such, the waiting time (number of simulations steps) for entering the Copenhagen lakes using the Uppaal Stratego controller is greater than both the Naive and the Default controller. As described in Section 3.4.3, the Uppaal Stratego controller applies a penalty to force pedestrians to enter the Lakes. This penalty could be increased to reduce the waiting times at the cost of potentially more Covid-19 exposure.
To illustrate our results Fig. 7 selects the scenarios where we fix the initial infected probability inf_Pr to 0.1, and time spawn to 100. Fig. 7(a) shows the boxplots for the number of new infected. We observe that the arithmetic mean of new infected is less for Uppaal Stratego than for the other controllers. In contrast Fig. 7(b) shows the number of simulation steps needed until the last agent has walked around the lakes. This shows that objectives new infected and number of simulation steps (agent waiting time) are conflicting objectives. For instance we could increase the social distancing parameter of the Naive controller to 10 m, which we expect will reduce the expected number of infected but increase the simulation steps. Similarly we can modify the penalty function that Uppaal Stratego optimize. A next step could consider in finding Pareto optimal solutions.

Safe and near-optimal strategies for family routines in cities
In [4] Section 4 a stochastic hybrid model of the behavior of a family under given itineraries is used to estimate Covid-19 exposure. Our goal in this work is not only to estimate but to minimize Covid-19 exposure. Towards this goal we transform the model from [4] to a stochastic hybrid game, where the choices in given itineraries are controllable in an uncontrollable environment. We compute different kind of strategies to minimize Covid-19 exposure, guarantee objectives, or both guarantee objectives and minimize Covid-19 exposure. Finally we run numerous experiments to demonstrate the effects and applicability of our approach.

Scenario description
We give a brief description of the scenario. The family consist of three members, a mother, a father, and a son. The family lives in Copenhagen with about 613,288 inhabitants, the members of the family spend time at work, school and occasionally at leisure activities. The following locations in Copenhagen are of interest for the family, the city of Copenhagen, Parken, ITU, Maersk, Vesterbro Ny School and Nyhavn. Each location have their own SEIHR ODE-based model c.f. Section 2.3. Each location has a population-size as well as a specific transitionrate for flow between susceptible (S) and exposed (E) reflecting the differences in being exposed at various locations.
Every family member have associated leisure activities, in particular Father is fanatic about FCK (FC København) and has season tickets for home matches at Parken (capacity of 38,000 spectators). In order to preserve the happiness of the father it is imperative that ∶ Father visits Parken a number of times in a given period.
In the rest of this section we will use Happy to refer to the above safety property.

Tools and methodology
As in the previous case study, the central formalism we use is a stochastic hybrid game  between a controller and the environment. In this game, a strategy indicates the controller which actions to take in order to minimize or maximize a given function. We use | to describe the resulting stochastic hybrid game where the choices of the controller are restricted by .
The Uppaal family provides a number of tools for computing strategies in different models. Fig. 8 [27] describes the workflow on the different models and strategies that we compute. First the case study is modeled as a stochastic hybrid game  where the controllable actions correspond to the itinerary choices of an agent, and the uncontrollable actions correspond to the environment. The detailed stochastic hybrid game  can be abstracted to a timed game [28]  where probability distributions have been abstracted to their support and non-linear constraints have been over-approximated. By using Uppaal Tiga on the timed game  one can compute the most-permissive strategy with respect to a property . Note that executions of the game | are guaranteed to satisfy the property . However is not optimal and can be further refined. Uppaal Stratego will use machine learning and statistical methods to produce the safe and near-optimal strategy • . The executions in the game | • are guaranteed to satisfy the property while minimizing a certain cost function. Finally, note that is possible to directly compute a near-optimal strategy which will not provide any guarantees. In the following we describe the strategies that we compute for this concrete case study.
Minimize A first approach consist on synthesizing a strategy which minimizes the Covid-19 exposure of Father during his daily activities. For this we can use Uppaal Stratego to synthesize a strategy such that behaviors restricted to the strategy i.e. | minimize Covid-19 exposure. Note that behaviors in | may violate safety property Happy, i.e. behaviors which are contra-productive to the well-being of Father.
Guarantee Let us consider the case when we would like to ensure the property Happy i.e. Father visits Parken a number of times in a given period. For this the model  is abstracted to a timed game  where Uppaal Tiga computes a permissive strategy which guarantee property Happy. Note that | include behaviors where the father is heavily exposed to Covid-19.
Guarantee and Minimize Ideally we would like to preserve the happiness level of Father while minimizing its Covid-19 exposure. Towards this goal the stochastic hybrid game under the permissive strategy | can be further restricted to behaviors which minimize Covid-19 exposure. To obtain this we use Uppaal Stratego to produce a safe and near-optimal strategy • . Note that all behaviors in | • satisfy Happy and attempt to minimize the risk of Covid-19 exposure for Father.

Model description
With the purpose of illustrating how strategy synthesis techniques and the Uppaal family of tools has been used to model, analyze and synthesize a variety of scenarios relevant for Covid-19 spreading, we transform of the model presented in [4] Section 4 to a stochastic hybrid game. Our transformation consist in syntactically declaring specific edges as controllable and the rest as uncontrollable. To illustrate this consider the itinerary of Father presented in Fig. 9(a) where solid edges correspond to controllable actions and dashed edges are uncontrollable actions.
The itinerary describes the whereabouts of Father on daily basis. He is at Home from 23:00 at night to 07:00 in the morning. In order to go to work ITU, he commutes between 07:00 to 08:00 through a part of Copenhagen Copen1. He stays at work from 08:00 to 16:00. After work at location ITU the Father has three controllable choices (solid edges), he can go to location NyHavn via Copen4, he can go to location Parken via Copen2, or he can go directly Home via Copen3. Note that for Father to return Home from locations Parken and NyHavn, Father has to visit extra locations in Copenhagen city area which increase Covid-19 exposure.
The model in [4] has an agent-based SEIHR model to describe the health status of each family member with transition-rates for flow between susceptible (S) and exposed (E). We accumulate the same transition-rates in the observable variable risk, e.g. the ODE risk '==risk _ parken accumulates the transition-rate (c.f. Fig. 2) from susceptible (S) and exposed (E) while at being at location Parken. We use the observable variable risk to guide the strategy synthesis process. Note that there is a one to one correspondence between and risk . Therefore, minimizing risk results in minimizing the probability of Father becoming exposed. In addition, we need to keep track of the number of visits from Father to Parken, for this every time location Parken is visited a counter visit _ parken is increased.
Finally, Fig. 9 describes a monitor for the property Happy. At the end of the period the monitor checks if the location Parken has been visited at least N times, if this is not the case the monitor goes to location Error indicating a violation to the property Happy.

Experiments
A stochastic hybrid game is a rich mathematical formalism which enables the design, analysis, and verification of complex systems. Computing strategies for such systems is computationally hard and difficult  to scale. In this section we demonstrate that it is possible to compute such strategies for large real world scenarios. And that our resulting strategies can have a positive effect on preventing Covid-19 spreading.
The property Happy expects a number of visits from Father to location Parken in a given period of time (horizon). For our experiments we consider different number of visits and different periods these can be observed in columns ''Desired Visits'' and ''Period'' in Table 3. For every scenario we compute the expected value for the variable risk under the given controller. We consider the following strategies: Random, Minimize, Guarantee, and Guarantee and Minimize. Where strategy Random uses a uniform distribution for the controllable choices at location ITU in the itinerary of Father, and the other strategies are as described in Section 4.2.
The expected risk computed under the strategy Random serve a baseline to compare with the other strategies. We observe that for all scenarios strategy Minimize reduces the risk of exposure of Father however it provides no hard guarantees. The strategy Guarantee has in several scenarios the highest risk of exposure for the Father, however this strategy guarantees the property Happy. Finally, the strategy Guarantee and Minimize which guarantees property Happy has higher risk than strategy Minimize but is less risky than strategies Random and Guarantee. The results suggest that the best itinerary for the father is the one induced by the strategy Guarantee and Minimize.  are not guaranteed. On the contrary the strategy Guarantee guarantees visits with higher risk 93.14, even higher than the Random controller. Finally, the strategy Guarantee and Minimize yields the value 71.42 which is closed to the minimal observed value while guaranteeing the number of visits. The strategy Guarantee and Minimize contains the itinerary that will enable Father to perform his activities while minimizing the risk of becoming exposed. All experimental evaluation was run on AMD Opteron 6376 Processors. We use a timeout of 2 days and for every row in Table 3 we use the limits of 28,40,60 80,100,120, and 140 GB of RAM respectively. We observe that as the period (horizon) increases, the computation time for the synthesis of strategies and the memory for storage of strategies increase significantly. Uppaal allows to compute expected values within a confidence interval, for this it uses a parameter . The results from Table 3 are computed with = 0.05. We have been able to compute results for the smaller instances with = 0.005. However, bigger instances timed out.

Conclusion
Summary. The Covid-19 disease has a negative impact on individuals, society and economy. Statistical methods are being heavily use to predict or assess different Covid-19 measures. In contrast in this work we focus on the synthesis of strategies which minimizes Covid-19 spreading. The resulting strategies can be used to suggest or refine preventing measures.
To demonstrate the applicability of our methodology we have considered to large case studies. In the first case study we model a hundreds of pedestrians walking with different speeds around the Lakes in Copenhagen. Infected agents can cough which might infect other agents. Agents request to enter the Lakes for a walk, the task of the controller is to grant or deny access. We are able to use Uppaal Stratego to generate online strategies which greatly minimize Covid-19 spreading. In addition, we observe that there is a trade-off between minimizing Covid-19 infections and the time agents need to wait to enter the Lakes. In the second case study we study a family with itineraries in the Copenhagen area. The task of the controller is find schedules which minimize Covid-19 spreading, guarantee itinerary objectives, and both guarantee itinerary objectives while minimizing Covid-19 spreading.
We have used stochastic hybrid games to model both case studies. We use several branches of Uppaal to compute such strategies. We run thousands of simulations on hundreds of CPUs. Our experiments show positive results, they show that it is possible to compute near-optimal strategies almost in real time while safe and near-optimal strategies can still be computed in hours or days. More importantly the computed strategies can greatly help in preventing Covid-19 spreading.
Contributions. Automata theory based modeling, verification and synthesis has been widely used in domains such as control of real timed systems, software verification, etc. Therefore one of the contributions of this paper is that is shows the applicability of formal method techniques in the epidemiological context for large and realistic scenarios. Another contribution is that we show how techniques as described in this work have been used in collaboration with Statums Serums Institute in Denmark to evaluate the impact of given measures. Finally, another important contribution is that our resulting strategies do provide a method to minimize the spreading of Covid-19 and that our experiments show that it is computationally possible to compute such strategies Caveats, limits and future work. The main caveat of our approach is that of most model based techniques and it relates with the accuracy of our models and reality. We have used some parameters from the literature and we have performed regression techniques using historical data to calibrate our parameters. The limits of the methods presented in this work are given by the size of the scenario, the accuracy of the strategies, and how realistic the model is. In both of our case studies when we reach large instances, the computations can take days or go out of memory. Similarly, if we increase confidence levels. Future research include efficient modeling and reduction techniques for large stochastic models. We also investigating on techniques for model calibration or learning models using historical data.