Skip to main content
Log in

Synthesizing efficient systems in probabilistic environments

  • Original Article
  • Published:
Acta Informatica Aims and scope Submit manuscript

Abstract

We present a formalism, algorithms and tools to synthesise reactive systems that behave efficiently, i.e., which achieve an optimal trade-off between a given cost and reward model. Synthesis aims to automatically generate a program from a specification. Most research in this area focuses on qualitative specifications, i.e., those that define a system as either correct or incorrect. The result can be a system that is correct, but still shows undesired behaviour, e.g., because it is too slow, inefficient or resource-intensive. Quantitative synthesis aims to use additional information to guide the synthesizer towards a desired implementation. Trade-offs between costs and rewards provide a natural source of information in order to guarantee efficiency. The systems we want to synthesize are open, i.e., they react to input signals from their environment. So, we have to specify how to combine the trade-offs the system decides to make for each input. There are several possible ways, e.g., worst or best case, or average case. In this paper we focus on the average case, i.e., we focus on the expected trade-off achieved by a system. We define the problem of finding the system with the best expected behaviour according to a quantitative specification. This specification associates costs and rewards with each decision the system makes and defines a probabilistic environment that the system operates in. We analyze the feasibility of this task (i.e., prove that such systems exist and are computable) and present three algorithms to compute an optimal system for a given specification. We compare a prototypical implementation of these algorithms against each other and, based on the best-performing algorithm, develop a novel symbolic implementation and integrate it into the probabilistic model checker PRISM. We report on experiments showing that our algorithm can analyze models with several million states.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. We can model a DDOS attack, e.g., by assuming that it happens with low probability: on attack, the environment suddenly changes its behaviour drastically.

  2. Our approach can also handle liveness specifications resulting in a Ratio-MDP with parity objective, which is then reduced to solving a sequence of MDPs with mean-payoff parity objectives [11].

  3. Pun intended.

  4. A feasible solution is an assignment that fulfills the linear equations.

  5. http://www.haskell.org.

  6. http://code.haskell.org/hmatrix/.

  7. http://hackage.haskell.org/package/glpk-hs.

  8. Note that, due to the way that we model the assembly line here in PRISM, these are different sized MDPs to the ones for the same example used in Table 1.

References

  1. Bahar, I., Frohm, E., Gaona, C., Hachtel, G., Macii, E., Pardo, A., Somenzi, F.: Algebraic decision diagrams and their applications. Form. Methods Syst. Des. 10(2/3), 171–206 (1997)

    Article  Google Scholar 

  2. Baier, C., Clarke, E., Hartonas-Garmhausen, V., Kwiatkowska, M., Ryan, M.: Symbolic model checking for probabilistic processes. In: Degano, P., Gorrieri, R., Marchetti-Spaccamela, A. (eds.) Proceedings of 24th International Colloquium on Automata, Languages and Programming (ICALP’97), volume 1256 of LNCS, pp. 430–440. Springer, New York (1997)

  3. Baier, C., Katoen, J.-P.: Principles of Model Checking. MIT Press, Berlin (2008)

    MATH  Google Scholar 

  4. Bloem, R., Chatterjee, K., Henzinger, T.A., Jobstmann, B.: Better quality in synthesis through quantitative objectives. In: Bouajjani, A., Maler, O. (eds.) CAV, volume 5643 of LNCS, pp. 140–156. Springer, Berlin (2009)

    Google Scholar 

  5. Bloem, R., Gabow, H., Somenzi, F.: An algorithm for strongly connected component analysis in n log n symbolic steps. In: Proceedings of 3rd International Conference on Formal Methods in Computer-Aided Design (FMCAD’00), pp. 37–54 (2000)

  6. Bloem, R., Greimel, K., Henzinger, T. A., Jobstmann, B.: Synthesizing robust systems. In: FMCAD, pp. 85–92. IEEE (2009)

  7. Brázdil, T., Brožek, V., Chatterjee, K., Forejt, V.Kučera, A.: Two views on multiple mean-payoff objectives inMarkov decision processes. In: LICS, pp. 33–42. IEEE ComputerSociety (2011)

  8. Bryant, R.E.: Graph-based algorithms for boolean function manipulation. IEEE Trans. Comput. 35(8), 677–691 (1986)

    Article  MATH  Google Scholar 

  9. Burch, J.R., Clarke, E.M., McMillan, K.L., Dill, D.L., Hwang, L.J.: Symbolic model checking: \(10^{20}\) states and beyond. Inf. Comput. 98(2), 142–170 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  10. Chatterjee, K., Henzinger, M.: Faster and dynamic algorithms for maximal end-component decomposition and related graph problems in probabilistic verification. In: Proceedings of 22nd Annual ACM-SIAM Symposium on Discrete Algorithms (SODA’11), pp. 1318–1336 (2011)

  11. Chatterjee, K., Henzinger, T.A., Jobstmann, B., Singh, R.: Measuring and synthesizing systems in probabilistic environments. In: Touili, T., Cook, B., Jackson, P. (eds.) CAV, volume 6174 of LNCS, pp. 380–395. Springer, Berlin (2010)

    Google Scholar 

  12. Chatterjee, K., Majumdar, R., Henzinger, T.: Markov decision processes with multiple objectives. In: Proceedings of 23rd International Symposium on Theoretical Aspects of Computer Science (STACS’06), volume 3884 of LNCS, pp. 325–336. Springer (2006)

  13. de Alfaro, L.: Formal Verification of Probabilistic Systems. PhD thesis, Stanford University (1997)

  14. Derman, C.: On sequential decisions and Markov chains. Manage. Sci. 9(1), 16–24 (1962)

    Article  MathSciNet  MATH  Google Scholar 

  15. Etessami, K., Kwiatkowska, M., Vardi, M., Yannakakis, M.: Multi-objective model checking of Markov decision processes. In: Proceedings of 13th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS’07), volume 4424 of LNCS, pp. 50–65. Springer (2007)

  16. Forejt, V., Kwiatkowska, M., Norman, G., Parker, D., Qu, H.: Quantitative multi-objective verification for probabilistic systems. In: Abdulla, P.A., Leino, K.R.M. (eds.) TACAS, volume 6605 of LNCS, pp. 112–127. Springer, Berlin (2011)

    Google Scholar 

  17. Fujita, M., Mcgeer, P.C., Yang, J.C.Y.: Multi-terminal binary decision diagrams: an efficient data structure for matrix representation. Form. Methods Syst. Des. V 10(2/3), 149–169 (1997)

    Article  Google Scholar 

  18. Gimbert, H.: Pure stationary optimal strategies in Markov decision processes. In: STACS’07, pp. 200–211. Springer (2007)

  19. Hachtel, G., Macii, E., Pardo, A., Somenzi, F.: Markovian analysis of large finite state machines. Comput. Aided Des. Integr. Circuits Syst. IEEE Trans. 15(12), 1479–1493 (1996)

    Article  Google Scholar 

  20. Haverkort, B.R.: Performance of Computer Communication Systems—A Model-Based Approach. Wiley, New York (1998)

    Book  Google Scholar 

  21. Isbell, J.R., Marlow, W.H.: Attrition games. Nav. Res. Logist. Q. 3, 71–94 (1956)

    Article  MathSciNet  MATH  Google Scholar 

  22. Kemeny, J., Snell, J., Knapp, A.: Denumerable Markov Chains, 2nd edn. Springer, New York (1976)

    Book  MATH  Google Scholar 

  23. Kwiatkowska, M., Norman, G., Parker, D.: Probabilistic symbolic model checking with PRISM: a hybrid approach. In: Katoen, J.-P., Stevens, P. (eds.) TACAS, volume 2280 of LNCS, pp. 52–66. Springer, Berlin (2002)

    Google Scholar 

  24. Kwiatkowska, M., Norman, G., Parker, D.: PRISM 4.0: Verification of probabilistic real-time systems. In: Proceedings of 23rd International Conference on Computer Aided Verification (CAV’11), volume 6806 of LNCS, pp. 585–591. Springer (2011)

  25. Kwiatkowska, M., Norman, G., Parker. D.: The PRISM benchmark suite. In: Proceedings of 9th International Conference on Quantitative Evaluation of Systems (QEST’12), pp. 203–204. IEEE CS Press (2012)

  26. Kwiatkowska, M., Norman, G., Parker, D., Sproston, J.: Performance analysis of probabilistic timed automata using digital clocks. Form. Methods Syst. Des. 29(1), 33–78 (2006)

    Article  MATH  Google Scholar 

  27. Lehmann, D.J., Rabin, M. O.: On the advantages of free choice: a symmetric and fully distributed solution to the dining philosophers problem. In: POPL (1981)

  28. Manna, Z., Pnueli, A.: Temporal Verification of Reactive Systems—Safety. Springer, Berlin (1995)

    Book  MATH  Google Scholar 

  29. Norman, G., Parker, D., Kwiatkowska, M., Shukla, S.K., Gupta, R.: Using probabilistic model checking for dynamic power management. Formal Asp. Comput. 17(2), 160–176 (2005)

    Article  MATH  Google Scholar 

  30. Norris, J.: Markov Chains. Cambridge University Press, Cambridge (2003)

    MATH  Google Scholar 

  31. Parker, D.: Implementation of Symbolic Model Checking for Probabilistic Systems. PhD thesis, University of Birmingham (2002)

  32. Parr, R., Russell, S.J.: Reinforcement learning with hierarchies of machines. In: Jordan, M.I., Kearns, M.J., Solla, S.A. (eds.) NIPS. The MIT Press, Cambridge (1997)

    Google Scholar 

  33. Pnueli, A.: The temporal logic of programs. In: FOCS, pp. 46–57. IEEE Computer Society (1977)

  34. Puterman, M.L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley-Interscience, Chichester (1994)

    Book  MATH  Google Scholar 

  35. Rabin, M.O.: N-process mutual exclusion with bounded waiting by 4 \(log_2 n\)-valued shared variable. J. Comput. Syst. Sci. 25(1), 66–75 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  36. Tijms, H.C.: A First Course in Stochastic Models. Wiley, Chichester (2003)

    Book  MATH  Google Scholar 

  37. von Essen, C., Jobstmann, B.: Synthesizing systems with optimal average-case behavior for ratio objectives. In: Reich, J., Finkbeiner, B. (eds.) Proceedings International Workshop on Interactions, Games and Protocols, iWIGP 2011, Saarbrücke, Germany, 27th March 2011. EPTCS, vol. 50, pp. 17–32 (2011)

  38. von Essen, C., Jobstmann, B.: Synthesizing efficient controllers. In: International Conference on Verification, Model Checking, and Abstract Interpretation (VMCAI), pp. 428–444 (2012)

  39. Wimmer, R., Braitling, B., Becker, B., Hahn, E.M., Crouzen, P., Hermanns, H., Dhama, A., Theel, O.: Symblicit calculation of long-run averages for concurrent probabilistic systems. In: QEST, pp. 27–36. IEEE Computer Society (2010)

  40. Wimmer, R., Derisavi, S., Hermanns, H.: Symbolic partition refinement with dynamic balancing of time and space. In: QEST, pp. 65–74. IEEE Computer Society (2008)

  41. Yue, H., Bohnenkamp, H.C., Katoen, J.-P.: Analyzing energy consumption in a gossiping MAC protocol. In: Müller-Clostermann, B., Echtle, K., Rathgeb, E.P. (eds.) MMB/DFT, volume 5987 of LNCS, pp. 107–119. Springer (2010)

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Barbara Jobstmann.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

von Essen, C., Jobstmann, B., Parker, D. et al. Synthesizing efficient systems in probabilistic environments. Acta Informatica 53, 425–457 (2016). https://doi.org/10.1007/s00236-015-0237-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00236-015-0237-y

Navigation