Skip to main content
SpringerLink
Log in

Automated quantized inference for probabilistic programs with AQUA

  • S.I. : ATVA 2021
  • Published:
Innovations in Systems and Software Engineering Aims and scope Submit manuscript

Abstract

We present AQUA, a new probabilistic inference algorithm that operates on probabilistic programs with continuous posterior distributions. AQUA approximates programs via an efficient quantization of the continuous distributions. It represents the distributions of random variables using quantized value intervals (Interval Cube) and corresponding probability densities (Density Cube). AQUA’s analysis transforms Interval and Density Cubes to compute the posterior distribution with bounded error. We also present an adaptive algorithm for selecting the size and the granularity of the Interval and Density Cubes. We evaluate AQUA on 24 programs from the literature. AQUA solved all of 24 benchmarks in less than 43s (median 1.35s) with a high level of accuracy. We show that AQUA is more accurate than state-of-the-art approximate algorithms (Stan’s NUTS and ADVI) and supports programs that are out of reach of exact inference tools, such as PSI and SPPL.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. We support common continuous distributions including Normal, Uniform, Exponential, Beta, Gamma, Student’s T, Laplace, Triangular, and any mixture of the above distributions.

References

  1. Carpenter B, Gelman A, Hoffman M, Lee D et al (2016) Stan: a probabilistic programming language. JSTATSOFT 20(2)

  2. Albarghouthi A, D’Antoni L, Drews S, Nori A (2017) Fairsquare: probabilistic verification of program fairness (OOPSLA)

  3. Sweet I, Trilla JMC, Scherrer C, Hicks M, Magill S (2018) What’s the over/under? probabilistic bounds on information leakage. POST

  4. Pardo R, Rafnsson W, Probst CW, Wasowski A (2021) Privug: using probabilistic programming for quantifying leakage in privacy risk analysis. In: European symposium on research in computer security. Springer, pp 417–438

  5. Huang Z, Wang Z, Misailovic S (2018) PSense: automatic sensitivity analysis for probabilistic programs. ATVA

  6. Aguirre A, Barthe G, Hsu J, Kaminski BL, Katoen J-P, Matheja C (2021) A pre-expectation calculus for probabilistic sensitivity. POPL

  7. Gehr T, Misailovic S, Vechev M (2016) PSI: Exact symbolic inference for probabilistic programs. CAV

  8. Narayanan P, Carette J, Romano W, Shan C-c, Zinkov R (2016) Probabilistic inference by program transformation in hakaru (system description). FLOPS

  9. Saad FA, Rinard MC, Mansinghka VK (2021) SPPL: a probabilistic programming system with exact and scalable symbolic inference. PLDI

  10. Sankaranarayanan S, Chakarov A, Gulwani S (2013) Static analysis for probabilistic programs: inferring whole program properties from finitely many paths. PLDI

  11. Dutta S, Legunsen O, Huang Z, Misailovic S (2018) Testing probabilistic programming systems. In: FSE

  12. Dutta S, Zhang W, Huang Z, Misailovic S (2019) Storm: program reduction for testing and debugging probabilistic programming systems. In: FSE

  13. Gorinova MI, Gordon AD, Sutton C (2019) Probabilistic programming with densities in SlicStan: efficient, flexible, and deterministic. POPL

  14. Gelman A, Stern HS, Carlin JB, Dunson DB, Vehtari A, Rubin DB (2013) Bayesian data analysis. Chapman and Hall/CRC, Boca Raton

    Book  Google Scholar 

  15. Bissiri P, Holmes C, Walker S (2016) A general framework for updating belief distributions. J R Stat Soc Ser B Stat Methodol 78(5):1103

    Article  MathSciNet  Google Scholar 

  16. Goodman N, Tenenbaum J Probabilistic models of cognition. http://www.probmods.org

  17. Nishihara R, Minka T, Tarlow D (2013) Detecting parameter symmetries in probabilistic models. arXiv preprint arXiv:1312.5386

  18. Neal RM (2012) Bayesian learning for neural networks. Springer, Toronto

    Google Scholar 

  19. https://github.com/stan-dev/example-models (2018)

  20. Wang Y, Kucukelbir A, Blei DM (2017) Robust probabilistic modeling with bayesian data reweighting. ICML

  21. Wang C, Blei DM (2018) A general method for robust Bayesian modeling. Bayesian analysis 13(4):1159–1187

    Article  MathSciNet  Google Scholar 

  22. Laurel J, Misailovic S (2020) Continualization of probabilistic programs with correction. ESOP

  23. Goodman N, Mansinghka V, Roy DM, Bonawitz K, Tenenbaum JB (2012) Church: a language for generative models. arXiv preprint arXiv:1206.3255

  24. Gilks WR, Thomas A, Spiegelhalter DJ (1994) A language and program for complex Bayesian modelling. The Statistician

  25. Pfeffer A (2001) Ibal: a probabilistic rational programming language. In: Proceedings of the 17th international joint conference on artificial intelligence-volume 1. Morgan Kaufmann Publishers Inc, pp 733–740

  26. Nori AV, Hur C-K, Rajamani SK, Samuel S (2014) R2: an efficient MCMC sampler for probabilistic programs. In: AAAI

  27. Wood F, van de Meent JW, Mansinghka V (2014) A new approach to probabilistic programming inference. In: AISTATS

  28. Mansinghka V, Selsam D, Perov Y (2014) Venture: a higher-order probabilistic programming platform with programmable inference. arXiv preprint arXiv:1404.0099

  29. Goodman ND, Stuhlmüller A (2014) The design and implementation of probabilistic programming languages. Retrieved 2015/1/16, from http://dippl.org

  30. Tran D, Kucukelbir A, Dieng AB, Rudolph M, Liang D, Blei DM (2016) Edward: a library for probabilistic modeling, inference, and criticism. arXiv

  31. Pyro. http://pyro.ai (2018)

  32. Minka T, Winn JM, Guiver JP, Webster S, Zaykov Y, Yangel B, Spengler A, Bronskill J (2013) Infer.NET 2.5. Microsoft Research Cambridge. http://research.microsoft.com/infernet

  33. Holtzen S, Van den Broeck G, Millstein T (2020) Scaling exact inference for discrete probabilistic programs. OOPSLA

  34. Borges M, Filieri A, d’Amorim M, Păsăreanu CS, Visser W (2014) Compositional solution space quantification for probabilistic software analysis. PLDI

  35. Luo Y, Filieri A, Zhou Y (2020) Sympais: Symbolic parallel adaptive importance sampling for probabilistic program analysis. arXiv preprint arXiv:2010.05050

  36. Shachter RD, D’Ambrosio B, Del Favero B (1990) Symbolic probabilistic inference in belief networks. In: AAAI, vol. 90, pp 126–131

  37. Chang K-C, Fung R (1995) Symbolic probabilistic inference with both discrete and continuous variables. IEEE Trans Syst, Man Cybern 25(6):910–916

    Article  Google Scholar 

  38. Moral S, Rumí R, Salmerón A (2001) Mixtures of truncated exponentials in hybrid bayesian networks. In: Symbolic and quantitative approaches to reasoning with uncertainty. Springer, Barcelona, Spain, pp 156–167

  39. Shenoy PP, West JC (2011) Inference in hybrid Bayesian networks using mixtures of polynomials. Int J Approx Reason 52(5)

  40. Sanner S, Abbasnejad E (2012) Symbolic variable elimination for discrete and continuous graphical models. In: AAAI

  41. Huang Z, Dutta S, Misailovic S (2021) Aqua: Automated quantized inference for probabilistic programs. In: International symposium on automated technology for verification and analysis. Springer, pp 229–246

Download references

Acknowledgements

This research was supported in part by NSF Grants No. CCF-1846354, CCF-1956374, CCF-2008883, and Facebook PhD Fellowship. The previous version of this work appeared in ATVA 2021 [41].

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Illinois at Urbana-Champaign, 201 North Goodwin Avenue, Urbana, 61801, IL, USA

    Zixin Huang, Saikat Dutta & Sasa Misailovic

Authors
  1. Zixin Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Saikat Dutta
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sasa Misailovic
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zixin Huang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, Z., Dutta, S. & Misailovic, S. Automated quantized inference for probabilistic programs with AQUA. Innovations Syst Softw Eng 18, 369–384 (2022). https://doi.org/10.1007/s11334-021-00433-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11334-021-00433-3

Keywords

Search

Navigation