Skip to main content
SpringerLink
Log in

Data-Driven Prediction of Confidence for EVAR in Time-Varying Datasets

  • Chapter
  • First Online:
Handbook of Dynamic Data Driven Applications Systems

Abstract

The key challenge for Dynamic Data Driven Applications Systems (DDDAS) operating in time-varying environments is to predict when the learned model may lose relevance. If the learned model loses relevance, then the autonomous system is at risk of making wrong decisions. The Entropic Value at Risk (EVAR) is a computationally efficient and coherent risk measure that can be utilized to quantify this model relevance. The value of EVAR is calculated with respect to an assumed confidence value; e.g., a 90% confidence may be desired for robust decision-making. Without a model on the confidence value directly, there is no guarantee that EVAR calculations will reflect the uncertainty present in dynamic real-world environments. In this paper, we present a Bayesian model and learning algorithms to predict the state-dependent confidence necessary for calculating the EVAR in time-varying datasets. We discuss applications of the data-driven EVAR to a monitoring problem, in which a DDDAS agent has to chose a set of sensing locations in order to maximize the expected EVAR of the acquired data. In this way, the DDDAS agent can learn a model on an underlying phenomenon of interest by prioritizing the areas where the model is most likely incorrect but highly valued. We empirically demonstrate the efficacy of the presented model and learning algorithms on five real-world datasets. We show that, overall, the EVAR-Real-time Adative Prediction of Time-varying and Obscure Rewards (EVAR-RAPTOR) algorithm outperforms EVAR-Predicted Information Gain* (EVAR-PIG*) as well as naive searches such as random and sequential search across these five real-world datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. F. Darema, Dynamic data driven applications systems: a new paradigm for application simulations and measurements, in International Conference on Computational Science, Krakow (Springer, 2004), pp. 662–669 https://link.springer.com/book/10.1007/b97988

    Chapter  Google Scholar 

  2. F. Darema, Grid computing and beyond: the context of dynamic data driven applications systems. Proc. IEEE 93(3), 692–697 (2005)

    Article  Google Scholar 

  3. F. Darema, DDDAS, a key driver for large-scale-big-data and large-scale-big-computing. Proc. Comput. Sci. 51, 2463 (2015)

    Article  Google Scholar 

  4. A. Hobson, A new theorem of information theory. J. Stat. Phys. 1(3), 383–391 (1969)

    Article  MathSciNet  Google Scholar 

  5. A. Ahmadi-Javid, Entropic value-at-risk: a new coherent risk measure. J. Optim. Theory Appl. 155(3), 1105–1123 (2012)

    Article  MathSciNet  Google Scholar 

  6. R.T. Rockafellar, S. Uryasev, Optimization of conditional value-at-risk. J. Risk 2, 21–42 (2000)

    Article  Google Scholar 

  7. T. Kim, A.V. Nefian, M.J. Broxton, Photometric recovery of ortho-images derived from apollo 15 metric camera imagery, in Advances in Visual Computing (Springer, 2009), pp. 700–709

    Google Scholar 

  8. T. Kim, A.V. Nefian, M.J. Broxton, Photometric recovery of apollo metric imagery with lunar-lambertian reflectance. Electron. Lett. 46(9), 631–633 (2010)

    Article  Google Scholar 

  9. D.Y. Little, F.T. Sommer, Learning and exploration in action-perception loops. Front. Neural Circuits 7, 1–19 (2013)

    Article  Google Scholar 

  10. A.M. Axelrod, Learning to exploit time-varying heterogeneity in distributed sensing using the information exposure rate. Master’s thesis, Oklahoma State University, 2015

    Google Scholar 

  11. C.C. Heyde, E. Seneta, Studies in the history of probability and statistics. XXXI. The simple branching process, a turning point test and a fundamental inequality: a historical note on I.J. bienaymé. Biometrika 59(3), 680–683 (1972)

    MathSciNet  MATH  Google Scholar 

  12. D. Applebaum, Lévy processes: from probability to finance and quantum groups. Not. AMS 51(11), 1336–1347 (2004)

    MATH  Google Scholar 

  13. M. Tokic, Adaptive ε-greedy exploration in reinforcement learning based on value differences, in KI 2010: Advances in Artificial Intelligence (Springer, 2010), pp. 203–210

    Google Scholar 

  14. W. Jouini, D. Ernst, C. Moy, J. Palicot, Upper confidence bound based decision making strategies and dynamic spectrum access, in 2010 IEEE International Conference on Communications (ICC), Cape Town (IEEE, 2010), pp. 1–5 http://icc2010.ieee-icc.org/

  15. A. Carpentier, A. Lazaric, M. Ghavamzadeh, R. Munos, P. Auer, Upper-confidence-bound algorithms for active learning in multi-armed bandits, in Algorithmic Learning Theory (Springer, 2011) pp. 189–203

    Google Scholar 

  16. C. Gehring, D. Precup, Smart exploration in reinforcement learning using absolute temporal difference errors, in Proceedings of the 2013 International Conference on Autonomous Agents and Multi-agent Systems, Saint Paul. International Foundation for Autonomous Agents and Multiagent Systems, 2013, pp. 1037–1044 https://dl.acm.org/citation.cfm?id=2484920

  17. D. Russo, B. Van Roy, An information-theoretic analysis of Thompson sampling. arXiv preprint arXiv:1403.5341 (2014)

    Google Scholar 

  18. J.C. Principe, D. Xu, Information-theoretic learning using Renyis quadratic entropy, in Proceedings of the First International Workshop on Independent Component Analysis and Signal Separation, Aussois, France, 1999, pp. 407–412 https://scholar.googleusercontent.com/scholar.bib?q=info:IFH_36rOMQgJ:scholar.google.com/&output=citation&scisig=AAGBfm0AAAAAW3GA7DJPrAxvpEvPRhC_z9ovoHvVeTWe&scisf=4&ct=citation&cd=-1&hl=en&scfhb=1

  19. P. Reverdy, R.C. Wilson, P. Holmes, N.E. Leonard, Towards optimization of a human-inspired heuristic for solving explore-exploit problems, in CDC, Maui, 2012, pp. 2820–2825 http://www.ieeecss.org/CAB/conferences/cdc2012/

  20. D. Ryabko, Time-series information and learning, in 2013 IEEE International Symposium on Information Theory Proceedings (ISIT), Istanbul (IEEE, 2013), pp. 1392–1395 http://www.proceedings.com/19451.html

  21. S.G. Nora Ayanian, Persistent monitoring of stochastic spatio-temporal phenomena with a small team of robots, in Proceedings of Robotics: Science and Systems, Berkeley, California, USA, July 2014 http://rll.berkeley.edu/RSS2014/

    Google Scholar 

  22. D. Russo, B. Van Roy, Learning to optimize via information-directed sampling, in Advances in Neural Information Processing Systems, Montreal, 2014, pp. 1583–1591 https://nips.cc/Conferences/2014/

  23. J.M. Hernández-Lobato, M.A. Gelbart, R.P. Adams, M.W. Hoffman, Z. Ghahramani, A general framework for constrained bayesian optimization using information-based search. arXiv preprint arXiv:1511.09422 (2015)

    Google Scholar 

  24. H. Ding, D.A. Castañón, Optimal solutions for adaptive search problems with entropy objectives. arXiv preprint arXiv:1508.04127 (2015)

    Google Scholar 

  25. A.M. Axelrod, S.A. Karaman, G.V. Chowdhary, Exploitation by informed exploration between isolated operatives for information-theoretic data harvesting, in Conference on Decision and Control (CDC), Osaka, vol. 54, 2015

    Google Scholar 

  26. A. Axelrod, G. Chowdhary, Uninformed-to-informed exploration in unstructured real-world environments, in Self-Confidence In Autonomous Systems, Washington, DC (AAAI, 2015)

    Google Scholar 

  27. A. Axelrod, G. Chowdhary, A hybridized bayesian parametric-nonparametric approach to the pure exploration problem, in Bayesian Nonparametrics: The Next Generation, Montreal. Neural Information Processing Systems, 2015

    Google Scholar 

  28. A. Axelrod, G. Chowdhary, The Explore-Exploit Dilemma in Nonstationary Decision Making under Uncertainty, 1st edn. (Springer, New York, 2015), pp. 2198–4182

    Google Scholar 

  29. P. Bodik, W. Hong, C. Guestrin, S. Madden, M. Paskin, R. Thibaux, Intel lab data. Technical report, Intel Berkely Research Lab, Feb 2004

    Google Scholar 

  30. P. Berrisford, D.P. Dee, K. Fielding, M. Fuentes, P. Kallberg, S. Kobayashi, S. Uppala, The era-interim archive (European Center for Medium Range Weather Forecasts, Reading, 2009)

    Google Scholar 

  31. J. Haslett, A.E. Raftery, Ireland wind data set. Technical report, Trinity College and University of Washington, pp. 1961–1978

    Google Scholar 

  32. M. Widmann, C.S. Bretherton, Validation of mesoscale precipitation in the NCEP reanalysis using a new gridcell dataset for the northwestern united states. J. Clim. 13(11), 1936–1950 (2000)

    Article  Google Scholar 

  33. Clean Air Status and Trends Network (CASTNET), Accessed Hourly Ozone data at www.epa.gov/castnet on Sept 2015

  34. D.P. Dee, S.M. Uppala, A.J. Simmons, P. Berrisford, P. Poli, S. Kobayashi, U. Andrae, M.A. Balmaseda, G. Balsamo, P. Bauer et al., The era-interim reanalysis: configuration and performance of the data assimilation system. Q. J. R. Meteorol. Soc. 137(656), 553–597 (2011)

    Article  Google Scholar 

  35. A. Gelman, J.B. Carlin, H.S. Stern, D.B. Dunson, A. Vehtari, D.B. Rubin, Bayesian data analysis, 3rd edn. (CRC Press, Boca Raton, 2013)

    MATH  Google Scholar 

  36. J. Duchi, Derivations for Linear Algebra and Optimization (Berkeley, 2007)

    Google Scholar 

  37. M.J Schervish, Theory of Statistics (Springer, New York, 2012)

    Google Scholar 

Download references

Acknowledgements

This work is sponsored by the Air Force Office of Scientific Research Award Number FA9550-14-1-0399, and the Air Force Office of Scientific Research Young Investigator Program Number FA9550-15-1-0146.

Author information

Authors and Affiliations

  1. Coordinated Science Lab, University of Illinois at Urbana-Champaign, Urbana, IL, USA

    Allan Axelrod & Girish Chowdhary

  2. Massachusetts Institute of Technology, Cambridge, MA, USA

    Luca Carlone & Sertac Karaman

Authors
  1. Allan Axelrod
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luca Carlone
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Girish Chowdhary
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sertac Karaman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Allan Axelrod .

Editor information

Editors and Affiliations

  1. Air Force Office of Scientific Research, Air Force Research Laboratory, Arlington, VA, USA

    Erik Blasch

  2. Earth, Atmospheric and Planetary Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA

    Sai Ravela

  3. Information Directorate, Air Force Research Laboratory, Rome, NY, USA

    Alex Aved

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Axelrod, A., Carlone, L., Chowdhary, G., Karaman, S. (2018). Data-Driven Prediction of Confidence for EVAR in Time-Varying Datasets. In: Blasch, E., Ravela, S., Aved, A. (eds) Handbook of Dynamic Data Driven Applications Systems. Springer, Cham. https://doi.org/10.1007/978-3-319-95504-9_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-95504-9_16

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-95503-2

  • Online ISBN: 978-3-319-95504-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation