Abstract
Given a model with multiple input parameters, and multiple possible sources for collecting data for those parameters, a data collection strategy is a way of deciding from which sources to sample data, in order to reduce the variance on the output of the model. Cain and Van Moorsel have previously formulated the problem of optimal data collection strategy, when each parameter can be associated with a prior normal distribution, and when sampling is associated with a cost. In this paper, we present ADaCS, a new tool built as an extension of PRISM, which automatically analyses all possible data collection strategies for a model, and selects the optimal one. We illustrate ADaCS on attack trees, which are a structured approach to analyse the impact and the likelihood of success of attacks and defenses on computer and socio-technical systems. Furthermore, we introduce a new strategy exploration heuristic that significantly improves on a brute force approach.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
We use here a prefix notation to avoid any ambiguity.
References
Ascough, J., Green, T., Ma, L., Ahuja, L.: Key criteria and selection of sensitivity analysis methods applied to natural resource models. In: Proceedings of the International Congress on Modeling and Simulation (2005)
Aslanyan, Z., Nielson, F., Parker, D.: Quantitative verification and synthesis of attack-defence scenarios. In: Proceedings of the 29th IEEE Computer Security Foundations Symposium (CSF 2016), pp. 105–119, June 2016
Beautement, A., et al.: Modelling the human and technological costs and benefits of USB memory stick security. In: Johnson, M.E. (ed.) Managing Information Risk and the Economics of Security, pp. 141–163. Springer, Boston (2009). doi:10.1007/978-0-387-09762-6_7
Cain, R., Van Moorsel, A.: Optimization of data collection strategies for model-based evaluation and decision-making. In: Proceedings of the 42nd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2012), pp. 1–10 (2012)
Chan, K., Saltelli, A., Tarantola, S.: Sensitivity analysis of model output: variance-based methods make the difference. In: Proceedings of the 29th Conference on Winter Simulation, pp. 261–268 (1997)
Cochran, W.: Sampling Techniques, 3rd edn. Wiley, New York (1977)
Deavours, D., Clark, G., Courtney, T., Daly, D., Derisavi, S., Doyle, J., Sanders, W., Webster, P.: The Möbius framework and its implementation. IEEE Trans. Softw. Eng. 28(10), 956–969 (2002)
Gamble, C., Pierce, K.: Design space exploration for embedded systems using co-simulation. In: Fitzgerald, J., Larsen, P.G., Verhoef, M. (eds.) Collaborative Design for Embedded Systems, pp. 199–222. Springer, Heidelberg (2014). doi:10.1007/978-3-642-54118-6_10
Gilmore, S., Hillston, J.: The PEPA workbench: a tool to support a process algebra-based approach to performance modelling. In: Haring, G., Kotsis, G. (eds.) TOOLS 1994. LNCS, vol. 794, pp. 353–368. Springer, Heidelberg (1994). doi:10.1007/3-540-58021-2_20
Haverkort, B., Meeuwissen, A.: Sensitivity and uncertainty analysis of Markov-reward models. IEEE Trans. Reliab. 44(1), 147–154 (1995)
Ho, Y.-C.: Introduction to special issue on dynamics of discrete event systems. Proc. IEEE 77(1), 3–6 (1989)
Kleijnen, J.P.: Sensitivity analysis and related analyses: a review of some statistical techniques. J. Stat. Comput. Simul. 57(1), 111–142 (1997)
Kordy, B., Mauw, S., Radomirović, S., Schweitzer, P.: Foundations of attack–defense trees. In: Degano, P., Etalle, S., Guttman, J. (eds.) FAST 2010. LNCS, vol. 6561, pp. 80–95. Springer, Heidelberg (2011). doi:10.1007/978-3-642-19751-2_6
Kwiatkowska, M., Norman, G., Parker, D.: PRISM 4.0: verification of probabilistic real-time systems. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 585–591. Springer, Heidelberg (2011). doi:10.1007/978-3-642-22110-1_47
Sahner, R., Trivedi, K., Puliafito, A.: Performance and Reliability Analysis of Computer Systems: An Example-Based Approach Using the SHARPE Software Package. Kluwer, Boston (1996)
Saltelli, A., Ratto, M., Andre, T., Campolongo, F., Cariboni, J., Gatelli, D., Saisana, M., Tarantola, S.: Global Sensitivity Analysis. The Primer. Wiley, West Sussex (2008)
Schneier, B.: Attack trees. Dr. Dobb’s J. 24(12), 21–29 (1999)
Acknowledgements
The research in this paper was supported in part by UK EPSRC through ‘Choice Architecture for Information Security’ (EP/K006568/1) within the Research Institute for the Science of Cyber Security and by US NSA through the University of Illinois at Urbana-Champaign lablet in ‘Science of Security Systems’ (H98230-14-C-0141).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Mace, J.C., Thekkummal, N., Morisset, C., Van Moorsel, A. (2017). ADaCS: A Tool for Analysing Data Collection Strategies. In: Reinecke, P., Di Marco, A. (eds) Computer Performance Engineering. EPEW 2017. Lecture Notes in Computer Science(), vol 10497. Springer, Cham. https://doi.org/10.1007/978-3-319-66583-2_15
Download citation
DOI: https://doi.org/10.1007/978-3-319-66583-2_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-66582-5
Online ISBN: 978-3-319-66583-2
eBook Packages: Computer ScienceComputer Science (R0)