Komsan Srivisut
ABSTRACT
Understanding extreme execution times is of great importance in gaining assurance in real-time embedded systems. The standard benchmark for dynamic testing---uniform randomised testing---is inadequate for reaching extreme execution times in these systems. Metaheuristics have been shown to be an effective means of directly searching for inputs with such behaviours but the increasing complexity of modern systems is now posing challenges to the effectiveness of this approach. The research reported in this paper investigates the use of metaheuristic search to discover biased random sampling regimes. Rather than search for test inputs, we search for distributions of test inputs that are then sampled. The search proceeds to discover and exploit relationships between test input variables, leading to sampling regimes where the distribution of a sampled parameter depends on the values of previously sampled input parameters. Our results show that test vectors indirectly generated from our dependent approach produce significantly more extreme (longer) execution times than those generated by direct metaheuristic searches.
- Wasif Afzal, Richard Torkar, and Robert Feldt. 2009. A systematic review of search-based testing for non-functional system properties. Information and Software Technology 51, 6 (2009), 957 -- 976. Google Scholar
Digital Library
- Ilhem Boussad, Julien Lepagnot, and Patrick Siarry. 2013. A survey on optimization metaheuristics. Information Sciences 237 (2013), 82 -- 117. Prediction, Control and Diagnosis using Advanced Neural Computations. Google ScholarDigital Library
- A. Burns and J. A. McDermid. 1994. Real-time safety-critical systems: analysis and synthesis. Software Engineering Journal 9, 6 (Nov 1994), 267--281.Google ScholarCross Ref
- Gunter Dueck and Tobias Scheuer. 1990. Threshold accepting: A general purpose optimization algorithm appearing superior to simulated annealing. J. Comput. Phys. 90, 1 (1990), 161 -- 175. Google ScholarDigital Library
- A. Engel. 2010. Verification, Validation and Testing of Engineered Systems. John Wiley & Sons. https://books.google.co.uk/books?id=H6N2CgAAQBAJGoogle Scholar
- M. Galassi, J. Davies, J. Theiler, B. Gough, G. Jungman, P. Alken, M. Booth, F. Rossi, and R. Ulerich. 2015. GNU Scientific Library Reference Manual (third ed.). The GSL Team.Google Scholar
- Patrick Graydon and Iain Bate. 2014. Realistic safety cases for the timing of systems. Comput. J. 57, 5 (2014), 759--774.Google ScholarCross Ref
- M. Harman, Y. Jia, and Y. Zhang. 2015. Achievements, Open Problems and Challenges for Search Based Software Testing. In 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST). 1--12.Google Scholar
- Courtney E. Howard. 2012. Modern microprocessors: Robust, high-performance aerospace and defense systems harness the power of innovative microprocessors. (March 2012). http://www.militaryaerospace.com/articles/print/volume-23/issue-3/technology-focus/modern-microprocessors.html/ {Online; posted 1-March-2012}.Google Scholar
- S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi. 1983. Optimization by Simulated Annealing. Science 220, 4598 (1983), 671--680. arXiv:http://science.sciencemag.org/content/220/4598/671.full.pdfGoogle Scholar
- M. Levy and T. M. Conte. 2009. Embedded Multicore Processors and Systems. IEEE Micro 29, 3 (May 2009), 7--9. Google ScholarDigital Library
- Sean Luke. 2013. Essentials of Metaheuristics (second ed.). Lulu. Available for free at http://cs.gmu.edu/~sean/book/metaheuristics/.Google Scholar
- Sean Luke. 2017. ECJ 24 and 25: A Java-based Evolutionary Computation Research System. (2017). http://cs.gmu.edu/-eclab/projects/ecj/ {Online; accessed 21-October-2017}.Google Scholar
- P. McMinn. 2011. Search-Based Software Testing: Past, Present and Future. In 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops. 153--163. Google ScholarDigital Library
- Matthew Patrick, Rob Alexander, Manuel Oriol, and John A. Clark. 2015. Subdomain-based test data generation. Journal of Systems and Software 103 (2015), 328 -- 342. Google ScholarDigital Library
- Hartmut Pohlheim and Joachim Wegener. 1999. Testing the Temporal Behavior of Real-time Software Modules Using Extended Evolutionary Algorithms. In Proceedings of the 1st Annual Conference on Genetic and Evolutionary Computation - Volume 2 (GECCO'99). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1795--1795. http://dl.acm.org/citation.cfm?id=2934046.2934210 Google ScholarDigital Library
- Freescale semiconductor. 2011. P4080 Development System User's Guide. Technical Report.Google Scholar
- Freescale semiconductor. 2014. Running AMP, SMP, or BMP Mode for Multicore Embedded Systems. Technical Report.Google Scholar
- David R. White. 2012. Software review: the ECJ toolkit. Genetic Programming and Evolvable Machines 13,1 (2012), 65--67. Google ScholarDigital Library
- Reinhard Wilhelm, Jakob Engblom, Andreas Ermedahl, Niklas Holsti, Stephan Thesing, David Whalley, Guillem Bernat, Christian Ferdinand, Reinhold Heckmann, Tulika Mitra, Frank Mueller, Isabelle Puaut, Peter Puschner, Jan Staschulat, and Per Stenström. 2008. The Worst-case Execution-time Problem --- Overview of Methods and Survey of Tools. ACM Trans. Embed. Comput. Syst. 7, 3, Article 36 (May 2008), 53 pages Google ScholarDigital Library
Index Terms
- Dependent input sampling strategies: using metaheuristics for generating parameterised random sampling regimes
Recommendations
One-test-at-a-time heuristic search for interaction test suites
GECCO '07: Proceedings of the 9th annual conference on Genetic and evolutionary computationAlgorithms for the construction of software interaction test suites have focussed on the special case of pairwise coverage; less is known about efficiently constructing test suites for higher strength coverage. The combinatorial growth of t-tuples ...
A New Swarm-Based Simulated Annealing Hyper-Heuristic Algorithm for Clustering Problem
AbstractHyper-heuristic is a new generation of heuristic algorithms which include a set of heuristics that could generate a set of heuristics or automate the process of selecting a batch of heuristics to address different optimization problems. Most of ...
Comparison of optimality and robustness between SA, TS and GRASP metaheuristics in FJSP problem
ICIC'10: Proceedings of the 6th international conference on Advanced intelligent computing theories and applications: intelligent computingThe paper presents the comparison of optimality and robustness between Greedy Randomized Adaptive Search Procedure (GRASP), Simulated Annealing (SA) and Tabu Search (TS) metaheuristics in solving of flexible job shop problem (FJSP). Each of the ...
Comments