Abstract
Statistical testing of pseudorandom number generators (PRNGs) is indispensable for their evaluation. A common difficulty among statistical tests is how we consider the resulting probability values (p-values). When a suspicious p-value, such as \(10^{-3}\), is observed, it is unclear whether it is due to a defect of the PRNG or merely by chance. In order to avoid such a difficulty, testing the uniformity of p-values provided by a certain statistical test is widely used. This procedure is called a two-level test. The sample size at the second level requires a careful choice because too large sample leads to the erroneous rejection, but this choice is usually done through experiments. In this paper, we propose a criterion of an appropriate sample size when we use the Frequency test, the Binary Matrix Rank test and the Runs test at the first level in the NIST test suite. This criterion is based on \(\chi ^2\)-discrepancy, which measures the differences between the expected distribution of p-values and the exact distribution of those. For example, when we use the Frequency test with the sample size \(10^6\) as the first level test, an upper bound on the sample size at the second level derived by our criterion is 125, 000.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bassham III, L.E., Rukhin, A.L., Soto, J., Nechvatal, J.R., Smid, M.E., Barker, E.B., Leigh, S.D., Levenson, M., Vangel, M., Banks, D.L., Heckert, N.A., Dray, J.F., Vo, S.: Sp 800-22 rev. 1a. a statistical test suite for random and pseudorandom number generators for cryptographic applications. Technical report, National Institute of Standards & Technology, Gaithersburg, MD, United States (2010)
Gibbons, J.D., Chakraborti, S.: Nonparametric Statistical Inference, 5th edn. Chapman and Hall/CRC, Boca Raton (2010)
L’Ecuyer, P.: Software for uniform random number generation: distinguishing the good and the bad. In: Proceedings of the 2001 Winter Simulation Conference, vol. 95–105. IEEE Press (2001). http://dl.acm.org/citation.cfm?id=564124.564139
L’Ecuyer, P., Simard, R.: TestU01: a C library for empirical testing of random number generators. ACM Trans. Math. Softw. 33(4), Art. 22, 40 (2007)
L’Ecuyer, P., Cordeau, J.F., Simard, R.: Close-point spatial tests and their application to random number generators. Op. Res. 48(2), 308–317 (2000). https://doi.org/10.1287/opre.48.2.308.12385
L’Ecuyer, P., Simard, R., Wegenkittl, S.: Sparse serial tests of uniformity for random number generators. SIAM J. Sci. Comput. 24(2), 652–668 (2002)
Marsaglia, G., Tsay, L.H.: Matrices and the structure of random number sequences. Linear Algebra Appl. 67, 147–156 (1985). https://doi.org/10.1016/0024-3795(85)90192-2
Matsumoto, M., Nishimura, T.: Mersenne twister: a 623-dimensionally equidistributed uniform pseudo-random number generator. ACM Trans. Model. Comput. Simul. 8(1), 3–30 (1998)
Matsumoto, M., Nishimura, T.: A nonempirical test on the weight of pseudorandom number generators. In: Monte Carlo and quasi-Monte Carlo methods 2000, pp. 381–395 (2000). Springer, Berlin (2002)
Pareschi, F., Rovatti, R., Setti, G.: Second-level NIST randomness tests for improving test reliability. In: 2007 IEEE International Symposium on Circuits and Systems, pp. 1437–1440 (2007). https://doi.org/10.1109/ISCAS.2007.378572
Pareschi, F., Rovatti, R., Setti, G.: On statistical tests for randomness included in the NIST SP800-22 test suite and based on the binomial distribution. IEEE Trans. Inf. Forensics Secur. 7(2), 491–505 (2012). https://doi.org/10.1109/TIFS.2012.2185227
Simard, R., L’Ecuyer, P.: Computing the two-sided Kolmogorov-Smirnov distribution. J. Stat. Softw. 39(11), 1–18 (2011). https://doi.org/10.18637/jss.v039.i11. https://www.jstatsoft.org/v039/i11
Acknowledgements
We are thankful to Editor Professor Art Owen and the referees, who informed of numerous improvements on the manuscript. This research has been supported in part by JSPS Grant-In-Aid #26310211, #15K13460, #16K13750, #17K14234, #18K03213, and JST CREST “Theory of hyper uniformity and its development in randomness appeared in sciences.”
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Haramoto, H., Matsumoto, M. (2018). A Method to Compute an Appropriate Sample Size of a Two-Level Test for the NIST Test Suite. In: Owen, A., Glynn, P. (eds) Monte Carlo and Quasi-Monte Carlo Methods. MCQMC 2016. Springer Proceedings in Mathematics & Statistics, vol 241. Springer, Cham. https://doi.org/10.1007/978-3-319-91436-7_15
Download citation
DOI: https://doi.org/10.1007/978-3-319-91436-7_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-91435-0
Online ISBN: 978-3-319-91436-7
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)