Overview of Randomness Test on Cryptographic Algorithms

Randomness is an important research topic in the field of information security, especially in cryptography. Randomness test techniques are used to examine the quality of random numbers so that they meet the requirements of the application. The randomness of cryptographic algorithms is one of the key concerns in the algorithm design. People have put forward different standards and test requirements for the randomness of cryptographic algorithms, as well as developed corresponding randomness test kits. This paper analyzes the randomness test technologies of cryptographic algorithms and the general randomness test methods, and compares them on this basis. At the end, the actual application scenarios to apply these randomness test methods are discussed.


Introduction
Randomness is a form of contingency, which is uncertainty of each event in the event set with a certain probability. Combined with the characteristics of statistics and cryptanalysis, the random sequence should satisfy three conditions: 1. The sequence follows a uniform distribution. 2. Each element of the sequence is independent of each other. 3. The rest of the sequence can not be predicted from any sequence. According to the ways of generating random numbers, random numbers can be divided into two categories: true random numbers and pseudo-random numbers. True random number generators (RNGs) are composed of two parts: entropy source and algorithm post-processing. The common entropy sources include thermal noise amplification, oscillator sampling, chaotic circuit, light source noise, chaotic lasers and quantum noise. Pseudo random number generators (PRNGs) take a seed as input and generate an output sequence by function. Pseudo random number generator is widely used because of its convenient and fast characteristics.
At present, the algorithm randomness test is mainly realized by examining the randomness of the output sequence. The randomness test of the output sequence is through the sample output, which is statistically tested to detect whether it has the characteristics of the real random number sequence. The statistical test method used is hypothesis test, which assumes that the sequence to be tested is a real random sequence, and the hypothesis is H0, corresponding to another hypothesis H1, that is, the sequence to be tested is not a random sequence. In the test, selected statistics are calculated for the sample sequence, and then compared with threshold value. Because probability distribution is related to threshold value, if statistical value exceeds threshold value, this kind of small probability event should not occur from the perspective of hypothesis testing. If statistical value exceeds threshold value, H0 is rejected, otherwise H0 is accepted.
In this article, Chapter 2 gives the background of randomness test methods and standards and state quo of randomness tests. The general test methods on different cryptographic algorithms are also described. Chapter 3 introduces the common randomness test standards, methods and toolkits, while Chapter 4 describes various application scenarios of randomness test. The test on key schedule algorithm is to expand and combine X keys into Y sequences, then perform the local randomness test, so that the degree of statistical independence between all the sub keys is decided. For stream cipher, the output should be indistinguishable with the true random number, and the seed key should be evenly spread in the key stream, which is through transformation and local randomness test.

Randomness Test Standards, Methods and Tools
With development of the randomness test, test standards have been formulated in different countries. At present, SP 800-22 of NIST, GB/T 32915-2016 of SCA and AIS 20 / AIS 31 of BSI are the most well-known. To measure the test results, SP800-22 and GB/T 32915-2016 use the p-value approach, which is to compare the p-value and significance level α, and decide whether it indicates failure. However, AIS 20/AIS 31 applies the threshold approach, by comparing the observed value with the threshold corresponding to the significance level.
NIST SP800-22 adopted 2 approaches including (1) the examination of the proportion of sequences that pass a statistical test and (2) the distribution of p-values to check for uniformity. If either of these approaches fails, additional experiments should be conducted to see if the failure was a statistical anomaly or clear evidence of non-randomness.
GB/T 32915-2016 is used to test the binary sequences generated by commercial RNGs. If it passes all the 15 test items, then we can say it conforms with this standard. AIS 20/ AIS 31 is applied to test the RNGs and contains 9 test methods of T0-T8 with 2 test procedures of A and B. Process A includes T0-T5, and process B includes T6-T8. DRNGs, PTRNGs and NPTRNGs need to take the test procedure A. The goal of test procedure A is to check whether the random numbers are statistically inconspicuously. The Binary valued das-random numbers of PRTNGs need to select test procedure B, which is to ensure that the entropy per das-bit is sufficiently large.
The following table summarizes and compares the three standard test methods, with a total of 24 test methods [20]. The frequency test mainly depends on the proportion of "0" and "1" in the whole sequence. This test is the basis of the randomness test, which should be carried out first, and other tests should be carried out after the frequency test is passed. If the frequency test can not pass, then the sequence is not random without other tests. In a real random sequence, when the length tends to infinity, the number of "0" and "1" should be roughly equal, that is, half of each.
2. Run test Run length is a subsequence composed of continuous "0" or "1" in a sequence. The purpose of run detection is to determine whether the number of "1" runs of different lengths and the number of "0" runs are consistent with the expected value of the ideal random sequence. Specifically, this test means to determine whether the oscillation between such "0" and "1" sub blocks is too fast or too slow.
3.Rank test of binary matrix Binary matrix rank test is to divide binary sequence into several equal lengths and non overlapping matrices, and then count the rank distribution of all matrices to detect whether the linear independence of each matrix meets the requirements of random sequence, so as to judge whether the sequence has randomness.

Linear complexity test
The Linear complexity test is to divide binary sequence into several m-bit non overlapping bit blocks, and count the shortest length of linear feedback register (LFSR) of each bit block to detect whether the linear complexity distribution of all bit blocks meets the requirements of random sequence, so as to judge whether the sequence has randomness.

Discrete Fourier test
This test is mainly to see the peak height of the sequence after split step Fourier transform. The purpose is to detect the periodicity of the signal to be tested, so as to reveal the degree of deviation between it and the corresponding random signal. The method is to observe whether the number of peaks above the 95% threshold is significantly different from that below 5%. 6. Overlapping template matching test Overlapped subsequence test is to count the occurrence times of all m-bit overlapped subsequence patterns in binary sequence to detect whether 2^m patterns appear with equal probability, so as to judge whether the sequence is random.
7. Accumulation sum test Accumulation sum test is to construct overlapping increasing subsequences after the binary sequence is standardized, and calculate the absolute value of random walk of each subsequence to detect whether the maximum value conforms to the expected value of the random sequence, so as to judge whether the sequence has randomness.
These tests focus on different characteristics of non randomness which may exist in sequences. A test suite is formed by selecting from test set to test necessary randomness properties as much as possible, where the eliminating ability and mutual information entropy are concerned. The [19] gives the eliminating orders of the 15 test methods of NIST SP800-22. The mutual information entropy is denoted by I(X,Y)=H(X)+H(Y)-H(X,Y), meaning the actual amount of information that two tests can provide. In 2020, Karell Albo Jorge Augusto et al. [22] proposed a method of detecting statistical dependence by using mutual information. The main advantage of using mutual information is that it has the ability to detect non-linear correlations, while the linear correlation coefficients used in previous work cannot be detected. Dependency detection between statistical randomness tests allows one to discriminate statistical randomness tests that measure similar characteristics, and thus minimize the amount of statistical randomness tests that need to be used.
Randomness test toolkit is realized by integrating test methods into a program. Some of the well-know toolkits include the officially released NIST STS, ENT, Diehard, TestU01 and etc. NIST STS is published in NIST SP800-22, which is a personal tool for statistical testing of PRNGs. ENT pseudo-random number sequence test program takes the byte stream of the file to be tested as the input, including five test items: entropy, chi square test, arithmetic mean, Monte Carlo value method to calculate pi and sequence correlation coefficient. The program can be used to evaluate pseudo-random number generators for encryption and statistical sampling applications, compression algorithms, and other applications where file information density is of concern. The Diehard test is a set of statistical tests used to test the quality of random number generator, which contains a total of 15 kinds of tests, such as Birthday spacings, Monkey tests, The squeeze test and so on. TestU01 software library implements several types of random number generators, classical statistical tests of random number generators, some other tests and some original tests proposed in the literature. These tests can be applied to predefined generators in the library, user-defined generators, and random number streams stored in files. TestU01 provides multiple sets of tests, including smallcrush (10 tests), crush (96 tests) and bigcrush (160 tests). These test toolkits can be improved and optimized to adapt to different platforms or embeded systems.

Application Scenario Analysis of Randomness Test
Randomness test is mainly used for checking the quality of RNG and PRNG. In 2017, G.S. Karimovich et al. proposed RNG and PRNG based on computer sources, and used NIST STS tools to test their randomness respectively [23]. The results show that the 128 or 256 bit key generated by RNG has at least 88% probability of passing all the randomness tests; the 106 bit sequence generated by PRNG has nearly 100% probability of passing the rest tests except for random offset test and random offset variable test. Therefore, RNG and PRNG have good randomness and reliability.
In the test on cryptographic algorithms, we use the randomness test to decide whether the algorithm is secure. In 2008, J. Nakahara proposed the 3D-AES crypto system, which extends the block length and key length to 512 bits and the number of iterations to 22 rounds. The purpose is to encrypt large capacity data safely and efficiently [24]. However, in 2017, S. Ariffin and N.A.M.Yusof tested the randomness of different data sets generated by the 3D-AES algorithm through two rounds of AES evaluation [25]. At the significance level of 0.001, the results showed that the key avalanche test failed, so it was determined that the 3D-AES crypto system did not have randomness.
In the progress of engineering applicability, it is mainly to apply randomness test to various cryptographic components or structures, such as random number generators, symmetric encryption systems, hash functions and so on. According to the test results, we can determine whether the test object has randomness property. The application scenarios of random number include: verification code generation, lottery activity, UUID generation, session id generation, token generation and CSRF Token, password recovery token, game (generation of random elements), shuffle, sequence with specific shape of Tetris, game explosive equipment and password application scenario, and generating keys (symmetric password, message authentication), generating key pairs (public key, digital signature), generating IV (CBC, CFB and OFB mode for block cipher), generating nonce (for defending replay attack, CTR mode of block cipher), generating salt: PBE for password based cipher, etc.
There are many security vulnerabilities of random numbers, and many random number vulnerabilities are involved in various programming languages, such as CVE2013-6386. The random numbers generated by the rand function are predictable pseudo-random numbers, which can be used by remote attackers to predict the security string and bypass the established access restrictions. In this case, random numbers generated by the application needs to be examined. Due to the lack of basic cryptography knowledge, developers do not know to use random numbers or use non-compliant random number generators, or imperfect interface documents in some application scenarios and frameworks, or developers do not read them carefully, which leads to the existence of random number security vulnerabilities. For example, a pseudo-random number is needed as a token for password recovery, but many businesses generate the token directly according to the user name, as well as in OAuth 2.0 protocol where a third party needs to pass a state parameter as a CSRF token to prevent CSRF attacks. Many developers do not use this parameter, or pass in a fixed or non-random value instead of a real random value. Because the authenticator can't verify the validity of this value at the business level, it leads to the CSRF attack of OAuth. In this case, we need to use the randomness test methods to check the random number generated by this kind of application, so as to prevent the vulnerability of pseudo-random numbers, which leads to system security problems.

Summary
Randomness test on cryptographic algorithms is to guarantee the correct operation of secure mechanisms in information security. The misuse of random numbers results in insecure RNGs, weak cryptographic algorithms and vulnerabilities in applications. In this paper, the standards and test methods of randomness test algorithms are analyzed and summarized. The analysis shows that at present, the randomness detection mainly focuses on the randomness evaluation of the algorithm output sequence, and the algorithm itself has an impact on the random number of the output sequence. At present, there is no relevant technology, standards or toolkit to analyze it. Therefore, the next step of this paper is to analyze the impact of the encryption algorithm's own implementation on the random number of the cipher algorithm's output sequence, and design related analysis and evaluation standards and tools for experiments.