Development of PYMUS + Code for Quantitative Evaluation of Nuclear Material Accounting ( NMA ) System

The NuclearMaterial Accounting (NMA) system is one of the main safeguards measures to detect the existence of nuclear material diversion. It has become more important for large reprocessing facilities to apply Near Real Time Accountancy (NRTA) system based on NMA and statistical techniques to meet quantitative and timeliness goals. It is also important to quantitatively evaluate the performance of NMA system including NRTA from the standpoints of Safeguards and Security by Design (SSBD) prior to construction of nuclear-material-handling facilities. Such evaluation improves safeguards effectiveness and efficiency. Modeling and Simulation (M&S) work is a good way to evaluate performance for various NMA systems and to determine the optimal one among different options. For these purposes, in the present study, the PYroprocessing Material flow and MUF Uncertainty Simulation+ (PYMUS+) code, which uses evaluation algorithms to calculate many safeguards factors such as MUF uncertainty, detection probability, and others, was developed. According to a previous report, the PYMUS code, the predecessor of PYMUS+, can calculateMUF uncertainties only for a fixed model having 10 tHM/year, whereas the PYMUS+ code can additionally calculate detection probabilities according to diverse nuclear diversion scenarios as well as MUF uncertainties. The most important feature of the PYMUS+ code is its capability to evaluate many process and NMA system model options that a user wants to evaluate. Furthermore, a user can make a static process model having simplicity and a matching NMA model based on the PYMUS+ code regardless of facility throughput and is not even required to have professional programming knowledge. In the present work, some intercomparative studies were conducted to verify the M&S techniques applied in this code. It is expected that this code will be a useful tool for evaluation of NRTA system of pyroprocessing and other reprocessing facilities.


Introduction
Nuclear Material Accounting (NMA) is defined as "activities carried out to establish the quantities of nuclear material present within defined areas and the changes in those quantities within defined periods" [1].NMA system is an important part in the safeguards field to quantitatively evaluate the material balance for nuclear materials such as uranium-235 and plutonium, which is referred to as Material Unaccounted For (MUF).Notably, the Near Real Time Accountancy (NRTA) system based on NMA recently has been introduced at large-scale reprocessing facilities such as the Rokkasho Reprocessing Plant (RRP) [2] for quantitative and timely detection goal for safeguards [3].NRTA also has been considered as a primary safeguards method in pyroprocessing, and in fact efforts to achieve an optimal NRTA system for pyroprocessing are already underway.The main difference between conventional NMA and NRTA is the number of Material Balance Periods (MBPs).Whereas conventional NMA has relatively longer MBP, such as only a single material balance evaluation per year, NRTA performs sequential material balance evaluation for on-time detection of abrupt nuclear diversions.Sequential material balance evaluation by NRTA has many advantages for large-scale facilities, especially those handling plutonium.
From the Safeguards and Security by Design (SSBD) standpoint, it is essential to conduct a preliminary performance evaluation of any NMA system, including NRTA, before construction of a facility.However, such evaluation is difficult without the appropriate simulation code, given the many operation options, measurement options, and diversion scenarios to be considered in case studies entailing extensive modeling work and lengthy simulation time.In the present study, the PYMUS+ code was developed to meet these requirements for evaluation of NMA system performance.It does not require the researcher to do additional programming or coding.The PYMUS+ code's detailed modeling and simulation (M&S) techniques and functions are introduced and explained in the following section.

Modeling & Simulation Techniques
In the PYMUS+, there are four key M&S techniques: simple process modeling, measurement simulation, covariance matrix calculation algorithm, and nuclear diversion detection algorithm.These techniques are essential to M&S work in safeguards and were developed for suitability to the PYMUS+ code in this study.

Simple Process Modeling
Technique.The simple process modeling technique simulates nuclear materials, such as uranium-235 and plutonium, which are targeted for supervision under safeguards.There are three values (input, output, and inventory) that can indicate the status of nuclear material flow within a defined area, Material Balance Area (MBA), during defined period (MBP).These values are essentially required to evaluate a performance of NMA system because its amounts influence simulation result.Input and output mean the amount of nuclear material transferred into and out of MBA during single MBP.Inventory is the amount of nuclear material remaining in MBA and it is classified as beginning and ending inventory according to inventory measurement timing.Simple process model in PYMUS+ is assumed to be static material flow.It means that input and output are identical every MBP and beginning and ending inventory have also same value for whole period.So each input, output, and inventory can be represented as simple values at points to be measured.The PYMUS+ code provides the input format (presented on several Excel sheets) for a user to easily establish the whole simulation model including a simple process model.Figure 1 provides an example of the making of a simple process model using the input format provided by the code.The first column in the figure indicates the process name determined by a user.This column is only for user's convenience and is not utilized i n the PYMUS+.The second column is for connecting measurement approaches and diversion location to process information.To simulate safeguards model, it is necessary to know which process has measurement approaches and in which process nuclear diversion occurs.These pieces of information about measurement approaches and nuclear diversion scenarios are entered in different sheets of the same Excel file and will be described in the next section.The third column indicates total nuclear amount of each process mentioned earlier as input, output, and inventory and the fourth one shows the number of batches during single MBP.The total nuclear amount is divided by the number of batches.It influences the measurement simulation method and the portion of random uncertainty by dividing the measured nuclear material into several batches.This parameter is only operated in the input and output flows (not  Information on MBPs and nuclear diversion scenarios can be organized in another sheet of Excel file named "MBP DS", as shown in Figure 2. In the example, the total number of MBPs is 10, and each MBP is 20 days in duration.This means that every 20 days, 25 kg of Pu is processed in this MBA and that the total throughput for 200 days is 250 kg of Pu.It is possible for the total number of MBPs to be configured up to 300. On the same Excel page and as shown in the figure, a diversion scenario also is configured, based on the essential parameters of diversion location, diversion mass, and diversion time.The parameter of diversion location represents the point of process (Point ID in Figure 1) to which nuclear material is expected to be diverted.In the PYMUS+ code, only output flows are considered for diversion location, because, in the case of a simple process model, it is difficult to simulate process model having nuclear diversion in the inventory process.This diversion scenario, which influences process simulation results of next MBP, cannot be expressed as a simple process mode.The parameter of diversion mass represents the nuclear mass to be diverted during the MBP.For each MBP, individual diversion mass can be enabled.Diversion time is reflected by inputting the parameters of location and mass into suitable Excel-file cells corresponding to the MBP.In the example model, 8 kg Pu diversion occurs in the Output-1 process extending from 60 to 80 days, which is the fourth MBP.
The advantage of this code for building a simple process model is that the user can make many input flows, output flows, and inventory points, without any restrictions.This code can load 10,000 lines of input, which means that it is possible to make a simple process model having 10,000 processes including input, output, and inventory.It also means that the PYMUS+ code can evaluate the performance of an NMA system for a facility that has many process lines.

Measurement Simulation
Technique.The measurement simulation technique, which has been applied in the PYMUS+ code, is based on the error model noted in IAEA reference [4].Equation (1) shows that model: where m = measured nuclear mass, M = true nuclear mass,   = random error for bulk measurement method,   = systematic error for bulk measurement method,   = random error for sampling method,   = systematic error for sampling method,   = random error for analytical method, and   = systematic error for analytical method.
As shown in the equation, the code considers random error and systematic error according to three types of measurement method, which are bulk measurement, sampling, and analytical.These error values are obtained from normal distribution with zero mean and relative standard deviation (), which are called measurement uncertainty.This equation is generally considered for use with the Destructive Analysis (DA) method, which requires application of the bulk measurement, sampling, and analysis methods as well.Additionally, the error model using only one set of random and systematic error, respectively, can also be considered for measurement simulation of Nondestructive Analysis (NDA) in the code.This error model does not require bulk measurement and sampling.The true nuclear mass in this equation is drawn from the information of the simple process model indicated in Figure 1. Figure 3 shows an example of the making of an NMA model for output flow using the input format of an Excel file.KMP in the input format represents the Point ID parameter in Figure 1 for connection of the measurement point to the process point.The ID parameter is used to check whether the same measurement is applied to other measurement points.The use of the same measurement at more than two measurement points generates covariance between those points by incurring the same systematic error.The PYMUS+ code applies the same systematic error if the ID parameters are the same at different measurement points.The No. parameter in the input format represents the number of repeated measurements.This parameter is also an important factor for simulation of an NMA system, because the value obtained by repeated measurements reduces the effect of random error.
The PYMUS+ code calculates the MUF and CUMUF for every MBP, as shown in the following: where   = MUF incurred for i th MBP,   = cumulative MUF incurred from 1 st to i th MBP,  () = total nuclear mass measured in input flow for i th MBP,  () = total nuclear mass measured in output flow for i th MBP,  V(−1) = total nuclear mass measured in beginning inventory at i th MBP, and  V() = total nuclear mass measured in ending inventory at i th MBP.These values represent the material balance within the MBA during any MBP(i) specified in the above equations.

Covariance Matrix Calculation Algorithm.
The covariance matrix calculation algorithm was built into the PYMUS+ code to calculate some important values such as MUF uncertainty, CUMUF uncertainty, and Standardized Independently Transformed MUF (SITMUF).Equation ( 4) represents the prevalent covariance matrix form: where  2  = covariance between i th and j th MBP,   = correlation coefficient, i = i th MBP, j = j th MBP, and n = total number of MBPs.

Science and Technology of Nuclear Installations
As indicated in the equation, the covariance matrix consists of an n-by-n matrix formed according to the total number of MBPs.The element placed in the i th row and j th column shows the covariance between the i th and j th MBPs.Correlation coefficient (  ) is indicated by ±1 or 0 according to the relationship between its points.Covariance in NMA is incurred mainly by systematic error when the same measurement is applied to other measurement points.It is also occurred by random and systematic error in case of |i-j|=1 because measured values in inventories are shared between i th and j th MBPs.
It is possible for the PYMUS+ code to automatically calculate the covariance matrix by the built-in algorithm based on the process model and NMA system model in the input format.Furthermore, in order to provide a more detailed analysis, this code performs more detailed calculation of matrix elements.In the code, covariance between the i th and j th MBPs is classified as detailed values of covariance according to random error, systematic error, and the ID parameters.These detailed results of the covariance matrix provide meaningful information such as the uncertainty contribution according to the type of error and measurement method.
Additionally, MUF uncertainty, its standard deviation, can be indicated as  ij , when i is the same as j.CUMUF uncertainty, its standard deviation, can be calculated as the sum of the elements of the covariance matrix by the k th row and k th column, as shown in where    = CUMUF uncertainty occurred from 1 st to k th MBP.

Nuclear Diversion Detection Algorithm.
The nuclear diversion detection algorithm built into the PYMUS+ code is based on statistical hypothesis test that has been considered as decision technology for NRTA system.There are 7 such tests [5][6][7][8], which are the CUMUF test, the Shewhart test based on MUF, the Sequential Probability Ratio Test (SPRT), the Shewhart test based on SITMUF, the SITMUF cusum joint test, the Page's test, and the GEMUF (geschätzter, or estimated, MUF) test.More detailed descriptions of these statistical tests are available elsewhere.
All of the statistical tests other than the CUMUF test have been applied for NRTA system to perform sequential material balance evaluation.The CUMUF test conducts only a single statistical hypothesis test for the whole MBP based on the conventional NMA system.This test can provide good reference data for comparison of the performances of statistical tests based on the NRTA system with those based on the conventional NMA system.
False Alarm Probability (FAP), which means the probability of a diversion alarm during normal operation conditions, is an important value to be configured appropriately.Generally, FAPs are considered to account for 5% of the error results of statistical tests in safeguards.So, in the PYMUS+ code, FAPs were set at 5% for each statistical test.However, it is difficult to set a 5% FAP for the Shewhart test and the SPRT when utilizing MUFs to make decisions on possible nuclear diversion, because there is covariance between MUFs.The deterministic method to determine the threshold used in the Shewhart test is usually defined under the condition that there is no covariance between MUFs.The deterministic method in the SRPT, likewise, does not reflect the effect of covariance.So, FAPs are lower than 5% when the contribution of systematic error increases.On the other hand, it is possible to set a 5% FAP in the case of the Shewhart test, the SITMUF cusum joint test, the Page's test, and the GEMUF test using SITMUFs instead of MUFs, because there is no covariance between SITMUFs, regardless of the contribution of systematic error.The thresholds of these statistical tests using SITMUFs are determined only by the number of MBPs, because SITMUFs have identical features, which are the facts that the standard deviation is always 1, and that there is no covariance between SITMUFs.Therefore, the thresholds of the SITMUF cusum joint test, the Page's test, and the GEMUF test are set in advance to 5% FAP according to the number of MBPs that are embedded in the PYMUS+ code.The embedded thresholds can reduce simulation time, as they are determined by the Monte Carlo technique, which requires considerable computing time.Each threshold information until total 300 of MBPs had been built in the code.So, in the PYMUS+ code, it is possible to simulate detection probability quickly, without consuming unnecessary computing time to determine thresholds.

Main Functions of PYMUS+ Code
There are many functions in the PYMUS+ code to help the user evaluate NMA system performance.Figure 4 shows the main screen of the PYMUS+ code.As indicated, there are several sections in the main screen for control of certain parameters and analysis of simulation results.
In the "Input Data Settings" section, the user can, for simulation work, control input data related to the entire model.He can open the Excel file that includes input data such as the simple process model, the NMA model, and diversion scenarios.This section also provides an examination function for prevention of mistakes incurred by the user while obtaining the input data.The parameter for the number of simulations also can be configured by the user in this section.This parameter is an important value influencing the computing time as well as the accuracy of the simulation result.The code performs iterative calculations based on the number of simulations (determined by user), because the simulation work to calculate the detection probability is based on the Monte Carlo technique.The code also provides, for confirmation of the reliability of the simulation result, an uncertainty value of detection probability.The user can increase or decrease the number of iterations according to the specific computing performance and simulation-result reliability.
In the "Process Mode" section, the user can select between a simple process model and a detailed process model.A simple process model can be established via the Excel-file input format, as the PYMUS+ code loads the information on the process model from the input file.However, using the input file, it is impossible to make a detailed process model having different nuclear masses and numbers of batches per MBP, because it is too limited to include a complicated process model.So, the PYMUS+ code loads the information on a detailed process model from an external file that includes the nuclear mass at each process point according to the operation time.Currently, performance evaluation of NRTA systems in pyroprocessing facilities is being conducted by the PYMUS+ code based on a detailed process model made by another process simulation code.This process simulation code produces an external file that contains information about amounts of nuclear material of inputs, outputs, and inventories as a function of operation time for the pyroprocessing model.
The other of main function of the PYMUS+ code is the "Sensitivity Study Mode".This function was introduced to simulate numerous options automatically, because it is important to analyze performance according to diverse model options.Performance according to nuclear mass in inventory, diversion mass, measurement uncertainty, and so on should be evaluated to understand its characteristics and be able to build an optimal NMA system thereby.The PYMUS+ code automatically calculates detection probabilities according to variables that the user had already configured in the Excel-file input format, without any additional user action.This function helps the user to evaluate many case studies to find the optimal NMA system.
For detailed analysis of simulation results, the PYMUS+ code provides many graphs on the main screen.Figure 5 shows some of them for the example model described in Section   the inventory points decrease while the contributions by ones in the input and output points increase because the amounts of Pu in input and outputs, which affect the variance of CUMUF, are accumulated according to the MBP.This graph also can provide meaningful information for studies on the reduction of MUF and CUMUF uncertainty by providing which measurement method has the largest portion for the variance of CUMUF.The fourth graph (d) indicates the cumulative detection probabilities according to the MBP.As shown in the graph, all statistical tests except the CUMUF test show an increase of detection probability from fourth MBP because nuclear material diversion happened during forth MBP in the example model.The GEMUF test shows the best performance for this abrupt diversion scenario of the example model.This information can be effectively utilized to review timeliness issues for detection of nuclear diversion.
The PYMUS+ code provides various other information such as the true material balance, the simulation results of each of the statistical tests, and the detection probabilities according to sensitivity studies.The function for saving and loading of simulation results was already built into the code.The main results, including MUF/CUMUF uncertainty and detection probability, are saved in an Excel file, while most of the other results are saved as a raw data in MATLAB file.The raw data is loaded as output data to indicate the simulation results on the main screen without recalculation.

Verification Study of PYMUS+ Code
Intercomparative studies were conducted based on reference data [4,5,9] to verify the M&S techniques applied in the PYMUS+ code.These techniques can be verified by comparing important values such as MUF/CUMUF uncertainty and detection probability.The models presented in the references can be good comparison tools, because they have quite different characteristics with respect to throughput, MBP number, measurement approach, and measurement uncertainty.(1).Table 1 provides a summary of the process and measurement models of reference [9] reinterpreted for the PYMUS+ code.There are three models called parallel, series, and total.The names of models are just referenced from the reference report and have no meaningful significance in this study.The parallel and series model have one input, one inventory, and one output and the total model has one input, five inventories, and one output.The inputs and outputs in all models are 2 kg per MBP.If the total period to evaluate CUMUF uncertainty is less than 30 days, the nuclear mass in the beginning and ending inventories is considered to be 5 kg.On the other hand, in case of 30 days, the nuclear mass in the beginning and ending inventories is considered to be 1 kg because the operation status is assumed as cleanout condition.There is also the difference regarding the throughput processed per day.In the parallel model, 10 kg is handled per day.It means that there are 5 of MBPs during 1 day.On the other hand, in case of the series and total model, 50 kg is handled per day, which is 25 of MBPs during 1 day.The random and systematic uncertainty are 2.0 % and 0.5 % in all inputs and outputs and the random uncertainty is 10 % in all inventories for all three models.The unknown values of systematic uncertainties in the inventories do not influence the simulation result, because those errors are removed by the effect of covariance in all models.

Intercomparative Study
The PYMUS+ calculated the MUF and CUMUF uncertainties based on the above models.Table 2 shows the comparison results between the reference and the PYMUS+.As shown in the table, the uncertainties calculated by the algorithm of the PYMUS+ code have exactly the same values as the reference data.This fact means that in the case of the simple model, the code performed the correct calculation to evaluate the MUF/CUMUF uncertainty.(2).The second intercomparative study was performed for a fuel fabrication facility model utilized as an example of the error propagation method by the IAEA [4].Table 3 provides the summary of this model.The NMA system of the model was assumed to be a conventional NMA system having only a single MBP.Nuclear material was divided into some batches per MBP, unlike the concept of the above-noted previous reference.Input material was divided into 12,000 batches of 20 kg U. Accordingly, the bulk measurements were conducted 12,000 times, once per batch.Sampling and DA analysis were conducted 400 times for the total input material.Information on the number of measurements during the MBP for each measurement approach is indicated in the Table 3.As shown, a main feature is that the ID parameters for the bulk and analytical measurements at the two inventory points are assigned the same value.This means that there is covariance between those two points, because the same systematic errors occurred by applying the same measurement approach.In this case, the method to calculate the covariance matrix can become more complicated than the case for the previous simple models.

Intercomparative Study
The simulation results obtained by the PYMUS+ code are as follows: MUF uncertainty, 212.16 kg U and the contributions by random error and systematic error, 8.17 and 91.83%, respectively.These results, notably, are exactly identical to the reference results.Therefore, it was confirmed that the algorithm for calculation of the covariance matrix in the PYMUS+ code well reflects the correlation between measurement points.

Intercomparative Study (3).
The final verification study was performed to confirm the algorithm's performance with respect to statistical tests and detection probability.The detection probabilities, which are called alarm probabilities in the reference paper [5], were evaluated according to various factors such as MBP number, measurement uncertainties, diversion mass, and total throughput.So, the simulation results obtained by the reference could be good intercomparison data for evaluation of the overall algorithm performance of the PYMUS+ code.There were a total of 12 reference model cases according to MBP number and measurement uncertainty.These models were slightly modified to match the format of the PYMUS+ code, because the reference paper used absolute measurement uncertainties whereas the PYMUS+ code considers only relative ones.All of the modified models had, as a protracted diversion scenario for the total MBP, 100 kg of nuclear material as the input flow, output flow, and inventory points, along with 3.9 kg of diversion mass.The numbers of MBP were set as 4, 12, 24, and 36.The measurement uncertainties were considered in 3 cases, as shown in Table 4. Accordingly, the ratios of random uncertainty to systematic uncertainty were 1:0, 2:1, and 1:1.The table shows the measurement uncertainties of the reference models according to the 3 cases and MBP number.The number of simulations for evaluation of detection probability was set as 100,000 for consistency with the corresponding parameter in the reference paper.Figures 6(a)-6(d) show the comparison results between the reference and the PYMUS+ code.The left graphs show the detection probabilities in the reference paper, and the right graphs show those by the code.The black, red, and green colors show the case 1, case 2, and case 3 models, respectively, and the dashed lines show the detection probabilities of the CUMUF test.The CUMUF test, the Shewhart tests based on MUF and SITMUF, and the SITMUF cusum joint test were considered in this study.The exception was the MUF cusum joint test, which was not applied in the code, because it is not suitable for actual application.As shown in the figure, the simulation results obtained by the code indicate almost identical values in comparison with the reference results.This means that the techniques to calculate the statistical tests and detection probability embedded in the code are operated well based on the conventional methodology, which is universally used in the safeguards field.However, there are small differences between the two results, because the reference paper provides its simulation results only as graphs.The small difference might have been caused by the threshold of the SITMUF cusum joint and method of calculating the detection probability, because these are determined by the Monte Carlo technique, which generates some uncertainties.However, uncertainty incurred by the Monte Carlo technique was not the main concern in this study, because it was small enough for the two results.

Summary
The PYMUS+ code has been developed for evaluation of NMA systems including NRTA in a pyroprocessing facility.The essential M&S techniques such as simple process modeling, measurement simulation, covariance matrix calculation, and the diversion detection algorithm were applied in the code to evaluate MUF/CUMUF uncertainty and detection probability, both of which are important safeguard parameters.The features of the Graphical User Interface (GUI) and the independently-executed standalone application on a 64-bit Windows PC without any software license problem allow users to easily understand the performance of an NMA system in any facilities handling nuclear material; also, they enable quick evaluation of relevant and important safeguards parameters without specialized knowledge of programming languages or of any license issues.
In the present work, verification studies were conducted by evaluating various parameters such as MUF/CUMUF uncertainties and detection probabilities for the reference models.It was confirmed that the M&S techniques applied in the code show good reliability by drawing simulation results identical to those of the reference.
It is expected that the PYMUS+ code can be utilized to evaluate the performance of various NMA systems as well as to develop an optimal one for facilities handling nuclear material, especially pyroprocessing.In the future, the code will be continuously updated by adding some functions such as consideration of short-term systematic error and integrated evaluation with Process Monitoring (PM) and Containment and Surveillance (C/S).

Figure 1 :
Figure 1: Example of input format (Excel file) for configuration of simple process model.

Figure 2 :
Figure 2: Example of input format (Excel file) for configuration of MBPs and diversion scenarios.

Figure 3 :
Figure 3: Example of input format (Excel file) for configuration of measurement model.

2 .
The example model has 1 input, 3 inventories, and 2 outputs in the defined MBA and there are total 10 of MBPs for the entire operation of 200 days.8 kg Pu diversion occurs in the Output-1 process during the fourth MBP.The first graph (a) represents MUF and CUMUF uncertainties according to the MBP.The user can review these uncertainties and utilize them as significant values in the safeguards field.The second graph (b) shows the contributions by random error and systematic error according to the MBP.The values of random error (red line) indicate the percentage of variance by random errors relative to the total variance of CUMUF.The values of systematic error (blue line) mean the percentage of variance by systematic errors relative to the total variance of CUMUF.It is a common pattern in all facility models that the contribution by accumulated systematic error increases according to the MBP.The result presented by this graph is fairly important for grasping of certain characteristics of SITMUF-based statistical tests such as the SITMUF cusum joint test, the Page's test, and the GEMUF test, because there is a strong correlation between their contributions and the detection probability results of the statistical tests.The third graph (c) shows the contributions by the measurement approaches to the variance of CUMUF according to the MBP.Each value indicates the percentage of variance by the measurement method relative to the total variance of CUMUF.The legends show the ID parameters in the input file as shown in Figure 3. ID 12 and 22 mean sampling methods in the Input and the Output-1 process.ID 31 is NDA method in the Output-2 process and ID 51 and 61 are NDA methods in the Inventory-2 and the Inventory-3.As shown in the graph, the contributions by the measurement approaches in