Reliability Assessment using Neural Networks of Computational Intelligence Based on Software Failure Data

The computational intelligence approach using Neural Network (NN) has been known to be very useful in predicting software reliability. Software reliability plays a key role in software quality. In order to improve accuracy and consistency of software reliability prediction, we propose the applicability of Feed Forward Back-Propagation Network (FFBPN) as a model to predict software reliability. The model has been applied on data sets collected across several standard software projects during system testing phase with fault removal. Unlike most connectionist models, our model attempt to compute average error (AE), the root mean square error (RMSE), normalized root mean square error (NRMSE), mean absolute error (MAE) simultaneously. A comparative study among the proposed feed-forward neural network with some traditional parametric software reliability growth model’s performance is carried out. The results indicated in this work suggest that FFBPN model exhibit an accurate and consistent behavior in reliability prediction.


Introduction
According to demand of the modern digital age, it is a big challenge for software developers to quickly design, implement, test, and maintain complex software systems.Also it is a difficult task for software companies to deliver good quality software in appropriate time (Bhuyan et al., 2014).The last two decades have witnessed a paradigm shift in the field of software engineering (Benala, 2012).
A typical definition of software reliability is "the probability of the failure free operation of a computer program for a specified exposure period of time in a specified use environment" (IEEE, 1991;Boland, 2002;Khatatneh and Mustafa, 2009;Musa et al., 1984;Goel et al., 1985).Another definition of software reliability is "the probability of the product working correctly over a given period of time" (Mall, 2005).Over the years, many numbers of parametric models and non-parametric growth models have been proposed; there is no such model that can predict reliability across all types of data sets in any environment and in any phase of software development.In present scenario, connectionist approach is showing future success approaches to software reliability prediction and modeling.
The major benefits of software reliability measurement are planning and controlling the resources during software development process for developing high quality software.It gives confidence about software correctness.Also it minimizes the additional cost of testing and improves software reliability (Goel, 1985).At the time of development of any product or system like commercial, military, or any other application, we need to ensure its reliability and consistency in its performance, because a system's reliability has a major impact on maintenance, repair costs, continuity of service, and customer satisfaction (Bhuyan et al., 2015).At the end, the project manager needs to ensure that the software is reliable enough to be released into the market.Keeping the above motivation in mind, in the next paragraph we define our contribution.
The main contribution of this paper is to investigate feed forward back-propagation network (FFBPN) model architecture to predict software reliability using failure data.Unlike the traditional neural network modeling approach, we first explain some useful mathematical expressions to our proposed network model.Our propose model's prediction is based on failure data collected in the process of software system testing or operation.The input to this network is designed as the order pair of cumulative execution time (CPU time) and the desired output value (i.e.number of failures).The network is trained to minimize the error between desired and predicted output.An algorithm named FFBPRP is described in Section 3.1.2to calculate the predictive measures.The model predicts the reliability using various prediction criteria that are described in Section 4. The model compute the predictive performance using common predictability measure such as: the Average Error (AE), the Root Mean Square Error (RMSE), Normalized Root Mean Square Error (NRMSE), and common data set.The model performance is further improved by measuring the Mean Absolute Error (MAE) along with the above predictability measure.We forecasted short-term prediction and long-term prediction for two numbers of data sets sample.The comparison is performed with the criteria Short-Term Prediction (STP) and Long-Term Prediction (LTP) with different research work proposed before.The whole idea is to support the project manager to monitor testing, estimating the project schedule, and helping the researchers to evaluate the reliability of the model.
The rest of the paper is organized as follows: Section 2 describes some related work proposed so far in the area of reliability prediction.Section 3 presents the concept feed forward networks model.We describe the proposed model framework, the basic terminologies, application, training of the network, and step-by-step procedure for reliability prediction in this section.In this section, an algorithm is used to compute various predictive criteria.An analysis on software failure data from a medium-sized command and control system, and experimental observation are presented in Section 4. Finally, in Section 4.2, some comparative studies of various analytical and connectionist models on software failures is carried out.In Section 5, the conclusion and future work are given.

Related Work
Artificial Neural Network (ANN) is a powerful technique for Software Reliability Prediction (SRP).Werbose (1988) proposed back-propagation learning and regression technique to identify sources of forecast in uncertainty in a recent gas market model.Though, the author discussed about various methods and techniques, not implemented using his proposed technique on software failure data.Shadmehr et al. (1990) estimated model parameters of pharmacokinetics system using feed-forward multilayered network and predicted the noise residing in the measured data sample.The authors not considered the back-propagation learning for training the network.The authors compared their results with that of the optimal Bayesian estimator and found that their performance was better than the maximum likelihood estimator.The feed-forward network with back propagation learning of artificial neural networks models are applied for software reliability and quality prediction (Khoshgoftaar, 1992;Singh and Kumar, 2010b;Thwin and Quah, 2002).The data sets construction procedure is not explicitly defined.Khoshgoftaar et al. (1992), Singh et al. (2010b), andThwin et al. (2002), have developed a connectionist model and taken failure data set as input to predict the software reliability, where as the predictive parameter taken is not sufficient for prediction.These work discussed network architecture, method of data representation and some unrealistic assumptions associated with software reliability models.Pan et al. (2014) made reliability analysis on high Light-Emitting Diode (LED).The authors used simulation method to carried out stress accelerated testing and life prediction.Vijay Kumar et al. (2016) proposed an approach to classify the stone textures based on the patterns occurrence on each sub window.The proposed method is tested on Mayang texture images, Brodatz textures, Paul Bourke color images, VisTex database, Google color stone texture images and also original photo images taken by digital camera.Qiuying et al. (2013) computed the number of test cases in the discrete software reliability demonstration testing.Sun (2012) constructed software reliability Gene Expression Programming (GEP) model based on usage profile.Maizir et al. (2013) used ANN to predict the axial bearing capacity from high strain dynamic testing (i.e.Pile Driving Analyzer (PDA)) data.The authors used regression mean and Mean Square Error (MSE) as predictable measures for predicting the axial bearing capacity.The authors excluded the strong predictable measures such as AE, RMSE, MAE etc. in their work.Mark et al. (2015) used directional coding and back-propagation neural network to recognize palm vein.The Mean Absolute Deviation (MAD) is implemented as feature vector that were used in the form of input to the beck-propagation network.Karunanithi et al. (1992) predicted software reliability using feed forward network and recurrent network.The authors compared their result with 14 different data sets and shown that NN produced better predictive accuracy compared to analytical models in end-point predictions.The authors considered only average error (AE), average bias (AB), normalized average error (NAE) for predictability measure, but not considered root mean square error (RMSE), normalized root mean square error(NRMSE).Sitte (1999), analyzed two models for software reliability prediction: 1) neural network models and 2) parametric recalibration models.The author used Kolmogorov distance as the common prediction measure in the comparison experiment.The comparison was performed for next-step4 prediction and end-point predictions.These approaches differentiate the neural networks and parametric recalibration models in the context of software reliability prediction and conclude that neural networks are much simpler and better predictors.Tian et al. (2005) proposed an useful model for online adaptive software reliability prediction using evolutionary neural network.The prediction is based on software cumulative failure time prediction on multiple-delayed-input single-output architecture.The challenge for this approach is to predetermine the network architecture such as the numbers of the layers and the numbers of neurons in each layer.RajKiran et al. (2007) predicted software reliability by using wavelet neural networks (WNN).In this paper, the authors employed two kinds of wavelets i.e.Morlet wavelet and Gaussian wavelet as transfer functions.They made a comparison on test data with various neural network and found that their proposed network performs better than others.In ( 2008), the authors introduced three linear ensembles and one nonlinear ensemble to compute software reliability.Various statistical and intelligent techniques were used such as multiple linear regressions (MLR), multivariate adaptive regression splines (MARS), and back-propagation trained neural network (BPNN) for software reliability prediction.Lo (2009) designed a model for software reliability prediction using ANN.Fuzzy Wavelet Neural Network (FWNN) was used for phase space reconstruction technology and for SRP (Zhao, 2010).The authors noticed that the failure history is a vital factor for SRP.Ho et al. (2003) proposed connectionist models and used modified Elman recurrent neural network for modelling and predicting software failure.A comparative study was carried out on their proposed model, with the feed-forward neural network, the Jordan recurrent model, and some traditional software reliability growth models.Their experimented results show that the proposed model performed better than the other model.Pai et al. (2006) used support vector machines and simulated annealing algorithms for reliability forecasting.They used lagged data in their analysis by dividing the 101 observations such as: 33 observations for training, 8 observations for validation and 60 observations for test.Since, it is not a standard method of splitting the data set for experimentation.

FFBPN Model Architecture and Reliability Prediction
In this section, we attempt to describe the FFBPN model's architecture.In order to frame the network, we first discuss the basic concepts of our proposed model FFBPN and its application to software reliability prediction.
Here feed-forward neural network is a static network as it is a network with no output feed-back (Chiang et al., 2004).The single-layer feed-forward neural network connected with several distinct layers to form a multilayered feed-forward network is shown in Figure 1.The learning rule applied in this model is parameter learning (Lin and Lee, 1996).The layer that receives input is called the input layer and typically performs no function other than buffering the input signal (Bhuyan et al., 2014).The input layer is not used for computation, so each node of input layer transmits input values to the hidden layer directly.Any layer between the input and output layers are called a hidden layer, because it is internal to the network and has no direct contact with the external environment.
The network reliability prediction process consisted of two steps: a) the training phase b) prediction phase.In the training phase, connecting weights are adjusted to reduce the errors of the network outputs as compared to the actual outputs.At the end of the training phase, the connecting weights between layers are fixed, so that state of any neuron is solely determined by the input-output pattern and not on the initial and past states of the neuron, that is, there is no dynamics involved (Chiang et al., 2004).Normally static-feed forward network is trained using two methods: a) back-propagation algorithm b) conjugate gradient algorithm.In this paper, the back-propagation training algorithm is applied to get back-propagated error using supervised learning.Initially an arbitrary pattern is applied in feed-forward back-propagation network between the units5 .The error in back-propagation process is propagated towards the hidden layer to minimize the error.
In this paper, we have implemented an algorithm to train the proposed FFBPN model and compute the output (i.e.number of failures).This FFBPN is work in two phases: 1) forward propagation and 2) back propagation.
A simple feed forward network assimilates the current activation in memory and generates the output.For each iteration, the errors are back propagated with a set of connection weights.The iteration process may not continue for large epochs6 , because for each layer's error, the error is back propagated, gets smaller and smaller until it converges to zero (i.e.desired output).In this section, an algorithm F F BP RP is applied to compute all predictability measures.The input data structure and its pattern for reliability prediction are also described in this section.

FFBPN architecture construction and training
The proposed model FFBPN consists of three-layer network.The network comprises of two step mappings as some compound function, which can be rewritten as nested function that is given in Equation 2, where x is the input vector to the input layer.
The hidden layer nodes are fully connected with input and output layer nodes.The feed-forward network does not consider feedback from output nodes.The basic feed forward neural network architecture comprises of two phases: 1) feed forward NN, 2) back propagation with error from output layer.Here the input vector is propagated from input layer to hidden layer as shown in Figure 1.The FFBPN model is a simple supervised learning model.The error is calculated for each pair of input pattern and then it is back propagated for training.The error is calculated and weights are folded back to compute the new updated weights for every iteration.
3.1.1FFBPN Architecture Construction: This section gives a brief discussion about FFBPN training using back-propagation learning.The operation in this paper is restricted to "hidden" and "output" layers.The input vector 'x (n-continuous-valued input nodes) is propagated from input layer with weight matrix V. Here, the input nodes x i receive external inputs (i.e.cumulative execution time) after (i−1) th failure interval.Let us consider x i = 0 for non-input nodes and d i is the desired output for the desired state of i th unit.Then by taking net input, the output from hidden layer is computed using Equation 1. z l = a(g l ), where, Here, n is the number of input nodes, θ j is a threshold value, x j is the input node(i.e.cumulative execution time), v lj is the weight link from j th node of input layer to l th node of hidden layer, and a is an activation function between the layers that need to be continuous, differentiable, and non-decreasing.
The output y i represents the numbers of cumulative failures and is calculated using Equation 2. The updated weight w il is computed for each network copy that is summed up with inputs before individual weights are refined.Output y i can be computed as (2) Here, m is the number of 'hidden' nodes.The weight matrix w il (connected from l th node to i th node) and θ l is the bias.

Tailoring Neural Network for FFBPN Model Prediction Performance:
The FFBPN is trained by using computed output data and desired output data as it belongs to supervised learning.Assume that there are i numbers of data points are available to train the networks.The first set of analysis is carried out starting with the observations in time to predict the (i+1) th data (Tian and Noore, 2005).This is the order that is used to generate the software failure time data for training purposes.The (i.e.Mean Square Error (MSE)) L is calculated by using the cost function using Equation 3. Here, L k is the summation ranges over all the output units, m is the total number of output nodes and it is an index over training sequence matrix.d k is the actual desired cumulative failures, y k is the predicted cumulative failures.
Using Steepest-descent method, FFBPN network gives a weight update rule that requires a matrix inversion at each step.The network is trained by minimizing the total error which is given in Equation 3. We accumulate the values of the gradient of the weights changes ∆w il and ∆v lj which are represented by Equations 4 and 5 respectively.
where η is a scalar parameter that is used to set the rate of adjustment, referred as learning rate and a is the derivative of a.We have proposed an algorithm for software reliability prediction.We have named our algorithm Feed-Forward Back-Propagation Reliability Prediction (FFBPRP) algorithm.Our model FFBPN is trained and the final output is predicted using FFBPRP algorithm.Below, we explain the steps of our FFBPRP algorithm.

Algorithm FFBPRP
Step-1 Initialization Initialize the weight matrices W, V and the thresholds of the neurons with values in range (0 . . .0.5).set k = 1, L = 0, where k is the number of pattern, L is the error.

Step-2 Set Tolerable Error
Set the maximum tolerable error E max .Consider the error precision as E max =0.005.

Step-4 Training Loop
Apply k th input pattern to the input layer x (k) .

Step-6 Compute Error
Calculate the errors L i for each output node y i , using Equation .(3)Step-7 Backward-Error Propagation and Weight Updation Update the weights W, V using Equation.( 4) and (5).

Step-8 Handle Training Conditions
If k p, go to Step-9 else update k = k + 1, go to Step-4.

Step-9 Error Comparison
If L E max go to Step-10 else set L = 0, k = 1, go to Step-4.

Step-10 Print Final Weights
Print the values of W and V

Step-11 Compute the Output
Compute the next failures using Equation .(2).

Step-12 Predictive Measures Computation
Calculate the various prediction criterions mentioned in Section 4. Now, we briefly explain our FFBPRP algorithm.First, our algorithm initializes the network parameters along with the weights associated with the layers in Step-1.The maximum tolerable MSE value is fixed in Step-2.The input to this network is structured in Step-3.In Step-4, the network is trained and in Step-5, the output of the network is computed.The errors between the predicted and actual values are calculated and compared with the MSE which is predefined in Step-6.The computation process and weight adjustment are continued until the MSE found falls below a minimum threshold value.Based on this stopping criterion, the network is back propagated towards hidden layer.Then the network is trained using back-propagation learning and weights are adjusted accordingly in Step-7.This process is continued and the maximum tolerable error is compared with MSE after each epoch in Step-8 and 9.The final weights are calculated in Step-10.At the end, the final weights are fixed and the output is recorded for the next cumulative failure in  Step-12, the algorithm computes all prediction criteria using formulas that are mentioned in Section 4.
We assume that there are multiple neurons present in the hidden layer.We have taken one hidden layer in our implementation.Although as many as hidden layers can be taken, but the result shows that there is no significant improvement in performance in considering more number of hidden layers.Rather the training performance becomes low.The input layer neurons are exempted from error computation.In the first epoch, the weights are typically initialized, next the set of weights are chosen at random and weights are adjusted in proportion to their contribution to error (Karunanithi et al., 1991).The error is computed in output layer and the difference between the actual output and target output values are calculated using Equation 3.
The cross-validation is carried out by the entire representative data set into two sets: a) a training data set, used to train the network, b) a test data set used to predict the reliability of the system.We split the data set as follows: 80% for training and 20% for testing.The data sets are pre-processed by normalizing with respect to their maximum values.The model work better in the close interval [0,1] for all data sets.The training pair consists of training inputs to network and target output.The training data is in ordered pair of two dimensional arrays (input-output): (I 1 , O 1 ), (I 2 , O 2 ) ... (I i , O i ) ..., (I n , O n ), where I i and O i represent the input values (i.e.cumulative execution time) (CPU time) and desired output value (i.e.number of failure) respectively.We can interpret number of failures as a function of cumulative execution time.The weights are structured in two dimensional data arrays (i.e.input-to-hidden layer and hidden-to-output layer).In this work, the logistic function, binary sigmoidal f (x) = 1/(1 + e −λx ) is used, where x is the cumulative execution time.The binary sigmoidal function is used to reduce the computational burden during training (Sivanandam and Deepa, 2007).The function f (x) is continuous, differentiable, and monotonically increasing.Here λ is the steepness parameter at each step of learning.The range of this logistic function varies from 0.0 to 1.0.

Experimental Results and Comparison with Related Work
We have implemented our proposed approach using MATLAB Version 7.10.We initialize V & W weight matrices with small random values before first epoch is started.After the first epoch, weights are adjusted randomly.The FFBPRP algorithm then calculates a sum squared error between the desired output and actual output with help of error function.The target values are not present for hidden units.Here the error precision (i.e. the maximum tolerable (MSE) is fixed and is taken as E max =0.005.The network model FFBPN is trained with initial weights and continues until the stopping criterion is satisfied and best weights are recorded.Here, we carried out two types of experimentation; (a) next-step prediction or short-term7 prediction of the reliability and (b) the end-point prediction8 is performed at the end of a future testing and debugging session (Karunanithi et al., 1992a).
Normally, the data set used for reliability growth model are having defect severities9 2 and 3 as per TANDEM report (Wood, 1996).In our experiment, we considered the failure data during system testing phase of various projects collected at Bell Telephone Laboratories, Cyber Security and Information Systems Information Analysis Center by John D. Musa (Musa, 1980) and from research work (Mohanty et al., 2013).For our experimentation, we collected two numbers of software failure data sets from various literature and that are shown in Table 1.The actual data set10 shown in Table 1 consists  of  We predicted the software reliability in Short Term Prediction (STP) that is used to measure the current reliability of a system and Long Term Prediction (LTP) used for decision making about long-term test plans.In STP, we considered x i as the time between the (i − 1) th and i th software failure and d i is the number of failures.We can interpret number of failures as a function of cumulative execution time.So, it can be written as f (x i ) = d i .The normalized values of the input to the network such as f (x 1 ), f (x 2 )...f (x i ) are used to predict the di+1 , where di+1 is the computed value (i.e. next number of failures) .In other way, we can forecast di+1 by using {x 1 , d 1 }, {x 2 , d 2 }, ...{x i , d i }, where d i+1 is the corresponding target value is known as short term prediction or 1-step ahead prediction or next-step prediction.Similarly, suppose f (x 1 ), f (x 2 )...f (x i ) are used to predict the di+1 , where d i+1 is the corresponding target value and f (x 2 ), f (x 3 )...f (x i+1 ) are used to predict the di+2 , where d i+2 is corresponding target value, where di+2 is the computed value (i.e. next number of failures).Continuing in this way up to n th pattern is known as long term prediction or n-step step prediction or end-point prediction.

Some prediction criteria are listed below:
-The Average Error (AE), computes how adequately a model predicts all over the system testing phase (Karunanithi et al., 1992b).AE, measures how well a model predicts throughout the testing phase (Karunanithi et al., 1992).AE is used to compare our model with other models to test the predictive accuracy.
Relative Error(%) The Root Mean Square Error (RMSE): RMSE is used to compute how far on average the error (i.e. between actual and target value) is from 0. The lower is RMSE, the higher is prediction accuracy.
is an average of an absolute error that computes how close predictions are to the final result.The MAE and RMSE are used together to analyze the variation in the errors on data set.
Where, F i is the computed output and D i is the target output.The lesser computed value of AE, RMSE, NRMSE, and MAE indicates the higher is the accuracy in prediction.The data sets DBS-1 and DBS-2 are considered for analysis and observation in next-step predictions and end-point predictions using FFBPN model.We summarize these computed values in Table 3 for next-step prediction and Table 4 for long-term prediction.

Analysis and Verification for DBS-1 and DBS-2
Using data set DBS-1, we carried out two analyses; 1) Results in next-step prediction, 2) Results in end-point prediction.Let us consider data set DBS-1 of Table 2 that contains 136 failures for analysis and verification purpose.After the network model is successfully trained, the weights are fixed and computations for measurement unit are performed.The predicted result for next-step prediction of various measurement units are shown in Table 3. Figure 2(a) shows the desired output and computed output against cumulative execution times.The Figure 2(a) shows the performance of the proposed model in terms of the number of cumulative failures w.r.t. the cumulative execution times, for the actual and predicted data points in short-term predictions.Then, the model under goes for end-point predictions using test data on data set sample DBS-1.The predicted result for end-point prediction of various measurement units are shown in Table 4.The prediction performance for end-point predictions are demonstrated in Figures 3(a)-3(c).Figure 3(c) is the plot between numbers of epochs and error rate in terms of RMSE during prediction.The observation on this predicted data and comparison is described in Section 4.2. Figure 3(c) shows the error rate (in terms of RMSE) w.r.t. the number of epochs.From Figure 3(c), it can be observed that, the error rate decreases as the number of epochs increases.This shows that our proposed model performs accurate prediction.5(c), it can be observed that, the error rate decreases as the number of epochs increases.This shows that our proposed model performs accurate prediction.
The short term predictions result shows better accuracy than end-point predictions for both the data sets.The AE, we found in this experiment show better accuracy and consistent result than some well-known methods that are presented in Table 6 Table 7.

Comparison with Related Work and Observations
The prediction accuracy on the whole yields by using a large number of data sets.Since it is not practically feasible to represent graphically for large number of data set within these limited pages, we summarize the comparison using DBS-1 and DBS-2.The comparison between some analytical models and the feed forward neural network models are summarized for DBS-1 and DBS-2 in Table 6 Table 7 respectively.It is observed that FFBPN is the better predictor than that of the described models in Table 6 Table 7.The accuracy and consistency of software are measured by the value of NRMSE and AE on data set which is also used as software measurement criteria.From the above result, it is found that the next-step prediction shows better accuracy than end-point predictions results for both data sets DBS-1 & DBS-2.From Table 3 and 4, we observed that DBS-1 produce better result than predictive results of DBS-2.
The comparison between some analytical models and the proposed feed-forward neural network model for software reliability prediction for DBS-1 is summarized in    In Figure 6, we experimented on well-known parametric software reliability growth models such as: Inv Polynomial model proposed by Littlewood and Verrall (2009), Duane Model (2009), Logarithmic model proposed by Musa andOkumoto (1987), G-O Model (1979) with our propose model FFBPN in STP.We observed that our propose model produce satisfactory results than the above model mentioned in Table 6 in STP.It is also observed that the propose model FFBPN, predictive performance in STP is better than in LTP for both data sets.
Moreover, in Table 7, the NRMSE value is showing satisfactory result than that of Mohanty et al. (2013).Mohanty et al. (2013) also used the same data set to test the efficacy of their method.However, since they used the lagged data in their experimentation.Here, our results compared with one of their lagged result.
It is observed that next-step prediction produces better accuracy compared to endpoint predictions for data sets DBS-1 & DBS-2.
The STP result in Table 6 shows that our proposed model has less AE values than the other models in (Karunanithi et al., 1992b).Again by comparing this NRMSE with the NRMSE of many related work (Mohanty et al., 2013), it is observed that FFBPN is giving satisfactory result.
Model quality is considered to be better if its prediction points are close to the ideal line passing through the zero error (Karunanithi et al., 1992).In these experiments, Figure 2

Approach
Values FFN Generalization (Karunanithi et al., 1992b) 9.48 (with encoding) (Tian and Noore, 2004) 4.51 (Cost et al., 2005) 7 two predicted values is significantly close to each other, which indicates that the lesser AE, higher is the accuracy in prediction.Some observations on software reliability prediction using our proposed feed forward neural network model are listed below: -The network can easily be built with a simple optimizing algorithm (e.g.Steepest descent method) and less memory storage requirement that accelerates the training process as well.-The training time cost is minimum in this model for less number of epochs required for output prediction.-It is producing better accuracy and consistency for experimented data set like DBS-1 & DBS-2.

Threats to Validity
Below we discuss the possible internal and external threats to the validity of our work.
-Our experiment uses MATLAB as tool for reliability prediction and therefore suffers the same threats to validity as MATLAB does (i.e.Finding the exact result and number of epochs may not be exact value for every run).-Weights are taken randomly, so every time expecting the same result on the same number of epochs is an issue.
-Our model may not produce satisfactory result for insufficient training data size.
-Our model cannot manage well with major changes that are not reflected in training phase.
Our FFBPN model is a powerful model because of it's nonlinear and logistic properties and having less constraints on the number of observations.

Conclusions
In this paper, we presented a novel technique for software reliability prediction using feed forward neural network with back-propagation.Our proposed technique accurately predicts the software reliability.Unlike most of the computational intelligence models, our model computes AE, RMSE, NRMSE, MAE simultaneously.In this work, most of the predictive criteria are considered.We presented experimental evidence showing that feed forward network with back propagation yields accurate result comparable to other discussed methods.Results shows with two data sets, suggest that the FFBPN model is better at short-term prediction than end-point predictions.Our experimental results show that this approach is computationally feasible and can significantly reduce the cost of testing the software by estimating software reliability.The proposed network can significantly influence the predictive performance.In our future work, we will try to develop a model to predict software reliability at an early stage of software development using some hybrid techniques such as neuro-fuzzy, neuro-genetic, fuzzy-genetic of computational intelligence.
a) Failure Number, b) Time Between Failures (TBF), c) Day of Failure of different medium-sized applications, such as Real-Time Command & Control Software System, Commercial System, Military Application, Operating System, Time Sharing system, and Word Processing System.
Figure 2(b) shows the deviation between the predicted value and actual target value in shortterm predictions.The relative errors of Figure 2(a) are represented in Figure 2(b), which shows the closeness (accuracy) of prediction of the model in short-term predictions.
(a) STP of FFBPN network using data set DBS-1.(b) Deviation between the STP value and the actual value using data set DBS-1.

Fig. 2 .
Fig. 2. Short Term Prediction and relative error of FFBPN network using data set DBS-1.
Figure 5(c) is the plot between number of epochs and (a) LTP of FFBPN network using data set DBS-1.(b) Deviation between the LTP value and the real value using data set DBS-1.(c) Performance result against RMSE using data set DBS-1.
(a) STP of FFBPN network using data set DBS-2.(b) Deviation between the STP value and the real value using data set DBS-2.
(a) LTP of FFBPN network using data set DBS-2.(b) Deviation between the LTP value and the actual value using data set DBS-2.(c) Performance result against RMSE using data set DBS-2.
(b), Figure 3(b), Figure 4(b), and Figure 5(b) show the prediction closeness between the actual vale and prediction value.It is found that the difference between

Table 3 .
Predictive results in STP using two data sets.

Table 4 .
Predictive results in LTP using two data sets.

Table 6 .
The AEs found in this experiment in next-step prediction shows better accuracy and consistent result than some well known methods.

Table 6 .
Comparison of AE values with various approaches.

Table 7 .
Comparison of NRMSE values with various approaches.