Software Effort Estimation Using Multilayer Perceptron and Long Short Term Memory

Software effort estimation is a hot topic for study in the last decades. The biggest challenge for project managers is to meet their goals within the given time limit. Machine learning software can take project management software to a whole new level. The objective of this paper is to show the applicability of using neural network algorithms in software effort estimation for project management. To prove the concept we are using two machine learning algorithms: Multilayer Perceptron (MLP) and Long Short-Term Memory (LSTM). To train and test these machine learning algorithms we are using the Desharnais dataset. The dataset consists of 77 sample projects. From our results we have seen that Multilayer Perceptron algorithm has better performance than Long Short-Term Memory, by having a better determination coefficient for software effort estimation. Our success in implementing a machine learning that can estimate the software effort brings real benefits in the field of project management assisted by computer, further enhancing the ability of a manager to organize the tasks within the time limit of the project. Although, we need to take into consideration that we had a limited dataset that we could use so a real advancement would be to implement and test these algorithms using a real life company as a subject of testing.


Introduction
In this paper our aim is to prove the effectiveness of machine learning in the planning and execution of a vast amount of projects. Machine learning, is a process of intense data analysis that concludes with the automation of analytical model building. It is a branch derived from artificial intelligence which consists of the fact that systems can learn from analyzed data, recognize patterns and make calculated decisions with minimal or no human interaction needed. The challenge that this paper undertakes is to indicate which of the two machine learning algorithms studied MLP or LSTM is more efficient in the field of the machine learning in project management. The direct result of the aforementioned algorithms is to simplify the tasks of managers, to produce more precise predictions and finally and perhaps most importantly to increase sales and efficiency within the target company. The primary challenge that most project managers face is the achieving of project goals within some different given constraints. Typically, this include but are not limited to, estimating a precise time frame in which the project will be completed which can be one of the trickiest parts of the job. Machine learning algorithms offer a simple solution to that conundrum as they can make project management simpler and more efficient. Different machine learning frameworks offer different benefits based on the particular field that they are being developed for. Some of the advantages of machine learning are as follows:  Predict and assign tasks to team members based on their skills;  Predict and determine when deadlines aren't going to be met and what can be done to prevent it;  Correct task time estimates as a means to update the project information and notify the manager that budget and time frames may require extension. If implemented correctly and trained accordingly a machine learning software can reach a level of automation in which it automatically tracks and detects different problems and delays caused by various unexpected factors. It can also automatically detect delays within a project and act according to what the situation demands. This way the industry would see a total shift in the way projects are run and executed and this should lead to a positive impact on overall team performance and efficiency. Based on the evolutionary trajectory of machine learning algorithms in project management they will fundamentally change the way we as people think about the running and execution of projects, the challenges that arise on the way, in the form of technology and software limitations is something that must be considered very thoroughly. The implications of undertaking the implementation of machine learning algorithms in a large-scale legacy project can prove to be and insurmountable obstacle if we are to consider the following problems: Hardware and software expenses, the amount of money required to be invested in both software development and hardware to train a software product can reach a staggering amount, due to the fact that machine learning algorithms are developed and trained with a particular objective in mind and quality of the required hardware components. While not necessarily a barring problem for big corporations with huge profit numbers and ability to invest in such a technology, it is still a time consuming operation which if not done correctly can lead to serious problems down the road. We aim to achieve this goal by implementing and testing two machine-learning algorithms on the Desharnais dataset: Multi Layered Perceptron (MLP) and Long Short Term Memory (LSTM). In the following sections we will cover the aforementioned machine learning algorithms: The sections that will follow this part of our study aim to provide and analyze real, tangible results. This will be realized though the construction of a machine learning system that will take as input data from a variety of real life projects, and will return the best possible result aimed at improving the planning and execution of similar projects. In order for this to be realized and correctly implemented various objectives must always be considered and treated accordingly, objectives such as:  Analyzing and identifying how workforce effort and performance is estimated and carried out in large-scale distributed projects;  Analyzing the accuracy of the aforementioned effort and performance estimation processes in large-scale distributed projects;  Identifying and investigating the major factors that impact the accuracy of effort and performance estimates in large-scale distributed projects.

Literature review
Machine Learning is an application of artificial intelligence that is revolutionizing the way companies can do business. ML algorithms allow programs to analyze big data to increase the prediction power of the business which in return increase the efficiency of the business and its sales. The biggest challenge for project managers is to meet the goals of their projects within the given time limit. Normally, to predict the time frame of the project can be the hardest part of the job. Machine Learning can take project management software to a whole new level. Many companies are currently testing Machine Learning to predict which actions to take.
There are already efforts of researching the effectiveness of ML algorithms for project management and our search has led us to a few publications [2], [3], [4]. A case study analysed by Usman et al. on a large-scale project concerned with the development and maintenance of a telecommunication software product at Ericsson shows that although traditional estimation methods may deliver reliable and accurate results, underestimation is a frequent trend at both quotation and analysis stages of a software project estimation [5]. Therefore, we can affirm that traditional estimation techniques may be inefficient and impractical as outlined by Pospieszny et al. due to the fact that, often, pressure from clients or management or limited knowledge about the domain or risks involved may lead to overoptimistic estimates, thus impacting severely project outcomes [3]. The same Pospieszny et al. suggests that with smart data preparation and implementation of ML algorithms, we can obtain very accurate results compared to other approaches and are suitable for deployment in practice, but for the best results, it is recommended that homogeneous data must be used for greater prediction accuracy [3]. Another context that needs to be addressed when implementing machine learning algorithms is data, specifically data preprocessing being a fundamental stage of a machine learning method that has a large impact on the accuracy of the predictions as presented by Huang et al. [1]. Furthermore, Panda et al. and Rijwani et al. demonstrate that machine learning is a viable approach in software effort estimation, but it is highly dependent on the quality of the input dataset and the types of algorithms used for the actual predictions [2] [5].
Outlined below, table 1 presents the approaches undertaken by the researchers in the articles chosen as references for this paper to validate machine learning as a viable software effort estimation technique. Our research of these publications has shown that artificial neural network algorithms are advantageous for software effort estimation.
Having that in mind we are proposing the use of artificial neural network algorithms for assistance in project management, our chosen algorithms being Multilayer Perceptrons (MLP) and Long Short-Term Memory (LSTM).

Research methodology 3.1 Data
In this study, the Desharnais dataset is used to train and test the system. The dataset consists of 77 sample projects. Through this data we can not only train our machine learning algorithm, but also help prove the real life applications of such a software and the edge that it can provide in the context in which it is applied. The data is divided by columns, such it goes as follows:  Project numberrepresenting the number of the project;  Team Experiencerepresenting the mean of the development's team experience is the respective field, measured in years;  Manager Experiencerepresenting the amount of experience the manager leading the project possess, measured in years;  Year endrepresenting the last two digits of the year corresponding to the date the project was finished;  Lengthrepresenting the number of months during which the project was in development;  Effortrepresenting the cumulated number of hours that each person has put in the respective project;  Transactionsrepresenting the number of basic logical transactions made within the system;  Entitiesrepresenting the number of entities in the system's data model;  Points Non Adjustrepresenting the size of the project measured in adjusted function points;  Adjustment -representing the size of the project measured in adjustment points;  Points Adjustrepresenting the size of the project measured in unadjusted points;  Languagerepresenting the programming language in which the project was written. Before implementing the algorithms, we must analyze the data to get an overview of the quality of the dataset and if it is homogeneous enough so that the algorithms implemented in this paper will produce sufficiently accurate results. The source of the dataset is Kaggle and the analysis was carried out in Microsoft Office Excel [12].

Descriptive statistics
As outlined in the tables 2 through 11, the standard deviation for all measures is large meaning that there are some outliers in the data set or the latter may have to be clustered into classes for better accuracy of the models.  Due to the fact that the standard deviation is greater than half the mean we determine the fact that it is not representative; Skewness -0.021332638 Due to the fact that the skewness is negative we can conclude that this is an asymmetrical distribution with a long tail to the left; Kurtosis 0.055703646 As the kurtosis is negative and by extension lower than 3 so that makes it Platykurtic; Due to the fact that the standard deviation is greater than half the mean we determine the fact that it is not representative; Skewness 1.598916732 Due to the fact that the skewness is positive we can conclude that this is an asymmetrical distribution with a long tail to the right; Kurtosis 3.081156302 As that the kurtosis is positive and greater than 3 so that makes it Leptokurtic; Due to the fact that the skewness is positive we can conclude that this is an asymmetrical distribution with a long tail to the right; Kurtosis 4.715274686 As the kurtosis is positive and greater than 3 so that makes it Leptokurtic; Due to the fact that the skewness is positive we can conclude that this is an asymmetrical distribution with a long tail to the right; Kurtosis 7.376622924 As the kurtosis is positive and greater than 3 so that makes it Leptokurtic; Due to the fact that the standard deviation is greater than half the mean we determine the fact that it is not representative; Skewness 1.340890535 Due to the fact that the skewness is positive we can conclude that this is an asymmetrical distribution with a long tail to the right; Kurtosis 1.475513847 As the kurtosis is positive and lower than 3 so that makes it Platykurtic; Due to the fact that the skewness is positive we can conclude that this is an asymmetrical distribution with a long tail to the right; Kurtosis 4.757254874 As the kurtosis is positive and greater than 3 so that makes it Leptokurtic; The standard deviation is low, so that means the observations are close concentrated around the mean; Skewness -0.110489192 Due to the fact that the skewness is positive we can conclude that this is an asymmetrical distribution with a long tail to the right; Kurtosis -0.283934607 As the kurtosis is positive and greater than 3 so that makes it Leptokurtic; Due to the fact that the skewness is positive we can conclude that this is an asymmetrical distribution with a long tail to the right; Kurtosis 3.98399961 As the kurtosis is positive and greater than 3 so that makes it Leptokurtic; The standard deviation is low, so that means the observations are close concentrated around the mean; Skewness 0.886121156 Due to the fact that the skewness is positive we can conclude that this is an asymmetrical distribution with a long tail to the right; Kurtosis -0.481012658 As the kurtosis is negative and by extension lower than 3 so that makes it Platykurtic; This fact is proved further by the high values of the skewness/kurtosis for each measure in the data set. We will further analyse this hypothesis after outputting the results from both machine learning algorithms.

Performance measures
For evaluating the performance of the model, as an accuracy criterion, we used the following metrics: where i is a project in the dataset.

Machine learning algorithms used
As mentioned in the introduction phase our study is focused on two machine learning algorithms: Multilayer perceptron (MLP) and Long Short-Term Memory (LSTM). A multilayer perceptron is a type of feedforward artificial neural network that consists of, at least, three layers of nodes: an input layer to receive the signal, one or more hidden layers that represent the computational engine of the MLP and an output layer that makes a decision or prediction about the input. Except for the input nodes, each node is a neuron that uses a nonlinear activation function. MLP utilizes a supervised learning technique called backpropagation for training. The typical MLP structure can be observed in Figure 1 [7].  A LSTM unit is composed of a cell (the memory part of the LSTM unit) and three gates, which regulate the flow of information inside the LSTM unit: an input gate, an output gate and a forget gate. The cell is responsible for keeping track of the dependencies between the elements in the input sequence. The input gate controls the flow of values into the cell, the forget gate controls for how long a value remains in the cell and the output gate controls how the value in the cell is used to compute the output activation of the LSTM unit. Figure  2 outlines the structure of a LSTM unit [10].

Implementation
For both algorithms, the 77 projects of the Desharnais dataset are divided into a training set and a test set. The training set consists of 62 projects which amounts to 80% of the da-taset, the rest of the projects are used for testing the models. We aim to predict the effort required for completion of a software project by using the rest of the quantitative variables in the dataset as inputs (excluding irrelevant data such as project number or year end). We have implemented the aforementioned algorithms using Python and several machine learning libraries: Keras, Tensorflow and Sklearn. The first step of our analysis consists of normalizing the data using a MinMax Scaler provided by Sklearn. The next step is building the network using Keras by training the models on 10000 epochs. Figure 3 shows both networks' structures.

Results and discussion
In this study, prediction of software effort estimation was done by using a Multilayer Per-ceptron and a Long Short Term Memory neural net. The effort required for 17 projects from the Desharnais dataset were predicted.
Predictions and performance measurements are presented in tables 12, 13 and 14.

Conclusions
Machine learning technology has the unique ability to process and interpret large amount of data without the need of constant human interference, the existing application are the real life, tangible proof of this fact. This paper aims to bring an incremental upgrade to the way programmers and scientists understand the diverse benefits and capabilities of machine learning in the management of singular or enterprise projects, to highlight this fact we have brought into foreground the following points of interest:  While there is a strong correlation between the amount and accuracy of the data involved in the training of the algorithm, the end result may still not be foolproof and may require further chiseling;  Throughout our analysis it is concluded that data related pitfalls, such as not having enough data, low quality of the data, or biased data, are the most severe in both the training of the machine learning algorithm, as it can lead to a wrongly executed training procedure and also in the formulation of future predictions and analysis;  It is shown that realizing the value of longterm solutions regarding machine learning in the management of projects can be a very difficult and arduous task, although the risks diminish and the ability increases with experience. Further improvements to the two machine learning algorithms used in this paper would be:  applying cauterization on the dataset and applying the algorithms on each cluster;  testing the two algorithms on other data sets such as ISBSG data set;  fine tuning the algorithms for each data set for improved accuracy and performance. The fact that we successfully implemented and developed a machine learning algorithm that brings real benefits in the field of project management constitutes one of the many stepping stones required for future development. A main path of future progress would be the implementation and testing of the algorithm using a real life company as a subject of testing.