Novel Incremental Ranking Framework for Biomedical Data Analytics and Dimensionality Reduction: Big Data Challenges and Opportunities

,


Introduction
Due to the advances in both digital electronic medical record (EMR) systems and next generation sequencing (NGS); most hospitals and biomedical research institutes have massive amounts of biomedical data (size of peta-bytes of data); which pose challenges to be stored; visualized; and analysed. Therefore, there is an essential need for a breakthrough computational intelligence engine to query, analyse, and handle this large amount of biomedical data, at least to (i) improve population health and physicians' reliability in their daily clinical practice and (ii) improve the research discoveries within translational biomedical research and precision medicine. The main focus of this research is to develop or apply machine learning frameworks to handle the most common challenge problems in biomedicine and healthcare, especially after integrating both genotype and phenotype data. The advances in both big data analytics and large-scale machine learning paradigms is the key to success in designing future biomedical computational tools that are required to meet the challenges in healthcare [1,2].
Machine learning and knowledge discovery methods fall into four main categories: (i) supervised learning (forecasting and classification) (ii) unsupervised learning (clustering) (iii) semi-supervised learning and (iv) association rules with inductions. The specific challenge that we address in our research is the development of suitable decision support strategies based on novel innovative machine learning algorithms for the design and interpretation of novel predictive models for specific healthcare events using the available large data sets of individual or population biomedical data to fulfil the essential needs for personalized medicine at the bedside.
Recently, the advances in biomedical technology (EMR with NGS) have greatly increased the amount of available information that is often relevant to clinical decision support and cohorts of experts. These advances have the enormous potential of creating innovative data mining and machine learning algorithms toward improving diagnostic or prognostic accuracy as well as therapy selection capabilities. However; the available big data biomedical information comes with a larger risk of data overload and suboptimal utilization of the information in clinical practice, especially in precision medicine with biomarkers and drug discoveries and in predicting an individual's appropriate responses to a specific drug.
The motivation behind our research is to develop the capability of harnessing big data advanced analytics to manage the correct patients at the correct time with the correct interventions and with high quality treatments while reducing the costs. There are many real-life examples of the use of predictive modelling in both clinical practice and biomedical informatics which include (i) minimizing the mortality and re-admission rates of specific patients and providing real-time monitoring to provide assessments to physicians to save lives within the ICU, especially those preterm children and patients who are admitted with chronic diseases and (ii) determining the Single Nucleotide Polymorphisms (SNPs) loci in a genome that contains single-base variations across a population, which is essential in identifying diagnoses and predicting many complex diseases and common traits.

Related Work and Literature Review
It is known that clustering is the main pillar of machine learning in addressing un-supervised learning and similarities in multidimensional data. Although there are many statistical and computer science techniques for handling such issues, they are still used in some applications, but most of them have drawbacks, specifically, decision trees, association rules, apriori optimization, self-organized map, and fuzzy clustering methods. The scope of this research is to propose new methodology to overcome most of these drawbacks, especially within imbalanced and sparse large biomedical data sets. Next, we conduct comparative studies and summarize existing methods with their limitations in a literature review section, and with implementations, provide results interpretations and future outlooks.
With the current challenges in biomedical data, the computations in machine learning algorithms must be expedited. This step can be performed using the key of successful new parallel and distributed HPC computing using multiple processors (multi-cores) within Sidra Biomedical Informatics Service Hub HPC distributed systems, based on either batch or online sequential modeling; to perform enormous computations through a GPFS/LSF platform or MapReduce and Spark scalable performance model. It has been proven that both MapReduce and Spark platforms have many implemented algorithms such as, genetic algorithms and particle swarm optimization [3][4][5]. We can use both MapReduce and Spark with GPFS/LSF platforms to implement the new framework and the best classifier computation bottleneck for estimating the parameters of the model; which are complicated matrix computations with matrixes from very large training data sets. Therefore; we introduce a reliable solution for matrix multiplications, transposition, and inverses through two separate MapReduce functions.
Assume that we are given biomedical data D={(x i , y i ); i=1, . . ., n} where y i ε {(0,1,…, c-1) for p binary predictors belongs to {0;1}; which are sparse variables with some limitations to connect with the desired categorical target Y and c ≥ 2. The focus is to develop a novel data mining algorithm to empower the desired group categorical target and overcome the difficulties in the building of significant relations/ partitions to enhance future predictions in both healthcare and biomedical industry classification problems. Therefore, select the set of variables that is appropriate for participating in empowering the target class categories (for example, we select k input variables, (X 1 ; X 2 ;...; X k ); and then, we determine the appropriate associated interactions that empower the target class categories based on the most appropriate strategy of selecting this k-significant list of risk drivers with a high volume of data and strong relationship to the desired biomedical healthcare events.
The benefit of the desired recursive screening clustering data mining algorithm is to select the best combination of input features within the biomedical data and compute the classification power of a specific combination, and then, to create intermediate risk drivers within their own specific cluster using the proper combinations based on the AND; OR associative relationships. The main focus is to overcome the most common challenge problems in addressing big biomedical data, namely, sparse characteristics in a high-dimensional domain and addressing the curse of dimensionality. Therefore, we must find the appropriate significant interactions (combinations) of the given data from the available sparse biomedical variables that will be utilized to empower the outcome class categories. The last part of the problem is to design/develop a new data mining classifier based on constrained functional networks with a least squares and conjugate gradient optimization criterion. This new classifier considers both domain experts and data-driven knowledge discovery using a minimum description length to overcome both local optima and complexity due to the high dimensionality and sparse input feature space.

Challenges and Motivations
In working with biomedical data, we face many challenges, namely, sparse and high dimensionality with its corresponding collinearity and reduction problems. Therefore, the attribute contributions in identifying a specific healthcare event will be meaningless. In addition, due to the sparse within the input-space features; the predictive modelling computations will face challenging problems in determining the importance and meaning of the high volume of data within a specific cluster during the visualization and predictions. Furthermore, the expansion of the high-dimensional input biomedical feature space will lead to an ill-condition feature-space matrix, which leads to over fitting problems.
To assess and overcome some of the above-referenced challenges, we are proposing a new recursive screening clustering incremental ranking machine learning algorithm to handle biomedical data's common challenges. This new machine learning framework uses the priori structure information algorithm and association rule strategies to improve the classification or regression performance and then performs an assessment to identify the important and significant features (input attributes). The advantage of our new mathematical algorithms framework will expedite the implementation processes of biomedical data analytics and the predictions of both continuous and categorical healthcare outcomes, then it provides high quality treatment while reducing the cost.
To achieve our research, it is essential to review the state-of-the-art of the popular machine learning techniques that can work with sparse biomedical data. Based on the findings, we specify the most common drawbacks and limitations of these techniques with multi-dimensional sparse overlapping big biomedical data. We recommend future work

Initializations and critical cases
Here, we present the basic steps within the novel desired training algorithm using many unconditional and conditional loops to reduce the rules; for example; the probability of each group is important; due to the following details: o The two above steps must be performed after computing the classifier power using the following formula:

Power (COMB (i))=Ns_TargetClass/Ns.
Based on this novel recursive clustering paradigm we will be able to handle a sparse and multidimensional input space and the curse of dimensionality. In addition, save these binary predictors within each combination. Next, the physicians or case managers will investigate this set of rules and check their adequacy from a medical point of view.
According to both the domain expert and data-driven conclusions, we keep only the rules that fall into common choices. During the implementations, we used the most popular data analytics and modeling quality measures and requirements to discover the risk drivers and to be certain of both the efficiency and reliability of the developed new framework using both simulated and real-life practical issues in both biomedicine and healthcare domains, in other words;

If the target is continuous values:
o For the given set of attributes within a set of data; D; we compute the following quality measures: to be conducted using simulated and real-life (integrated clinical and omics) data to check the performance and comparative studies with statistical techniques, decision trees, association rules, fuzzy clustering means, k-nearest neighbour, SOM and apriori algorithms.

Novel Recursive Clustering Algorithm
The desire training algorithm of the recursive screening clustering technique is to select the proper combination of predictors (input variables) from given biomedical data with multidimensional sparse binary attributes. This set of rules will be revised by the physicians and then used to enhance the classification power of the desired binary target. The entire implementation to obtain a specific combination is briefly summarized in algorithm 1:  o Select the attributes that have the higher correlations.

If the target is in the multinomial/binary category:
o For the given set of attributes within a set of data; we compute the following quality measures: o Select the attributes that have the higher correlations.
Clustering and associations/matching: Similarities with handling the biomedical sparse data and dimensionality reduction.

Advantage of the new clustering framework
The advantages of the new clustering technique can be summarized as follows:

▪
The proposed novel recursive clustering paradigm can handle a sparse and multidimensional input space and the curse of dimensionality then, reduce the number of combinations during the development of the data mining predictive modeling classifier;

▪
The binary predictors within each combination are investigated by the physicians or case managers and are checked for adequacy from a medical point of view. Therefore; based on both the domain expert and datadriven conclusions; ▪ We keep only the ones that fall into common choices;

▪
The new paradigm can be deployed within parallelized distributed HPC systems to handle big biomedical data and fulfill the need of personalized medicine;

▪
The novel data mining recursive clustering technique is simple and has O(n) computations with no prior information/probability similar to the ones with the current techniques; ▪ In our study; we use a new data mining classifier that is based on constrained functional networks with a least squares and conjugate gradient optimization criterion. This new classifier considers both the domain expert and data-driven knowledge discovery using a minimum description length to overcome both local optima and complexity due to high dimensionality and a sparse input feature space; ▪ With the new framework; there is no need to worry about the number of clusters; k; and no sensitivity to outliers that leads to skewed means or to initial conditions that produce different results of the clustering;

▪
The new paradigm is unlike a k-means cluster technique; but we have no non-linear ratio-scale. Therefore; the new paradigm has O(n); whereas; the k-means cluster computation has a complexity of O(K*n*No. Iterations);

▪
The new algorithm uses both associations that are based on volume of each class category and its quotation power classification support; unlike the Apriori algorithm O (2*No. of items). In addition; the Apriori algorithm is a time-consuming algorithm; especially when k is large;

▪
In the new recursive clustering algorithm; there is no need to specify the distance functions among the clusters, unlike Fuzzy C-Means and k-means clustering with the Mahalanobis distance with a mean and covariance matrix, which are very sensitive to outliers and centers;

▪
The new algorithm is optimal due to the use of a genetic algorithm and minimum description length, which leads to fast local optima, unlike the ID3, which is not optimal, due to its use of expected entropy reduction;

▪
The new algorithm does not suffer from any problems when we build the rules, unlike the decision trees, which suffer from the problem of errors propagating throughout the tree, which is a very serious problem as the number of classes' increases.
▪ Therefore, we have developed the appropriate significant interactions (combinations) of a given data set from the available sparse biomedical variables that will be utilized to empower the outcome class categories. The next sections will include the entire process and implementations of the proposed novel recursive and incremental algorithm to empower functional networks classifier using numerous of clinical and genomics data and simulation studies within high performance computing platform: scalable MapReduce HDFS, Spark MLlib, and Google Sibyl [6][7][8].

Novel Machine Learning Classifier Background
In the past few years; functional network models have become popular frameworks for the predictive modeling of different real-life applications, such as medicine and business. The obtained results have proven that functional networks can be considered to be a remarkable data mining knowledge discovery paradigm for predicting both continuous and categorical outcomes [9][10][11]. However, this new intelligence system framework has not been utilized in the biomedicine and healthcare industries, especially for big data. The motivation behind this research is to propose machine learning based on functional networks and use the strength of the MapReduce and Spark distributed platforms to expedite the computations of functional networks that are based on maximum-likelihood estimation (FunNets-MLE) classifiers and then design a suitable decision and overcome the common challenges within big biomedical data and fulfill the requirements of personalized medicine.
In FunNets-MLE, the MLE is a good concave optimal solution that is certainly convergent but is very sensitive to noisy data, additionally, it takes an enormous number of iterative computations to approximate. The Newton-Raphson method is a fast iterative process for approximation, especially within a distributed file systems (DSF) platform; such as Hadoop or Spark, to calculate matrix transposes; computations; and inverses; especially with sparse and multidimensional data. In this research; we are utilizing the minimum description length criterion that take care of both outliers/missing values and handle collinearity among attributes; then the RSVD methodology can work fast with no obstacles. In addition; we use different criteria for imputation to deal with missing data, depending on the percentage of missing values within each attributes; for instance; use mean/median of the rest of non-missing values within each attribute and assign a high value when missing; then this will prevent a record with a missing value to be integrated in the neighbourhood or significant of attributes.

Iterative Newton-Raphson functional networks classifier
The core of Newton's technique is to guess the initial root; Xi.; to estimate the new value of the root; X 1 . .
. . Given the data D={Y, X}, X , . . . , c; and i=1, . . . , n. The goal is to find π ik ; which is the probability that observation i falls in class A k . Assume that where p (•) is a probability function, which must satisfy the probability conditions; and ik τ is the desired model: Here, then; the goal is to estimate k Θ . The function p(•) can be sigmoidal; logit; probit and Cumulative Distribution Function (CDF). During the implementation process in our research; the function that we are interested in applying Newton-Raphson to, for the purpose of expediting the computations, is the first derivative of the Log-Likelihood function; where y ik is the coded matrix; then; the desired Newton-Raphson's is written as in [7]. .
for j, l=0,1,. . . ., (m+1) P and k, k 1 =0, 1, c-1. We note that if the process is initialized at zero and the MLE is known to have concave shape properties, then convergence is guaranteed. Therefore, the implementation processes turn out to be matrix multiplications and inverses: To avoid the enormous computational time, we use one of the available parallel distributed HPC platforms; namely; the Hadoop MapReduce framework, or Spark, or GPFS/LSF systems. Therefore; we will easily perform the mapping and reducing computations and specify the key value outputs/inputs of each of the Map and Reduce functions; as accomplished in [3]. The parallel-FunNets-MLE architecture can be drawn easily; which expresses both the first derivatives of the log-Maximum likelihood function and its corresponding Jacobian or the Hessian matrix; . The processes and training algorithm of FunNets-MLE is based on Newton-Raphson's technique according to equations 1 through 5.

Randomized large-scale singular value decomposition
To minimize the computational time of our algorithm; we utilize the scalable randomized large-scale singular value decomposition (RLSVD) within the available parallel distributed HPC platforms: 16 and 32 cores with the Hadoop MapReduce framework and Spark or GPFS/LSF systems. This novel randomized algorithm is a scalable singular value decomposition for optimizing the performance of the desire machine learning algorithm (Functional networks with Maximum-Likelihood estimations based on Newton-Raphson iterative approximations) [12]. The core idea is to quickly calculate the inverse of a matrix A or determine a rank-k approximation for an m-by-n matrix; A; for large m and n (10 6 or more). Therefore; the goal is to find a rank of (k<<n); where The advantage of the RSVD method is in presenting a fast and easy way to solve problems using Eigen-value decomposition and finding spanning columns or rows of a given matrix and then minimizing the computational cost. The Matlab script for RSVD can be written as follows: This type of methodology can be tested by simulating random data with large-scale dimensionalities; say 5 × 10 5 by 5 × 10 6 ; and for 500 attributes and having a non-zero covariance matrix; then; we have an almost 1000-times reduction in the computational time by using RSVD versus the common original SVDs in Matlab. The new RSVD elapsed computational time is 1.09 seconds; while the regular existing SVDs method in Matlab has elapsed computational time of 998.9 seconds. The greatest use for such a methodology in the future is; for example; biomedical informatics of next generation sequencing and wholegenome and DNA sequencing or calculating the associations and similarities within genome-wide association studies and with respect to common or rare diseases. Furthermore; this approach can be applied to determine the Principal Component Analysis using an empirical covariance matrix from some collection of statistical data and then to compute the singular value decomposition of the given matrix to find the directions of maximal variance.

Implementation and Discussion
The goal is to deploy the developed machine learning methodology based on functional networks maximum likelihood classifier with the Newton-Raphson iterative algorithm for many simulated and real-life biomedical and healthcare industry data. The results with the desired statistical quality measures for both regression and support vector machines are summarized in the tables and figures.

Data acquisition
Congestive heart failure (CHF) data: This clinical data consists of 4,310 observations and 199 input variables. The target of these data involves both groups: Control individuals with 3,288 (76.1%) patients and case individuals with 1,031 (23.9%) patients. The data were combined together to form classification biomedical data of 4,319 individuals with a precision of 23.9%; which is a small percentage (imbalanced data) and represents a challenge for any machine learning or data mining classifier. The entire de-identified data and its ownership was supported by MEDai Inc; Elsevier Company, Orlando FL, USA.

Implementations and discussion
All of the implementations were accomplished using Mahout in the Hadoop DFS MapReduce platform and MLlib Spark system. The new script of the novel recursive clustering framework was implemented in R and Matlab using 32-and 16-core HPC distributed systems. The implementations were performed based on the data-in-hand of both real-life congestive heart failure (CHF) data, very imbalanced breast cancer data; genomic wide association study data and gene mutations to identify children severe asthma exacerbation; and simulated studies.
Congestive heart failure data: The utilization of the new recursive screening clustering mechanism on this clinical data has been achieved; and at the end; we list the significant risk drivers (the high-risk variables) and all of the processes in the tables and graphs. The entire set of processes and implementations are as follows: Step 1: Split for (a) Babies+pregnant women; and (b) all others (Everything below is for all others).
Step 2: Run the proper statistical inference pre-processing calculations as explained above for classification problems with binary categorical outcomes. The desired target (dependent variable) is 0/1; where 1 is the target class; (for example; Readmission; Mortality; ICU admission; Sepsis). Therefore; we consider the treatment cost and length of stay (LOS) to be continuous outcomes (regression problem with the other script code: LOS ≥ 10 and LOS ≥ 25. Day 1 model truncation: LOS=max 25; the goal is ≥ 10 (red); and therefore; the PPV is the N1 criterion; not R 2 .
Step 3: Either for continuous or categorical outcomes; select the top 10 measures with the max ratio (in general F-ratio ≥ 4 is a good value for a high risk) and (volume of clustering data: N>20 or 10); if N=G 0 +G 1 and G 1 (target class observations) ≥ 5.

END
After repeating this procedure several times, we will come up with a list of High-Risk (X's) variables. Suppose that all of the measures (input variables) that left have a ratio of <4.
o Now; select these with a ratio of >3 or >2 and a high volume (N=several thousands). Assume that Y is such a measure. Therefore; we split Y and go to Step 1, in this way, we search for interactions: (Y>0)(Z>0) that have a ratio of ≥ 4.
o The list of high-risk measures will contain thirty to seventy variables and interactions; along with their statistics.
We note that any continuous variable can be transformed into several binary variables by dividing it into intervals or following some binary code structures. For example; the variable age can be written as follows: Next, the physicians or case managers revised the obtained list of significant variables; and then; the results produced the desired High-Risk Cluster. The same procedures are performed for the first two clusters: (i) Babies; and (ii) Pregnant women. An example is the high risk variable for sepsis (High-Risk-Sepsis).
To determine the scoring performance of the new clustering paradigm and build a suitable risk score; we calculate all of the combinations of size's two; three; four; and five attributes out of the available 199 input attributes; which sums up to For simplicity; we propose only 20 rules out of the obtained list of rules in Table 1, using different combinations of at least five predictors to empower the classifier; which considers the physician's point of view with the data-driven measures as well. These sets of rules were built using the new recursive screening clustering approach. In addition; there were other measures that were included in this list, but they were correlated with measures that were already selected with higher ratio values; for example, two, three, and four predictors within different combinations of the risk factor out of the 198 input attributes in highdimensional clinical data of congestive heart failure databases; as is shown in Table 1. Moreover, the area under the curve (ROC) for the average performance of the functional network predictive models that are based on the fast Newton-Raphson computational techniques with over 1000 runs is presented in the curve in Figure 1; which represents the specificity versus sensitivity for the congestive heart failure disease.

Breast cancer surveillance consortium (BSCS) imbalanced data:
To test the performance of the new predictive modeling framework on an imbalanced data with less than 10% (positive class or casedata has only 10% or less of the whole given observations). This can be achieved for "Breast Cancer Surveillance Consortium (BCSC) data: http://breastscreening.cancer.gov/rfdataset/. The data contains approximately; 2.4 million screenings mammograms and associated self-administered questionnaires. The primary goal of these studies was not readability; but rather highest risk detection performances and impact levels of each risk factor: our goal is to provide a risk level without making the decision (breast cancer or not) in place of the physician. The data originally contained 2,392,998 records of index screening mammograms from women included in the Breast Cancer Surveillance Consortium (BSCS) [13][14]. Among the 2,392,998 records of the database; 9314 cases of invasive breast cancer were diagnosed in the first year of follow up. Here; we are facing highly imbalanced data with a positive class accounting for only 0.39% of all records. We observe that; the BCSC database contains most of the known breast cancer personal factors. It is the largest database publicly available that includes breast density information.
As pointed out by [14][15] "information and dialog with more patient involvement in the decision-making process" are key words in dealing with cancer. Therefore a major challenge in the field of medical counselling is to assess clinicians and radiologists with adequate tools to help them to assess their patients' breast cancer risk and to show easily how risk factors impact global risk. Yet, the risk scores that were built upon statistical models were not adopted at bedside regardless their accuracy. This is due to the end-users of these tools are neither oncologists nor clinicians and underlying too difficult to use during a medical consultation. Hence; it is essential to develop a new risk score reliable technique to be used within the current medical decision process.
To build a risk score that helps to detect highest risk profiles among general population; the mining algorithms has to provide a risk value without labelling a woman profile [14] and therein references. Dealing with general population means we are facing highly imbalanced data with a breast cancer incidence rate lower than 1000 new cases for 100,000 women. Dealing with such imbalanced data can be done at both algorithmic and data levels. At data level by choosing a different cost or rebalancing positives or negatives examples. At algorithmic level; it is possible to make a K-nearest-neighbour algorithm more sensitive to the minority class by modifying the neighbourhood boundaries or by using a class confidence weight to handle imbalanced data during the labelling step. The authors in [14] implemented logistic regression; neural networks; k-nearest neighbour with different values of k; and decision trees risk models were built with 4 to 10 risk factors depending on the menopausal status. Comparative studies were carried and they reported that the obtained area under ROC curve results were in between 0.631and 0.642 for premenopausal/postmenopausal women.
In our study; we have done our best to handle both missing values and outliers using different criteria and then overcome the illconditions and collinearity within the attributes in feature-space. In addition; we follow different imputation criterion to deal with missing data; depending on the percentage of missing values within each attributes; for instance; use mean/median of the rest of non-missing values within each attribute and we follow the same strategies that were shown in [14] by assign a high value when missing; then this will prevent a record with a missing value to be integrated in the neighborhood or significant of attributes. Therefore, we have cleaning data with 181,903 observations and 13 sparse attributes (describing various pathological and mammography characteristics of the women); while the target (indicating diagnosis of breast cancer within one year of the screening mammogram). The summary statistics of the entire features of the BCSC data are shown in Table 2; where G0+=Summation of non-zero values supported "control group"; G0%=G0+/actual total number of control group; G1+=Summation of non-zero values supported "case group"; G1%=G1+/actual total number of Case group. These input variables have been determined to influence a woman's risk for developing breast cancer and will henceforth be referred as risk factors. We observe that the given data is a very imbalanced data; where it has 175,629 individuals without breast cancer diseases (control data); which is 96.55% of the available complete observations; while it has only 6,274 individuals with breast cancer diseases (case data); which is representing only 3.45% (a very small percentage compare to the control subset) of the available complete observations. As it is shown in [14]; the goal is to use this breast cancer data (for two cohort groups: premenopausal and postmenopausal women) and then develop novel predictive models to (i) present the dependencies between breast cancer risk factors and then (ii) how to have accurate breast cancer risk score.
The authors in [14] created their own scoring Performances as an experiment set that was designed to test how the k-nearest-neighbour algorithm perform on the BCSC data (choosing k=5). As one of their constraints is to build a readable risk score; they select all combinations with a size s of 1 to 6 attributes among n=12 available attributes; meaning they have in total 2,509 combinations to test. A first way of assessing results of these combinations is to look at the best combinations by size. These results are obtained using an Euclidian space using a 2-norm Euclidian distance as they are not significantly better; when improved; using another p-norm measures. In Gauthier et al.; the authors find that the first list of all possible combinations (from 1 to 6 attributes) [14] performs better than the two specialized regression models obtained on pre-and postmenopausal women by Barlow WE [13] with an AUC of 0.642; based on agegrp; density; brstproc; lastmamm combination. In addition; they use the domain knowledge expert and they recorded the top 15 performance results before and after expert advice.
As before; we run the new algorithm to build a suitable risk score to empower the classifier for better prediction; by calculating all of the combinations of size: two to 9 attributes out of the available 12 input attributes, which sums up to   and would involve expensive computations. The new recursive incremental algorithm reduces the time of the computations of these combinations and produces a reasonable number for the minimum number of combinations (3+8+11+14+9+12+10+5); which is 72 sets of combinations that can be easily interpreted by physicians and use it to empower the desire classifier; then build a proper clinical decision system. These set of rules are very easy to interpret and smaller than the actual rules and it can be instead of (66+220+495+792+924+495+220) =4,004 comprising sets of rules. For the sake of space and simplicity; we are shown we are shown in Table 3; the top 5 rules of the combinations of 7 and 9 attributes with its corresponding AUC values.
We observe that the new framework for sure will outperform logistic regression; neural networks; and decision trees according to the published results within both [13][14]. In addition; the ROC/AUC show improvement around 24%; which means that, we may save lives of 1,506 patients from those 6,274 patients. Therefore; if each one of these individuals is spending $50,000 a year; then; we reduce the total cost of more than $75 million a year. In addition; minimize their future complications; minimize their LOS; ICU; and minimize both lengths of stay and/or emergency room visits with Readmission rates. In addition; the area under the curve (ROC)/AUC is 0.795 for the average performance of the novel method over 1000 runs is presented in the curve in Figure 2, which represents the specificity versus sensitivity for the congestive heart failure disease. Furthermore, we can develop novel large-scale predictive models to present the dependencies between breast cancer risk factors and have accurate breast cancer risk score that is easy to be used by physicians or case managers.

SNP selection and Genomic Wide Association Study Data in
Identifying Children Asthma Exacerbation: By utilizing the real-life genomic wide association study (GWAS) data from [16] to forecast asthma exacerbations in children using random forests classifiers. According to the National Institute of Allergy and Infectious Disease report in (2015): https://www.aafa.org/display.cfm?sub=42&id=8; it is known that asthma is a complex disease known to be influenced by genetic; clinic; and environmental factors: There are 26.7 million individuals or about 9.7% of the U.S. populations have had asthma during their lifetime. In the year 2000, asthma exacerbations resulted in 1,499 deaths; 1.1 million hospital days, and $2.9 billion in direct expenditures in the U.S. There are more than 3,600 deaths due to asthma each year, many of which are avoidable with proper treatment and care. It is the leading chronic disease among children and it is the third-ranking cause of hospitalization in children. Approximately 25.9 million Americans suffer from asthma (8% of adults; 10% of children); women account for almost 65% of asthma deaths; and asthma affects over 230 million people worldwide. The prevalence of asthma has been increasing since the early 1980s across all age; sex and racial groups. Each year; asthma accounts for more than 14 million doctor visits; almost 2 million emergency room visits; 439,000 hospitalizations (average length of stay (LOS) for asthma is 3.6 days); and more than 3,600 deaths. The annual cost of asthma is estimated to be $56 billion. The direct costs accounted for nearly $50.1 billion (hospitalizations the single largest portion of direct cost) and indirect costs of $5.9 billion (lost earnings due to illness or death).
Recently; the authors in [16] predicted the children severe asthma exacerbations using ensemble learning based on "Random Forests" (RF), Bagging and Boosting algorithm using several clinical factors including the forced expiratory volume in one second as a percent of predicted (FEV 1%); oral corticosteroid usage; age; and gender (sex). However; these factors by themselves are limited in their ability to successfully predict severe asthma exacerbations. Therefore; they explore the potential power of a multi-single nucleotide polymorphism (SNP) model and GWAS data as incorporated into RF together with clinical relevant risk factors to effectively predict such complex diseases; this algorithm is applied to the prediction of exacerbations in a population of childhood asthmatics participating in the Childhood Asthma Management Program (CAMP): Stage 1 (population of 127 exacerbation cases and 290 non-exacerbation controls) and Stage 2 (population of 50 exacerbation cases and 114 non-exacerbation controls). However; the accuracy limitations; sparse attributes; and the selection of the significant SNP mutations and find both similarity and associations with complex diseases are still challenges. Therefore, to meet these challenges and be able to provide proper personalized medicine for every individual, it is essential to use the state-of-theart-of large-scale machine learning predictive modeling with big data analytics to predict the children severe asthma exacerbations using integrated genomic-clinic and environmental big biomedical factors. This kind of prediction would therefore have direct prognostic significance and might form the basis for the development of novel therapeutic interventions.
According to the SNP data from genome-wide scans that are available through the National Institutes of Health (NIH) database of Genotypes and Phenotypes (dbGaP) and CAMP parents participations as it is shown in [16] the data acquisition and the primary outcomes of the problem-in-hand can be summarized as follows: • Clinical covariates: Age; gender; pre-bronchodilator FEV 1%; and treatment group (clinical traits) are known to be associated with asthma exacerbations: Age and pre-bronchodilator FEV 1%; are coded as numeric variables; gender/sex is coded as 1 for male; 2 for female; treatment group is coded as 1; 2; 3 for three different treatments.
• Primary outcome: The occurrence of either an emergency room visit or a hospitalization for asthma symptoms at any time during the clinical trial period was used to define a severe asthma exacerbation.
GWAS data: Of the CAMP participants; 417 Caucasian parenttrios-children were genotyped using the Infinium II HumanHap550v3 Genotyping BeadChip, and 164 Caucasian non-trio cohort children were subsequently genotyped using the Human660W-Quad BeadChip. Over 500,000 SNPs were successfully genotyped in the CAMP trios; with a reproducibility of >99.99%. Reproducibility is based on 4 samples that were each genotyped 15 times in the experiment. According to the explanations within [16] the Genotype quality is validated using the Mendel option of PLINK v0.99r http://pngu.mgh.harvard.edu/purcell/ plink; [17] verifying allele calls against Ref-Seq to ensure correct orientation; and testing for extreme departures from Hardy Weinburg equilbrium in the parents.

Selection of SNPs:
Focusing on the trio probands as their initial test population; they used RF importance scores to rank and select SNPs in two steps. At each step; they used SNPs as predictors to predict asthma exacerbations with RF; and obtained the RF importance score of each of the SNPs. At the first step; the computed RF importance scores for all SNPs genome-wide; 4,000 at a time; in chromosomal order. At the second step; they ranked all SNPs based on their RF importance scores; selected the top 4,000 SNPs; and re-run RF with these selected SNPs to re-rank them.

Prediction model building with RF:
The 417 Caucasian trio samples (Stage 1 samples) were genotyped before the 164 cohort samples (Stage 2 samples), and were used to build and train the RF models to predict asthma exacerbations. The R package random forest version 4.5-25 was used to build RF models in this study: http:// cran.r-project.org/web/packages/randomForest/index.html. The RF predicted score is the percentage of trees voting for "yes". During this step and the steps described in "selection of SNPs" above; RF parameter "ntree" (number of trees to grow) were set to be 1,500 -a relatively large number to ensure stable prediction results, and all other parameters, including mtry, were set to use the default values.
The authors in [16] used emergency room visits or hospitalizations as the definition of a severe asthma exacerbation; they first identified a list of top GWAS-SNPs ranked by Random Forests importance score for the CAMP of 127 exacerbation cases and 290 non-exacerbation controls. They predict severe asthma exacerbations using the top 10 to 320 SNPs together with age; gender; pre-bronchodilator FEV 1 percentage predicted; and treatment group. The authors in [16] found that the Area Under the Curve (AUC) score of 0.54 using the clinical traits alone; suggesting the phenotype is affected by genetic as well as environmental factors. On the other hand; they test/validate the predictions in an independent set of the CAMP population shows that severe asthma exacerbations can be predicted with an AUC=0.66 with 160-320 SNPs in comparison to an AUC score of 0.57 with 10 SNPs. Their study shows that a random forests algorithm can effectively extract and use the information contained in a small number of samples. In addition; random forests; and other machine learning tools, can be used with GWAS studies to integrate large numbers of predictors simultaneously.
We follow the same procedures as of [16] in order to assess the performance of the new classifier based on the integrated phenotypic and omics data with the environmental factors (417 CAMP asthmatic family-trio children genotyped and 164 CAMP asthmatic familytrio children genotyped) using stratified sampling methodology of random partition 70% (training) and 30% (testing) with the repetition of computations for 1000 times and one of the common benchmark stopping criterion. In addition; we use the selected clinical traits and SNPs as predictors with the two types of controls (i) permutation control; (ii) other random SNP control; and (iii) the black demarcation separates the top 4,000 SNPs (important and significant towards the target) from the rest (not significant towards the target). The entire population stages were utilized for both training and validation of the new parallelized functional networks classifier to identify the children severe asthma exacerbations and compute the common statistical quality measures; namely; sensitivity and specificity; with the Receiver Operating Characteristic curve (ROC curve) and the Area under the ROC Curve (AUC).
To gain the usefulness of random-forest ensemble learning with the new classifier; we note that the importance of random forest score measures the relative contribution of a predictor to the desire target; which is similar to our set of combinations of rules and apply the propensity score of each of the SNPs and plot it in chromosomal order. By drawing the demarcation separates to identify the top 4000 SNPs with the highest significant score to the target.
To compare the performance of the new classifier with the ensemble learning random forest classifier; we implement both incremental clustering-set of combination rules and parallelized functional networks classifier using the following schemas: (i) The clinical data (age, gender, pre-bronchodilator FEV 1%, and treatment group) alone; and then (ii) similar to [16] we use the integrated repository data (age; gender; pre-bronchodilator FEV 1%, and treatment group; different numbers of SNPs selected based on the propensity score based on the significant SNPs predictors; then at the end we are able to empower the parallelized functional networks classifier to identify the children severe asthma exacerbations with varying degrees of success. We carried over during the implementation processes; the minimum description length criterion that take care of both outliers/missing values and handle collinearity among attributes, then the RSVD methodology can work fast with no obstacles. We concluded that the new classifier with the support of the set of combined rules add trust and reliable to the classifier and increase its AUC values based on the nonlinear relationship with the available phenotypic attributes with demographic predictors is 0.62, which is better than the corresponding values in the existing RF values. On the other hand, by adding of the 10 significant SNPs as it was done in the case of RF, we found that the exacerbations increased the AUC to 0.65, which is better than 0.57. Furthermore; by adding more important SNPs columns to increase the predictability of asthma exacerbations; we got the independently replicated AUC values of of 0.67, 0.71, and 0.71 for 40, 160 and 320 SNPs, which is 7.5% improvement, respectively compared to the RF values within [16]. We conclude that the new parallelized functional networks classifier empowered through the novel criterion of set of combination rules according to propensity score for the important SNPs and we are able to identify the children asthma exacerbations through the use of hundreds of significant SNPs in a novel large-scale machine learning predictive modeling based on functional networks and minimum description length. The comparative studies versus the achieved ensemble learning and random forests model by [16] were reasonable and improve the classification accuracy, which has impact on each individuals treatment cost. In addition, it will improve the healthcare outcomes, for instance, minimize the length of stay, minimize re-admission and emergency room visits; and can be used for disease intervention profiling. Furthermore, as it was expressed in [16], "the new classification model" can increase our understanding of the biologic mechanisms behind why only certain individuals with asthma are at risk for exacerbations; as well as the basis for the epistatic (gene-gene) interactions underlying asthma severity; providing insight into novel preventative and therapeutic strategies".
Simulated mixture data: Classification and time of the computations: Investigating the strengths and capabilities of the new technique requires both real-life data and simulation studies using a 32-core machine. The simulated data are from mixture bivariate predictors and the binomial response of two groups; where each group consists of a mixture of two subclasses (groups); C 1 and C 2 , as follows:  Figure 3 shows the scatter plot; and there is overlap between the two classes; C 1 and C 2 . A mixture of the two normal distributions for each class is supposed to make the decision boundary more complex.
We consider different training sets and different testing sets according to the two-tenth-fold stratified criterion to measure the performance of many classifiers; including functional networks (FunNets), with a maximum likelihood (MLE) based on the iterative Newton-Raphson's method. We summarized the results in the tables and graphs in Figure 4 and Table 3.
Based on the obtained results; we conclude that the comparative studies among these classifiers over the 1000 runs can assess in identifying the desire target in a proper stable decision. We find that the new functional networks classifier with the empowering combination sets of rules is having the lowest time of execution with the highest average correct classification rate, and the lowest misclassification error. On the other hand; the support vector machine is the second highest in both correct classification rate classifier and misclassification error; but it has execution time more than logistic regression, this is because kernel function and high-dimensional input-space. Moreover, the neural networks have the highest execution time; due to its trial and errors and randomly chosen initial weights and its architecture is very complex; but it has reasonable performance with the third place in accuracy. The logistic regression in this simulated study has the worst performance with reasonable execution time because its fast linear approximations that leads to less reliable performance. Therefore, overall the new functional network outperforms the most common techniques, namely; logistic regression, neural networks, and support vector machines; it is faster than all classifiers and has reliable and efficient performance (Table 4).

Conclusions and Future Outlook
The new algorithm is optimal due to the use of the genetic algorithm and minimum description length, which leads to fast local optima, unlike the ID3 method, which is not optimal, due to its use of expected entropy reduction. In addition, the new algorithm does not suffer from any problems when we build rules; unlike the decision trees, which suffer from the problem of errors propagating throughout the tree, which is a very serious problem as the number of classes' increases. Moreover, the propensity score handled the imbalance, which empowers the functional network classifiers.
As is shown from the empirical simulated data; we observed that the randomized singular value decomposition out-performs the regular singular value decomposition for computing matrixes within multi-processors and big biomedical data and rapidly reduces the time of the computations with an almost 1000-times reduction; and the information is gathered at an even faster pace.
▪ We observe that the benefit of having the Newton-Raphson iterative matrix computation was in expediting the quality and performance of the recursive and modeling processes using the HPC and scalable HDFS MapReduce and Spark MLlib. The implementation processes are initialized at zero with a starting approximation and then use the concave shape properties of    MLE; the convergence is guaranteed, which determines the computations for only the matrix multiplications, transpose, and inverse. Furthermore, the new recursive incremental algorithm reduced the times of the computations of these combinations to only 0.06% and produces a reasonable number for the minimum number of combinations, which are 283 sets of combinations instead of 443,667 sets of rules. These small numbers for the number of sets of rules will be easy to retrieve or investigate by either physicians or case managers for specific patients and fast to use in clinical practice at the bedside.
▪ For the imbalanced breast cancer data, we observe that the ROC/ AUC show improvement around 24%, which means that; we may save lives of 1,506 patients from those 6,274 patients. Therefore, if each one of these individuals is spending $50,000 a year, then; we reduce the total cost of more than $75 million a year. In addition, minimize their future complications, minimize their LOS, ICU, and minimize both lengths of stay and/or emergency room visits with read mission rates. Furthermore; we can develop novel large-scale predictive models to present the dependencies between breast cancer risk factors and have accurate breast cancer risk score that is easy to be used by physicians or case managers.
▪ Based on the two utilized biomedical applications that are used for the cases of breast cancer and congestive heart failure, the results show that the new frameworks have a reliable performance with a high impact on empowering the classifier and building a minimum number of rules that is easy to be revised by physicians and fast to use in clinical practice. We recommend future work to be conducted using simulated and real-life data to compare the performance of the new paradigm with the existing techniques.
▪ We investigate the strength and capabilities of the new classifier on imbalanced biomedical big SNP selection and genomic wide association study data, we conclude that the new classifier with the support of the set of combined rules add trust and reliable to the classifier and increase its AUC values based on the nonlinear relationship with the available phenotypic attributes with demographic predictors. In addition, we find that the exacerbations increased the AUC to 0.65. Furthermore; by adding more important SNPs columns to increase the predictability of asthma exacerbations; we got the independently replicated AUC values of 0.67, 0.71 and 0.71 for 40, 160 and 320 SNPs; which is 7.5% improvement better than the existing quality measures in literature. Therefore; we recommend the new classification model for more complex biomedical data to increase our understanding of the biologic mechanisms behind why only certain individuals have specific complex disease, as well as the basis for the epistatic (gene-gene) interactions underlying asthma severity; providing insight into novel preventative and therapeutic strategies".