On finite 3-component mixture of exponential distributions: Properties and estimation

Abstract: To study reliability problems, life time and survival analysis, a new mixture model, called the 3-component mixture of the Exponential distributions, is introduced. This study is mainly concerned with the problem of investigating the different statistical properties of the newly developed 3-component mixture of Exponential distributions. Firstly, some basic properties of the 3-component mixture model are discussed. Secondly, we discuss hazard rate function, cumulative hazard rate function, reversed hazard rate function, mean residual life function, and mean waiting time function. Different measures of entropy and inequality indices are also discussed. Closed form expressions of the density functions of order statistics and their statistical properties are derived. Finally, the parameters of the proposed mixture distribution are estimated by making use of the maximum likelihood approach under complete and censored sampling. The results on maximum likelihood estimation are also supported through a simulation study and a real-life data.


ABOUT THE AUTHORS
Muhammad Tahir received his MPhil degree in Statistics from the Quaid-i-Azam University, Islamabad, Pakistan. Also, he is a PhD Scholar. He has published more than 15 research papers in international research journals. Currently, he is a faculty member at Government College University, Faisalabad, Pakistan. His research interests include Bayesian inference, reliability analysis, and mixture distributions.
Muhammad Aslam is a professor of Statistics at Riphah International University, Islamabad, Pakistan. He has received the PhD degree in Statistics form University of Wales. He has published over 110 refereed publications. His research interests include Bayesian inference and mixture distributions.
Zawar Hussain graduated from Quaid-i-Azam University, Islamabad, Pakistan. He has published 70 research papers in reputed journals. His research interests include sampling techniques, randomized response models, and quality control.
Akbar Ali Khan is a PhD Scholar at Quaid-i-Azam University, Islamabad, Pakistan. His research interests include Bayesian inference and sampling techniques.

PUBLIC INTEREST STATEMENT
In the field of industrial engineering, an engineering system may fail due to different causes. These causes of failures may be suitably modeled by the mixture distributions. As a special case, we considered a situation where component failures follow a 3-CMED. For a practitioner, in this study, we gave different reliability properties such as hazard rate function, cumulative hazard rate function, reversed hazard rate function, MRL function, and MWT function. The results in this study are helpful in identifying the causes of failure of an engineering systems and estimating its future life, chances of hazards in future, and behavior of its decay. A common man, not an expert in reliability modeling, can simply use the results of this study for making decision about lifetimes of a system where causes of the failure follow a 3-CMED.

Introduction
During the last few decades, finite mixtures of life distributions have been proved of considerable interest both in terms of their methodological development and practical applications. Mixture models play a dynamic role in many real-life applications. The use of mixture models in the situations, when the data are given only from overall mixture distributions, is known as direct application of the mixture models. The direct applications of mixture models can be seen mostly in industrial engineering, medicine, botany, zoology, paleoanthropology, agriculture, economics, life testing, reliability, and survival analysis. Li (1983) and Li and Sedransk (1988) discussed different features of mixture models and defined two types of mixture models, namely, type-I and type-II mixture models. The mixture of probability density functions (pdfs) from the same (different) family is known as type-I (type-II) mixture model. Several authors have extensively applied mixture modeling in different practical problems. For a detailed review, discussion, and applications of mixture modeling, one can refer to Mendenhall and Hader (1958), Rider (1961), Everitt and Hand (1981), Harris (1983), Titterington, Simth, and Makov (1985), Kanji (1985), Maclachlan and Basford (1988), Jones and Mclachlan (1990), Lindsay (1995), Maclachlan and Krishnan (1997), McLachlan and Peel (2000), AL-Hussaini and Sultan (2001), Sultan, Ismail, and Al-Moisheer (2007), Abu-Zinadah (2010), Afify (2011), Kamaruzzaman, Isa, and Ismail (2012), Kazmi, Aslam, and Ali (2012), Mohammadi, Salehi-Rad, and Wit (2013), Ali (2014), Ateya (2014), Mohamed, Saleh, and Helmy (2014), Zhang and Huang (2015) and many others. In many applications, available data can be considered as data coming from a mixture of two or more distributions. This idea enables us to mix statistical distributions to get a new distribution.
The Exponential distribution, because of its memory-less property, has many real-life applications in testing lifetime of an object whose lifetime does not depend upon its age. The Exponential distribution is often used in different fields of physics to model certain processes. There are many electronic devices whose failure rate does not depend on their ages, therefore, the Exponential distribution is suitable to model the lifetimes. It is also used to model lifetime of tubes, resistors, networks, crystals, knobs, transformers, relays, and capacitors in aircraft radar sets. On the other hand, this distribution has got valuable attention in the field of reliability theory and survival analysis, probability theory and operations research. In all of above-mentioned applications, it is not uncommon to assume that life of particular equipment does not depend upon its age. Thus, to model the lifetimes of certain devices/equipment, Exponential distribution may be a suitable candidate distribution.
Motivated by wide application of mixture modeling, in this article, we plan to develop a mixture of Exponential distribution for efficient modeling of a given time-to-failure data. A random variable is said to follow a finite mixture distribution with q components if the density function of Y can be writ- The cdf F m (y; Ψ m ) of the mth component density is given by: The rest of the article is organized as follows: some basic statistical and reliability properties are discussed in Section 2. Some results about order statistics are presented in Section 3. The maximum likelihood estimation of unknown parameters is developed in Section 4. Section 5 consists of a simulation study. An application of the 3-CMED is studied in Section 6. Finally, concluding remarks are given in Section 7.

Properties
Here, we derive computable representations of some statistical properties associated with the 3-CMED having pdf given in (1).

Properties of a 3-CMED rth moment about origin:
The rth moment about origin of a 3-CMED for random variable Y is defined as: hth order negative moment: The hth order negative moment can readily be determined by replacing r with "−h" in (5) as given below: Factorial moments: Using the result by Khan (2015), the factorial moments can be derived as follows: where ′ u s are non-null real numbers. The E(Y −u ) can be evaluated by replacing r with "β − u" in (5) as: Mean and variance: The mean and variance of a 3-CMED are given by: ( 1 , 2 , 3 , p 1 , p 2 ) affect the 3-CMED. These graphs illustrate the versatility of the 3-CMED.

3-Component Mixture of Exponential Distributions
Using the expressions in (5), (8), (9), and (10), mean, variance, median, mode, and coefficient of skewness are evaluated for the parametric values fixed in Figures 1-3. The numerical results, so obtained, are presented in Table 1. Table 1, it is observed that the 3-CMED is a positively skewed distribution because Mean > Median > Mode and SK > 0. Also, as we increase proportion (component) parameters for fixed values of component (proportion) parameters, variance of the distribution increases (decreases). The variance of the 3-CMED is seen to be a decreasing function of all the component and proportion parameters.

Reliability properties
In reliability theory, classification of lifetime models is defined in terms of their reliability function (survival function) and failure rate function (hazard rate function). Hazard rate function is a ratio of lifetime model to reliability function. If value of reliability function is smaller (which means item or component have less survival time), then hazard rate will be higher (chance of failure will increase) while on contrary, if value of reliability function is larger, hazard rate will be lower (chance of failure will decrease). We now study the reliability properties of the 3-CMED.

Reliability function or survival function:
The reliability function or survival function of the considered 3-component mixture model is written as:  where R m (y; Ψ m ) the reliability function of the mth component, is given by: where for m = 1, 2, 3,

Statistical functions
The different statistical functions of a 3-CMED whose density function is specified by (1) are now derived here.

Moment generating function:
The moment generating function (mgf) of a 3-CMED is defined as: Characteristic function: The characteristic function (cf) can be determined by replacing t with 'it' in (17) as:

Probability generating function (pgf):
The pgf can be obtained by replacing t with "ln(α)" in (17) as:

Entropies
Entropy has wide application in science, engineering, and reliability theory, and has been used in various situations as a measure of uncertainty. In other words, entropy of a random variable Y is a measure of uncertain amount of information in a function. Numerous measures of entropy have been studied and compared in the literature. Here, we derive explicit expressions for the three most important entropies, namely, Shannon's entropy, Re′nyi entropy, and β-entropy. The Shannon's entropy has a similar role as that of measure of kurtosis in comparing the shapes of various densities and measuring heaviness of tails.
Shannon's entropy: The Shannon's entropy of a 3-CMED for a random variable Y is defined by: Note that, the above function can easily be evaluated by numerical integration.
Re′nyi entropy: Introduced by Re′nyi (1961), Re′nyi entropy is one of the main extensions of the Shannon's entropy.
For a random variable Y, Re′nyi entropy is defined as: where α > 0, and α ≠ 1. The Re′nyi entropy of a 3-CMED for a random variable Y is defined by: After simplification, the Re′nyi entropy becomes:

β-entropy:
The β-entropy was originally introduced by Havrda and Charvat (1967) and later it was applied to physical problems by Tsallis (1988). Tsallis exploited its non-extensive features and placed it in a physical setting (hence it is also known as Tsallis entropy). Moreover, β-entropy is a one-parameter generalization of the Shannon's entropy which lead to models or statistical results that are different from those obtained using the Shannon's entropy. It is to be noted here that the β-entropy is a monotonic function of the Re′nyi entropy (Ullah, 1996). For a random variable Y, the β-entropy is defined by: where ɛ > 0, and ɛ ≠ 1. The β-entropy of a 3-CMED for a random variable Y is defined as: After simplification, the β-entropy becomes:

Inequality measures
Income inequality indices not only have applications in economics to study the income or poverty, but also in other fields like reliability, demography, insurance, and medicine. The most popular income inequality indices are Gini index (G), Lorenz curve L(p), and Bonferroni curve BC(p).
Gini index: The Gini index (G), proposed by Gini (1914), of a 3-CMED for a random variable Y is defined by:  (1905), of a 3-CMED for a random variable Y is defined by: Bonferroni curve: Bonferroni (1930) proposed a measure of income inequality, based on partial means, which is desirable when the major source of income inequality is the presence of units whose income is much below those the others. The Bonferroni curve BC(p) for a 3-CMED can be computed through the following relation: where

Order statistics
Order statistics deals with properties and applications of ranked random variables. When it comes to studying natural problems related to flood, longevity, breaking strength, atmospheric pressure, wind .  etc., using order statistics becomes essential in the sense that the problem of interest in these cases reduces to that of extreme observations. Suppose a system runs on six batteries and shuts off when the third battery dies. In this case, one may want to know the distribution of third-order statistic. In addition, suppose the system becomes less efficient when the second battery dies and incurs cast in terms of money every time it runs that way. In this situation, the problem of entrust may be to know the distribution of the range between the second and third occurrences. This shows the importance of order statistics in different fields of study. Here, we provide the density of the kth order statistic y k:n , say g(y k:n ; ), in a random sample of size n from a 3-CMED. The expressions for rth raw moment, mean, and variance of the first and nth order statistics are also provided.
pdf of kth order statistic: The pdf of kth order statistic of a 3-CMED for a random variable Y is given by: Thus, the pdf (27) of kth order statistic becomes: where pdf of first-order statistic: Substituting k = 1 in (28), the pdf of first order statistic or smallest observation Y 1 becomes: × (1 − p 1 − p 2 ) v p 1 1 exp(− 1 y 1 ) + p 2 2 exp(− 2 y 1 ) + (1 − p 1 − p 2 ) 3 exp(− 3 y 1 ) g(y 1:n ; ) = n 1 g(y 1:n ; ) = n pdf of nth order statistic: Substituting k = n in (28), the pdf of nth order statistic or largest observation Y n becomes: rth moments about origin, mean, and variance for first order statistic: The rth moment about origin, mean, and variance for first order statistic or smallest observation are obtained as: (1 − p 1 − p 2 ) l+2−1 g(y n:n ; ) = n 1 rth moments about origin, mean, and variance for nth order statistic: The rth moment about origin, mean, and variance for nth order statistic or largest observation are obtained as:

Parametric estimation
In this section, we discuss maximum likelihood (ML) estimation under censored and complete sampling situations. We, first, give the censored sampling scheme in a life testing problem and then develop the likelihood function for a censored sample from a 3-CMED. As we shall show later, ML estimator under a complete sampling situation may be readily obtained using the same likelihood assuming infinite censoring time.

Sampling scheme for a 3-CMED
Suppose n units from a 3-CMED are used in a life testing experiment with fixed test termination time t. The experiment is performed and it is observed that r out of n units failed until fixed test termination time t is over. The remaining n − r units are still functioning. As defined by Mendenhall and Hader (1958), there are many practical situations where only failed objects can be recognized easily as subset of either subpopulation-I or subpopulation-II or subpopulation-III. For example, based on cause of failure, an engineer may divide a certain failed object as a member of either subpopulation-I or subpopulation-II or subpopulation-III. It may be pointed out that out of r failures, r 1 , r 2 , and r 3 failures belong to subpopulation-I, subpopulation-II, and subpopulation-III, respectively, depending upon the reason of failure. So the numbers of uncensored observations are r = r 1 + r 2 + r 3 and the remaining n − robservations are censored that give no information about to which subpopulation they belong. Now, define y lh , 0 < y lh ≤ t, be the failure time of the hth unit belonging to the lth subpopulation, where l = 1, 2, 3 and h = 1, 2, ⋯ , r l .

Maximum likelihood estimators and their variances based on censored data
This method is very important for estimation in classical statistics and it is widely used for estimating the parameters of mixture models. The ML estimators for unknown parameters = 1 , 2 , 3 , p 1 , p 2 of a 3-CMED are obtained by solving the following non-linear system of It is very difficult to workout closed form explicit expressions for ML estimators. These equations cannot be solved analytically. To obtain the ML estimates of parameters 1 , 2 , 3 , p 1 and p 2 , Mathematica software can be used to solve the above non-linear system of Equations (39)-(43) by some iterative numerical procedure.
(2) A sample censored at a fixed test termination time t is selected. The observations which are greater than a fixed test termination time t are taken as censored ones. This step is skipped when generating a complete (uncensored) sample.
(3) Using the Steps 1-2 for the fixed values of parameters, test termination time and sample size, 1,000 samples are generated.
From Tables 3 and 4, it can be seen that differences of ML estimates of component and proportion parameters from assumed parameters reduce with an increase in sample size at a fixed test termination time and same is the case with large test termination time as compared to small test termination time for a fixed sample size. Also, if 1 > 2 > 3 and p 1 > p 2 , first and third component parameters are under-estimated but second component and both proportion parameters are overestimated at different sample sizes (test termination times) for a fixed test termination time (sample size). Moreover, first and second component and both the proportion parameters are over-estimated, however, third component parameter is under-estimated at varying test termination times (sample sizes) for a fixed sample size (test termination time) in case of 1 < 2 < 3 and p 1 < p 2 . The extent of under-estimation (over-estimation) of component and proportion parameters is lesser for larger sample size as compared to smaller sample size for a fixed test termination time. Also, the extent of over-estimation (under-estimation) of component and proportion parameters is greater for smaller test termination time as compared to larger test termination time.
From Tables 5and 6, it is also observed that differences in ML estimates of component and proportion parameters from assumed parameters reduce with an increase in sample size when test termination time tends to infinity. Also, all three component parameters are over-estimated at different sample sizes for a fixed sample size in both cases 1 < 2 < 3 , p 1 < p 2 and 1 > 2 > 3 , p 1 > p 2 .The extent of over-estimation of component parameters is greater for smaller sample size as compared to larger sample size. The extent of under-estimation or over-estimation of component and proportion parameters is greater for censored data as compared to complete data at varying sample sizes because we get less information from sample. Davis (1952) reported a mixture data on lifetimes (in thousand hours) of many components used in aircraft sets. To illustrate the proposed methodology, we take the data on three components, namely, Transmitter Tube, Combination of Transformers, and Combination of Relays. It is unknown that which component (Tubes, Transformers and Relays) fails until a failure (of a radar set) occurs at or before the test termination time t = 0.4. The total number of tests are conducted 702 times. For test termination time t = 0.4, the data are summarized as:

Real data application
Since n − r = 63, we have almost nine percent censored sample. Thus, this is a type-I right censored data. Whereas, for test termination time t → ∞, the complete data-set are summarized as: n = 702, r 1 = 310, r 2 = 148, r 3 = 181, r = 639, n − r = 63,