Portfolio selection of new Product Projects: a Product reliability PersPective wybór Portfela Projektów nowych Produktów z uwzględnieniem niezawodności Produktu*

New product development (NPD) and its launch is one of the most important business processes in contemporary enterprises. Dynamic development of technology, increasing customer requirements and growth of global competition result in a reduction of product life cycle [8, 11]. Product complexity and limited resources (e.g. financial, personal) hinder quick development of innovative products. Therefore, companies often modernise existing products to develop a new product according to changing customer requirements [8, 31]. Modernisation of an existing product reduces time, cost and risk compared with designing a new product from scratch [13, 15, 28]. A faster launch of a new product can ensure company an advantage over competitors. However, a reduction of product design time often leads to the decrease of product reliability, and consequently, increases the warranty cost in the post-sale phase. Moreover, a decline of customers’ loyalty due to unsatisfactory product quality can decrease future sales and profits. On the other hand, the greater expenses for improving product quality usually lead to increasing product price, what is also adversely perceived by customers. Hence, there is a need to develop a system approach to ensure the desired product reliability from the viewpoint of entire business, and from the stage of selecting portfolio of new product development projects. Product reliability is defined as the ability of a product to perform required functions, under given environment and operational conditions and for a stated period of time [22]. Product reliability is widely considered in the literature from an engineering perspective (e.g. determining stress-strain models of materials in the stage of testing a new product) that aims to improve durability of a product and ensure reliability-related standards [7, 12]. Product reliability is less often considered in a system approach that includes all stages of product life cycle and aligns reliability with business goals such as customer satisfaction, sale/profit growth, and a reduction of development, production and warranty costs. Murthy [22] proposes a decision support system for determining parameters of product reliability based on development cost model, warranty cost model, and reliability and usage models. There is considered product reliability in the context of three levels (business, product and component), and three stages (pre-development, development, and post-development). In turn, Kumar [19] presents a knowledge based reliability engineering approach to manage product safety that takes into account manufacturing process of a new product and business environment (customer requirements, quality of materials purchased from suppliers). These studies consider product reliability from the perspective of a system approach, however, they do not Marcin Relich


Introduction
New product development (NPD) and its launch is one of the most important business processes in contemporary enterprises.Dynamic development of technology, increasing customer requirements and growth of global competition result in a reduction of product life cycle [8,11].Product complexity and limited resources (e.g.financial, personal) hinder quick development of innovative products.Therefore, companies often modernise existing products to develop a new product according to changing customer requirements [8,31].Modernisation of an existing product reduces time, cost and risk compared with designing a new product from scratch [13,15,28].A faster launch of a new product can ensure company an advantage over competitors.However, a reduction of product design time often leads to the decrease of product reliability, and consequently, increases the warranty cost in the post-sale phase.Moreover, a decline of customers' loyalty due to unsatisfactory product quality can decrease future sales and profits.On the other hand, the greater expenses for improving product quality usually lead to increasing product price, what is also adversely perceived by customers.Hence, there is a need to develop a system approach to ensure the desired product reliability from the viewpoint of entire business, and from the stage of selecting portfolio of new product development projects.
Product reliability is defined as the ability of a product to perform required functions, under given environment and operational conditions and for a stated period of time [22].Product reliability is widely considered in the literature from an engineering perspective (e.g.determining stress-strain models of materials in the stage of testing a new product) that aims to improve durability of a product and ensure reliability-related standards [7,12].Product reliability is less often considered in a system approach that includes all stages of product life cycle and aligns reliability with business goals such as customer satisfaction, sale/profit growth, and a reduction of development, production and warranty costs.
Murthy [22] proposes a decision support system for determining parameters of product reliability based on development cost model, warranty cost model, and reliability and usage models.There is considered product reliability in the context of three levels (business, product and component), and three stages (pre-development, development, and post-development).In turn, Kumar [19] presents a knowledge based reliability engineering approach to manage product safety that takes into account manufacturing process of a new product and business environment (customer requirements, quality of materials purchased from suppliers).These studies consider product reliability from the perspective of a system approach, however, they do not Słowa kluczowe: estymacja kosztu, warianty alternatywne projektu, problem spełniania ograniczeń, programowanie w logice z ograniczeniami, system wspomagania decyzji.
present the impact of reliability on selecting an optimal portfolio of NPD projects.Taking into account the fact that product reliability impacts customer satisfaction, sales volume and level of costs, it seems significant to include reliability in determining a portfolio of NPD projects and supporting the decision-maker in selecting an optimal portfolio.This is the motivation to elaborate an approach for designing decision support system of selecting portfolio of new product development projects, taking into account the aspect of ensuring the desired reliability of products.Reliability assessment in different stages of product life cycle can base on objective information (e.g.computational simulation, testing prototypes data, field data of the products) and subjective information (e.g.past experience of similar products, judgments of expert) [25].In turn, Chin [10] presents the use of information acquired from customers (needs, requirements) and company (business goals, resources and constraints) in the stage of concepts evaluation, and in the context of product, process, time, and cost.These approaches use information from enterprise and its environment to assess product reliability and product development cost that can be specified in the form of variables and constraints.This study proposes the use of the sets of variables and constraints to formulate the problem of selecting portfolio of NPD projects in terms of a constraint satisfaction problem (CSP), as distinct from the above-mentioned approaches.CSP enables the design of a knowledge base for a decision support system and the use of declarative programming paradigms to the effective determination of alternative admissible solutions.
Standard failure data analysis requires specifications of parametric failure distributions and justifications of certain assumptions that are at times difficult to validate [33].Among other things, these obstacles are reason of estimating product reliability with the use of heuristic algorithms such as neural networks [21,33], fuzzy logic [34,35], and evolutionary algorithms [9,30].In this study, a fuzzy neural system has been used to identify the relationships used farther to estimate product reliability and cost of new product development.
Product reliability can be measured with the use of indicators such as the mean time to first failure, the mean time between failures, the number of failures per unit time, the mean time between maintenance, durability (the mean length of a product's life), or availability (operating time expressed as a percentage of operating and repair time) [2,16].A design engineer perceives product reliability through product characteristics (e.g.reliability of used materials), whereas a customer perceives product reliability through product attributes (e.g.durability).Each product failure decreases the level of customer satisfaction from the used product, nevertheless, the time to first failure significantly impact this level [22].For this reason, product reliability has been measured in this study as the average number of product usage to first failure.
The remaining sections of this paper are organised as follows: Section 2 presents problem formulation for selecting portfolio of NPD projects in terms of CSP.The proposed method of developing a decision support system (DSS) for selecting portfolio of NPD projects is presented in Section 3.An example of estimating the cost of a NPD project and product reliability, selecting portfolio of NPD projects, and determining admissible solutions for the desired product reliability is illustrated in Section 4. Finally, some concluding remarks refer to the advantages and limitations of the proposed approach are contained in Section 5.

Problem formulation for selecting portfolio of NPD projects in terms of CSP
The new product development process consists of a sequence of the following stages: identification of customers' needs, concept generation of new products, evaluation and screening of concepts, development of the selected concepts (including design and build of prototypes), testing prototypes, and commercialization of new products [27,31,32].A particular place in this process takes the stage referring to evaluation and screening of new product concepts, because wrong identification of the potential success of a new product results in significant expenses for development and marketing of unsuccessful products, and a reduction of financial means for development of alternative more profitable products.
Limited resources in an enterprise impose selection and development of only the most promising NPD projects from a set of the generated concepts.In the case of the limited budget of research and development (R&D), especial importance is related to quality of estimating the cost of a NPD project.If NDP projects are similar to the previous projects in the extent of tasks and time, then the cost of a NPD project can be estimated with the use of the average of the cost for the specific product line [5,14].However, NDP projects often have the different extent of tasks related to developing a new product, from slight modifications to large changes in product structure [14,23].In this case, estimation model of the NPD cost can base on the variables referring to product, enterprise and its environment.The variables are chosen to model taking into account their impact on the NDP cost and the possibility of estimating the values of these variables at the stage of conceptual design of a product, before the stages of detailed design, and build and testing of prototypes.For example, among these variables can be the number of: attributes of a new product that are preferred by customers, components of a new product, new components of a new product, employees participating in new product development, machines and appliances needed to build and test prototypes, components of a new product for processing/assembly, materials needed to build a new product.-Ensuring the desired product reliability R is the expensive process that is connected with fulfilling customers' requirements, the complexity of a new product, testing prototypes, and acquiring the required materials and new technology for manufacturing [20].Improvement of product reliability aims to reduce the potential warranty cost C Wi and increasing customer satisfaction from the used product and goodwill, and consequently, product lifetime.However, the limited budget on research and development of new products imposes optimisation of R and C Wi in order to avoid a situation of generating significant expenditures on a slight improvement of product reliability [1].
An enterprise can allocate funds on the R&D budget B that is intended for market research C M and the development of a portfolio of I most promising products C Di : Market research aims to identify the customers' needs, the acceptance level of a new product by target price, and the strength of competitors.New product development is also limited by the number of team members (TMT) who develop the i-th new product.The number of project teams is limited by the total number of the R&D employees (TM) in the t-th time unit: , 1 1 1 Another factor that impacts the decision of selecting portfolio of NPD projects is the unit cost of manufacturing new product c Ui that depends on the cost of labour, materials and technology needed for ensuring the desired product reliability.The price of a new product is sciENcE aNd tEchNology limited by the price of substitutionary products p i .The excessive cost of material and technology can reduce margin of the i-th product, and make impossible to obtain the target return on investment.For this reason, a portfolio should include such new product projects that minimise the cost of ensuring the desired product reliability, the potential warranty cost and the unit cost of production, and consequently, that maximise return on sales.The relation between price, the unit cost of production and margin of the i-th product is as follows: As a model of new product development includes variables and constraints, the problem of selecting portfolio of NPD projects can be formulated in terms of the constraint satisfaction problem (CSP) that is defined as follows [4,29]: where: A solution of CSP can be an admissible solution in which the values of all variables fulfil all constraints, or an optimal solution in which an extremum of the objective function for the selected subset of decision variables is sought.The problem of selecting portfolio of NPD projects belongs to multicriteria optimisation, in which the selection of the i-th product to portfolio depends on minimising: cost of new product development 1.
C Di cost of warranty 2.
C Wi unit cost of production 3.
c Ui The solution of the presented problem is connected with seeking the answer to the following question: Is there a portfolio of NPD projects by the as-1) sumed constraints, and if yes, which NPD projects constitute this portfolio?
The answer to that question is related to estimating the cost of new product development and the unit cost of production, and determining the optimal product reliability in relation to the cost of warranty.
If the assumed constraints make impossible to obtain a portfolio of NPD projects or the found solution is not satisfactory for the decision-maker, then the problem can be reformulated towards seeking the answer to the following question: Which values should have the decision variables 2) (e.g. the number of R&D employees, the cost of materials for manufacturing product) to fulfil the assumed constraints (e.g. the NPD budget, the unit cost of production, the desired product reliability)?
The presented two classes of questions refer to forecasting and diagnosing tasks.The first class of tasks concerns problems in which the values of the selected decision variables determine the values of objective function.In turn, the second class of tasks refers to problems in which the alternative sets of values of decision variables are sought to meet the target values of objective function.Both classes of problems can be formulated in a natural way as CSP and solved with the use of constraint logic programming [6].

Method of designing DSS for selecting portfolio of NPD projects
In the case of the modernisation of existing products, estimation of the NPD cost may base on the data from the specifications of past products.The data is stored in an enterprise system (e.g. in enterprise resource planning system, computer-aided design system), and it may be used to identify exogenous variables that significantly impact an endogenous variable (e.g. the NPD cost, warranty cost).Exogenous variables are selected to model taking into account their impact on an endogenous variable and the possibility of estimating values of these variables at the stage of conceptual design of a product.In the next step, principal component analysis is carried out for the selected set of exogenous variables in order to reduce the number of variables and avoid data redundancy.The next step of the proposed method refers to identify the relationships between exogenous variables and an endogenous variable.These relationships may be identified with the use, for example, linear regression models and machine learning methods [26].The identified relationships in the form of the conditional rules expand and/or update the knowledge base that is used to estimate costs, and determine a portfolio of NPD projects and alternative scenarios for the given range of input variables.The knowledge base also includes facts such as the level of accessible resources in an enterprise.
The rules stored in the knowledge base are used to estimate the cost according to values of exogenous variables for the considered NPD projects.The estimates of NPD cost, production cost, and warranty cost are the basis of selecting a portfolio of NPD projects.The identified optimal portfolio is presented for the decision-maker who can change the range of input variables and/or their values to investigate other alternative portfolios of NPD projects.Figure 1 presents a framework of decision support system for selecting portfolio of NPD projects (PNPDP).
A constraint satisfaction problem may be seen as a well-tailored representation of the knowledge base.Let us assume that the knowledge base describing a system is represented in the form of the sets U, W, Y that define some system properties U∈ U, W∈ W, Y∈ Y. U consists of input variables, Y consists of output variable, and W consists of auxiliary variables.Knowledge specifying the properties of the system is described in the form of a set of facts F(U,W,Y) and relationships

. Framework of decision support system for PNPDP
A framework of the knowledge base may be described with the use of the logic-algebraic method that has been presented in the context of project prototyping in [3].The logic-algebraic method enables the considered problem to implement in constraint logic programming (CLP).CLP is a platform for solving combinatorial problems that are specified by a set of variables, their domains, and constraints that limit possible combinations of solutions.CLP is a well-suited platform to configuration because of its flexibility in modelling and the declarative nature of the constraint model, where the problem description is also a program that solves this problem [24,29].The inference mechanism includes two components: constraint propagation and variable distribution.Constraint propagation uses constraints to prune the search space and accelerate finding possible solutions.Constraint propagation and variable distribution are available in CLP languages such as CHIP, ILOG and Oz Mozart [6].

Illustrative example
An example aims to present the possibility of the use of a fuzzy neural system to identify relationships between variables and specify these relationships in the form of conditional rules.Moreover, an example illustrates the use of constraint logic programming to search alternative portfolios of NPD projects.An example consists of two parts corresponding to problems (questions) presented in Section 2. The first part is related to estimation of the NPD cost (Subsection 4.1) and product reliability in relation to the warranty cost (Subsection 4.2) in order to select a portfolio of NPD projects (Subsection 4.3).The second part presents the use of a CLP environment to search a set of values of input variables, for which all constraints are fulfilled (Subsection 4.4).

Estimating the new product development cost
The estimation of the NPD cost is based on three variables as follows: where: C D -the cost of new product development, V 1 -the number of components in a product, V 2 -the number of new components in a product, V 3 -the number of employees participating in new product development.
The dataset includes 38 completed projects that belong to the same product line as the considered new product projects.The dataset has been divided into training set (30 cases) and testing set (8 cases) to evaluate quality of an estimating model.The estimation of the NPD cost has been carried out with the use of the average, linear regression, and an adaptive neuro-fuzzy inference system (ANFIS).A fuzzy neural system combines the advantages of the artificial neural networks (ability to learning and identifying the complex relations) and fuzzy logic (ability to incorporating expert knowledge and specifying the identified relationships in the form of if-then rules) [17,18,26].The learning method and parameters of ANFIS have been experimentally adjusted by comparison of errors for methods implemented in Matlab ® environment such as grid partition and subtractive clustering.The smallest errors for the considered dataset have been generated with the use of subtractive clustering method with the following parameters: squash factor -1.25, accept ratio -0.5, reject ratio -0.15 and range of influence (RI) from 0.1 to 1.5.Table 1 presents the root mean square error (RMSE) in the training set (TRS) and the testing set (TES), and the number of rules for different values of RI in ANFIS, linear regression and average.
The ANFIS has generated in the training set less RMSE than the average and linear regression model.However, the RMSE in the testing set for the ANFIS with parameter RI from 0.2 to 0.5 is greater than for the linear regression model.The least RMSE and the relatively small number of rules have been generated by the ANFIS with parameter RI from 0.6 to 1.5.Figure 2 presents the use of the ANFIS (with RI = 1) to estimate the NDP cost C D , for the following values of input variables: Estimation of the NPD cost (79.6thousand Euro) can be further extended towards sensitivity analysis to investigate cost changes for the given values of input variables.Figure 3 presents estimation of the NPD cost for the number of components in a product from 50 to 60, the number of new components in a product from 8 to 14, and 3 employees (the first figure) and 4 employees (the second figure).
Figure 3 presents the growth and direction of changes of the NPD cost in relation to changes of V 1 , V 2 , and V 3 .A unit increment of the number of component in a new product results in the average growth of the NPD cost of 0.7 thousand Euro.In turn, a unit increment of the number of new component in a new product results in the average growth of the NPD cost of 4.3 thousand Euro.Moreover, an additional employee increases the NDP cost of 2.4 thousand Euro.The sensitivity analysis is carried out for each potential NPD project, indicating the growth and direction of changes of the NPD cost depending on changes of input variables.

Estimating product reliability and warranty cost
The warranty cost is another criterion of selecting portfolio of NPD projects.The warranty cost includes settle complaints and repair or replacement of a permanently damaged product.The warranty cost is measured as the average cost of 1,000 sold products from the specific product line in the first 2 years from date of sale.In turn, product reliability is measured as the average number of usage of a product up to the first failure.The relationship between reliability and warranty cost enables determination of the optimal value of investment in improving product reliability.Estimation of product reliability can be based on four variables as follows: where: R -product reliability, V 1 -the number of components in a product, V 2 -the number of new components in a product, V 4 -the number of materials in a product, V 5 -the cost of required materials.Table 2 presents the RMSE in training and testing set and the number of rules for the ANFIS, linear regression and average.
The learning process of ANFIS has been carried out according the same parameters as in the previous subsection.The least RMSE in the testing set has been generated by the ANFIS with RI = 0.9.In two cases of using the ANFIS (for RI = 0.5 and RI = 0.8), the RMSE in the testing set has been greater than in the linear regression model.This example indicates the necessary of comparison of the RMSE generated for the different learning parameters of the ANFIS, what is undoubtedly a drawback of the use of computational intelligence techniques.However, the more precise estimation of the cost by the relatively small number of rules (for RI from 0.9 to 1.5) indicates the attractiveness of using this tool to expand and/or update the knowledge base.
In the next step, the relationship between the average number of product usage to the first failure and the warranty cost is determined.In the case of the significant relationship between these variables (absolute value of the correlation coefficient greater than 0.8), there is estimated the expected warranty cost at the stage of selecting portfolio of NPD projects.Figure 4 presents the average number of product usage to the first failure R (left y-axis, solid line) and the warranty cost C W (right y-axis, dashed line) for 38 previous products (x-axis).The number of product usage to the first failure has been increasingly sorted to illustrate the relationship between these variables.The value of the correlation coefficient equals -0.908, indicating a strong dependence between the increase in the average number of product usage to the first failure and the decrease of the warranty cost.The results show that the increment of product reliability above 390 cycles of product usage to the first failure does not significantly reduce the warranty cost.
The new product specification (e.g. the number of components, materials) is also used to estimate the unit cost of production that is  the third criterion (besides the cost of NPD projects and warranty cost) for selecting portfolio of the NPD projects.

Selecting portfolio of NPD projects
The limited R&D budget and other (e.g.personal) constraints impose the selection of the most promising NPD projects.Assuming that the sales volume of products belonging the same line is similar, the criteria for selecting portfolio of NPD projects include the NPD cost C D , unit cost of production c U and warranty cost C W .The NPD and warranty cost is expressed in other values than the unit cost of production.To use these criteria in the considered problem, their values have been normalised.
Let us assume that a set of potential NDP projects includes 11 cases, for which the values of input variables V 1 -V 5 and the project time T are specified.These values enable estimation of the average number of product usage to the first failure R, the NPD cost, the unit cost of production, and the warranty cost.Table 3 presents the values of variables and criteria that are used to select a portfolio of NPD projects.
New products should be developed within 150 working days, the R&D budget of 120 thousand Euro, and maximal 6 members of project teams.Other limitations refer to the minimal reliability of a new product (350 cycles of product usage to the first failure) and the maximal unit cost of production (400 Euro).Moreover, a project portfolio should include at least two new products for development.For the above values and constraints, 5 admissible solutions have been found.The optimal portfolio consists of project P4 and P6, for which the expected cost reaches 119 thousand Euro and the expected total time of portfolio completion reaches 148 working days.
If there is no solution or the presented solution does not satisfy the decision-maker, then the considered problem can be reformulate towards seeking the answer to the following question: which values should have input variables to fulfil all constraints?In this case, a set of admissible solutions is sought with the use of methods employed in a CLP environment.

Seeking admissible solutions for the desired product reliability
The average number of product usage to the first failure for the considered products ranges from 314 to 368 (R in Table 3).If these values do not correspond to company policy of ensuring the desired product reliability, then the proposed approach identifies a set of val-ues of input variables (if it exists) for which is possible to obtain the desired product reliability.
Let us assume that product reliability should be increased above 370 cycles of product usage to the first failure.The number of new components in a product (V 2 ) and the number of the required materials for manufacturing a product (V 4 ) has been chosen as potential variables for modifying.There is sought the answer to the following question: can the change of V 2 and/or V 4 of maximal 2 pieces result in improving product reliability above 370 cycles of product usage to the first failure, by fulfilling other constraints (financial, temporal, personal).The set of admissible solutions has been identified with the use of Oz Mozart environment that includes CLP paradigms.
The use of CLP enables the problem specification in declarative manner that in conjunction with constraint propagation techniques and variable distribution significantly reduces a set of potential solutions, and consequently, accelerates finding a solution.Table 4 presents the number of admissible solutions for various variants of changes in V 2 and V 4 , as well as the optimal portfolio of NPD projects with the expected number of cycles of product usage to the first failure.
Increasing the average number of product usage to the first failure results from reducing new components in a product and/or increasing the number of the used materials.The proposed approach enables determination of values of input variables (project parameters) that ensure the desired value of decision criterion (the desired number of product usage to the first failure for the considered problem).Moreover, the proposed approach presents directions of potential changes ensuring fulfilment of the assumed constraints, and consequently, it enables the optimal portfolio selection of NPD projects.

Conclusion
Selecting portfolio of NPD projects is one of the most important decisions in an enterprise influencing future profits and business growth.A reduction of product life cycle imposes the need of continuous development of new products and their launch in order to sustain company competitiveness.In this case, the decision to select the most promising NPD projects gains especial significance.This decision is made on the basis of many often contradictory criteria, and taking into account accessible resources in an enterprise.For example, contradictory criteria refer to improving product reliability and reducing the unit cost of manufacturing a product.Hence, it seems important to support the decision-maker in selecting portfolio of NPD projects.
Portfolio selection depends on available resources in an enterprise and bases on cost and time estimates of new product projects and their market success.New product success mainly relies on customer satisfaction that is connected with product price, product features, and especially product reliability.The contribution of the proposed approach includes the incorporation of a product reliability aspect into problem of selecting portfolio of NPD projects.In the system approach, improvement of product reliability reduces the potential warranty cost and increases customer satisfaction, business goodwill, and finally, sales and profits.The proposed approach uses technical specifications of existing products to identify the relationships between product at-tributes and the NPD cost, or expenditures on product reliability and the warranty cost.These relationships are the basis of estimating values of criteria used to select NPD projects.Moreover, the proposed approach presents the possibility of formulating the considered problem in terms of a constraint satisfaction problem and using constraint logic programming to obtain a solution of this problem.Problem specification in the form of variables, their domains and constraints that link and limit these variables enables the use of the logic-algebraic method to describe a framework of the knowledge base and facilitates its extension and/or updating.In turn, the use of constraint logic programming results in a time reduction needed to find a solution.
Limitations of the proposed approach include the requirements referring to acquiring a numerous data set (specifications of past NPD projects among the same product line) to estimate the NPD cost or warranty cost.Moreover, the build and adjustment of parameters of a fuzzy neural system can be seen as a drawback in comparison with a linear regression model.

Fig. 1
Fig. 1.Framework of decision support system for PNPDP

Marcin Relich Portfolio selection of new Product Projects: a Product reliability PersPective wybór Portfela Projektów nowych Produktów z uwzględnieniem niezawodności Produktu*
Portfolio selection of new product development projects is one of the most important decisions in an enterprise that impact future business profits, competitiveness and survival.Ensuring reliability in a new product is costly but it increases customer satisfaction and reduces the potential warranty cost, contributing to product success.This paper aims to develop an approach for designing decision support system of selecting portfolio of new product development projects, taking into account the aspect of ensuring the desired reliability of products.A portfolio selection problem is formulated in terms of a constraint satisfaction problem that is a pertinent framework for designing a knowledge base.A set of admissible solutions referring to the new product alternatives is obtained with the use of constraint logic programming.The proposed approach is dedicated for enterprises that modernise existing products to develop new products.

Table 1 .
RMSE and the number of rules for estimating the NPD cost Fig. 2. Estimation of the NPD cost using ANFIS sciENcE aNd tEchNology

Table 2 .
RMSE and the number of rules for estimating product reliability

Table 3 .
Data for selecting portfolio of NPD projects

Table 4 .
Number of admissible solution for various portfolios of NPD projects