Current practice and challenges towards handling uncertainty for effective outcomes in maintenance

In today’s business environment, the trend towards more product variety and customization is unbroken. Due to this development, the need of agile and reconfigurable production systems emerged to cope with various products and product families. To design and optimize production systems as well as to choose the optimal product matches, product analysis methods are needed. Indeed, most of the known methods aim to analyze a product or one product family on the physical level. Different product families, however, may differ largely in terms of the number and nature of components. This fact impedes an efficient comparison and choice of appropriate product family combinations for the production system. A new methodology is proposed to analyze existing products in view of their functional and physical architecture. The aim is to cluster these products in new assembly oriented product families for the optimization of existing assembly lines and the creation of future reconfigurable assembly systems. Based on Datum Flow Chain, the physical structure of the products is analyzed. Functional subassemblies are identified, and a functional analysis is performed. Moreover, a hybrid functional and physical architecture graph (HyFPAG) is the output which depicts the similarity between product families by providing design support to both, production system planners and product designers. An illustrative example of a nail-clipper is used to explain the proposed methodology. An industrial case study on two product families of steering columns of thyssenkrupp Presta France is then carried out to give a first industrial evaluation of the proposed approach. © 2017 The Authors. Published by Elsevier B.V. Peer-review under responsibility of the scientific committee of the 28th CIRP Design Conference 2018.


Introduction
Due to the fast development in the domain of communication and an ongoing trend of digitization and digitalization, manufacturing enterprises are facing important challenges in today's market environments: a continuing tendency towards reduction of product development times and shortened product lifecycles. In addition, there is an increasing demand of customization, being at the same time in a global competition with competitors all over the world. This trend, which is inducing the development from macro to micro markets, results in diminished lot sizes due to augmenting product varieties (high-volume to low-volume production) [1]. To cope with this augmenting variety as well as to be able to identify possible optimization potentials in the existing production system, it is important to have a precise knowledge of the product range and characteristics manufactured and/or assembled in this system. In this context, the main challenge in modelling and analysis is now not only to cope with single products, a limited product range or existing product families, but also to be able to analyze and to compare products to define new product families. It can be observed that classical existing product families are regrouped in function of clients or features. However, assembly oriented product families are hardly to find.
On the product family level, products differ mainly in two main characteristics: (i) the number of components and (ii) the type of components (e.g. mechanical, electrical, electronical).
Classical methodologies considering mainly single products or solitary, already existing product families analyze the product structure on a physical level (components level) which causes difficulties regarding an efficient definition and comparison of different product families. Addressing this

Introduction
Decision-making in industrial maintenance today is typically based on two broad factors: recorded data and subjective expert opinions. The prior presents hard facts, subject to a degree of uncertainty which can be quantified statistically by standard deviation of the dataset. The latter attributes qualitative uncertainty by what traits qualify them as an expert and the basis of their view to establish its validity. Data recording methods, accuracy of equipment used, or maintainer performance are rarely considered as an attribute to overall uncertainty. A combination of the hard facts and subjective opinion needs to be considered to make informed and effective decisions leading to prosperous outcomes in maintenance. Some cases require more expertise; some require more data. The question here is whether a holistic view of these uncertainties can improve decision-making capabilities and reduce through-life costs as well as unforeseen challenges. This paper presents a survey questionnaire to rank prominent factors that influence uncertainty in maintenance based on literature and input from industry experts. Respondent qualities are attributed in a pedigree assessment. Results are

Introduction
Decision-making in industrial maintenance today is typically based on two broad factors: recorded data and subjective expert opinions. The prior presents hard facts, subject to a degree of uncertainty which can be quantified statistically by standard deviation of the dataset. The latter attributes qualitative uncertainty by what traits qualify them as an expert and the basis of their view to establish its validity. Data recording methods, accuracy of equipment used, or maintainer performance are rarely considered as an attribute to overall uncertainty. A combination of the hard facts and subjective opinion needs to be considered to make informed and effective decisions leading to prosperous outcomes in maintenance. Some cases require more expertise; some require more data. The question here is whether a holistic view of these uncertainties can improve decision-making capabilities and reduce through-life costs as well as unforeseen challenges.
This paper presents a survey questionnaire to rank prominent factors that influence uncertainty in maintenance based on literature and input from industry experts. Respondent qualities are attributed in a pedigree assessment. Results are reviewed and discussed in a series of interviews and validated with wider industrial practitioners before producing a refined survey and pedigree criteria. Results are ranked using the wellestablished Analytical Hierarchy Process (AHP) to determine areas facing the most significant challenges and uncertainties.

What is uncertainty?
Uncertainty is defined as the difference between the information required and that already possessed [1,2]. Risk is the effect of uncertainty on specific objectives [3,4]. Uncertainty is influenced by multiple factors. Some are highly significant, some are negligiblehaving a positive, neutral or negative effect on system performance.
There are two key types of uncertainty, namely: Type A (quantitative, consisting of recorded statistical data); and Type B (qualitative, consisting of heuristic estimates obtained from expert opinion, manufacturer specifications and equipment accuracy) [2]. The quantification of Type A is well documented it is essentially the standard deviation of a given dataset. However, Type B are often overlooked in practice [5].
Uncertainty is further defined as epistemic and aleatory. The former originates from model or data accuracy, influenced by the level of knowledge available, and can therefore be mitigated or optimised. The latter represents statistical variables that constantly fluctuate and therefore cannot be reduced [6][7][8][9].

Decision-making techniques
Saaty's [10] AHP has been extensively implemented and validated to prioritise alternative options via a set of evaluation criteria. Pairwise comparisons are applied to each criterion in a set of matrices to generate weighted scores, which are then aggregated to give a global indication of the best or most popular option [11][12][13][14].
Other multi-criteria decision making (MCDM) methods such as TOPSIS and PROMETHEE can be applied in tandem with AHP to compare complex parameters such as algorithms through fuzzy theory [15][16][17][18]. Other qualitative approaches such as SWOT (strengths, weaknesses, opportunities, threats) analysis can be used to quickly identify risks and factors influencing uncertainties in a group setting, but may result in a plethora of factors that can't be accurately summarised in a quantitative manor with resources available [6,13,19]. AHP is therefore adopted in this study to identify the most significant challenges with a high level of accuracy.

Survey questionnairecore challenges influencing uncertainty
A survey questionnaire is composed to rank prominent factors that influence uncertainty in maintenance based on literature and input from industry experts to gather heuristic data on challenges in industrial maintenance and the underlying uncertainty propagation. Nine responses were obtained from a leading defence company. Respondents scored 32 factors according to their influence on uncertainty on eight-point Likert scales (0-7) from "no influence" to "high influence" to avoid the neutral middle point, with a '0' option for 'no effect' [4,20]. These were refined and adapted by respondents and the author from a list defined by Erkoyuncu et al. [21], divided into 5 categories: commercial, affordability, maintainer performance, operational and engineeringillustrated in Fig.  1. Respondents were each assigned a random ID to protect their anonymity. Respondent years of experience in current and relevant previous roles is illustrated in Fig. 2.

Pedigree assessment
The pedigree matrix scores qualitative, expert opinion against predefined criteria to permit quantitative reliability assessment [22,23]. These criteria are defined according to the contextual application of the study [23][24][25]. The criteria were scored according to: (1) years of experience in current role, (2) years of relevant experience prior to current role and (3.1-5) years of experience working on 5 select ship classes. Each criterion adhered to the same 1 to 5 scale: 1 = <5 years, 2 = 5-9 years, 3 = 10-14 years, 4 = 15-20 years, 5 = >20 years. Explicit roles were not included here to uphold anonymity.
An example of pedigree scores for two respondents is shown in Table 1. The weighted mean of these scores were used as a scaling factor to attribute proportionate scoring to their survey answers. These are compared with the mean scores in Fig. 3. The weights of each criterion were defined by the author and are in themselves inherently subjective. The mean and range for each influencing factor and category were evaluated in MS Excel. This is represented for all factors in Fig. 4, numbered in the x-axis corresponding to Fig. 1. Agreement between respondents is represented by the range, where a high range reflects high disagreement. These can be influenced by a specific project and not necessarily reflect their overall view. Factors that showed contrasting levels of agreement between the respondents are summarised below.

AHP implementation
AHP estimates relative magnitudes of inputs through pairwise comparisons [10]. These were represented in a positive reciprocal matrix adopting an algorithm defined by Erkoyuncu [24] for each of the 5 categories. The resulting weights highlighted the most prominent factors in each category, which were elaborated on in the interviews.

Interviews with industry
Survey results were analysed and discussed in a series of semi-structured interviews with respondents to obtain subjective views across maintenance departments. This structure allowed discussion of relevant topics while permitting respondents to provide further detail on their viewpoint from the survey [13,21,26]. Strategies and examples from literature [20,[26][27][28] were used to structure and phrase the questions to obtain relevant information that can then be put forward to compose a framework capable of predicting the level of subsequent uncertainty influenced by challenges raised. Respondents were assured that responses would be handled confidentially and would not be linked to individuals.
Industrial maintenance today is generally carried out under through-life product-service system (PSS) contracts, where the client makes use of a product in their possession but does not take ownership [29]. This ownership resides with the primary contractor, who coordinates and manages maintenance for the product over its lifetime or the contract duration. Each maintenance manager is responsible for a different class of ship, which present their own challenges.

Core challenges summary
Core challenges that influence uncertainty prediction in maintenance, as highlighted from the questionnaire and interviews, can be summarised in six factors as follows:  Intellectual property rights (IPR), where modern systems are comprised of a vast number of components, many of which can only be maintained by the OEM due to IPR. This yields a degree of information asymmetry leading to uncertainty around the accuracy and availability of technical data; validated by the 'OEM logistics' factor having the single greatest influence on uncertainty in the survey.  be sub-contracted to a third-party OEM, who the primary contractor has no control over. That OEM could be operating under a one-off contract to maintain a specific part or system. Significant uncertainties are raised here for the primary contractor as the time schedule and cost incurred from the third-party OEM cannot be finalised until the contract is completed, which may have knock-on effects for interconnected systems.  Stakeholder communication, where subcontractors may be fully qualified to sign off work done but cannot due to conflicts of interest, so the same task is repeated, resulting in unnecessary time and cost losses. An example was given in the interviews where two maintainers who have not conversed did not know the current material state or planned maintenance schedule of systems that connect at a platform level. The asset, maintained by the OEM, was rendered obsolete by ship staff while on deployment. It therefore missed a planned maintenance period when in dock, meaning the ship could not carry out its tasked duties.  Technology integration, where the exponential progression of technology means that training may not have kept up and software required to interrogate a system for diagnostic checks is not held by maintainers. New builds often have maintenance procedures locked in the design phase. Older platforms experience multiple upgrades over their lifetime which can result in examples such as seven different types of ship under one platform grouped into a maintenance procedure, even though procedures for each type are different. Customer requirements may also change through design and upgrade programmes, which induce substantial costs and schedule delays.
A summary of the core factors that influence uncertainty in industrial maintenance for PSS and current approaches to maintenance is represented by Fig. 5 in a broad sense between the OEM, provider and client.

Wider industrial input
A live survey was carried out with industry practitioners and cost estimators at a workshop on modelling risk and uncertainty. The six core challenges identified were presented using Mentimeter live voting software. Respondents were asked if they considered a combination of statistical (Type A) and heuristic (Type B) uncertainty in their work and to identify their background, achieving 58 responses.
Segmentation of respondents according to their answer to the first two questions is illustrated in Fig. 6 (unknown means the first question was unanswered). 41% of respondents were from the defence sector, 16% from aerospace and 24% cost analysists. A near 50:50 division of backgrounds was found and was relatively equal across each sector. Finally, respondents ranked the six challenges according to their influence on uncertainty, which gained 40 responses. The weighted mean score of each factor is show in Fig. 7, with an area plot response distribution on the Likert scales. Quality of information showed the greatest influence on uncertainty, with a weighted mean score of 6.3. This is followed by technology integration, with a weighted mean score of 5. Maintainer performance, resistance to change and stakeholder communication were found to have a relatively wide spread, indicating disagreement between respondents. However, maintainer performance shows a higher distribution towards 'high influence'. As before, disagreement can be due to respondents' own comparative experiences in their industry in general or on a specific project they are working on. IPR showed the lowest influence, with a weighted mean of 3.1.

Discussion and conclusions
The aim of this paper was to identify and rank core factors that influence uncertainties originating from challenges in the maintenance of complex assets under PSS. Maintenance managers from a leading defence company completed a survey questionnaire identifying these factors. An assessment of the validity of their responses was made through defined pedigree criteria, the results of which were applied to each respondent to normalise their answers. Results were discussed and developed in a series of semi-structured interviews. Mean scores for each factor were weighted using AHP to identify the most influential factors. Core challenges were discussed in Section 5.
The derivation of pedigree criteria is inherently subjective. The criteria selected for this study (Section 3.1) were deemed, through the interviews and academic input, most applicable to score a level of expertise to respondents. Ranking more detailed qualifications against each other adds levels of complexity deemed out of scope for this study.
The AHP allowed factors to be weighed against each other within the survey categories. From this, the six core challenges were determined. These were validated through wider industrial input in a live survey, where quality of information was deemed the most influential factor on uncertainty.
There are approximately 300 different data repositories in use across the studied company, the majority of which are not linked and consist of numerous duplicate entries [30]. This includes DRACAS (Data Reporting, Analysis and Corrective Action System) and UMMS (Unit Maintenance Management System), where data may not be recorded in a useable fashion.
A shared understanding of material state across all departments is required to fill gaps in the supply chain, improve communication between stakeholders, overall decision-making and cost effectiveness of ship support. A common support model (CSM) is being developed to tackle this challenge, featuring five management disciplines for through-life ship support: enterprise, class, design, maintenance and equipment [30]. These are endorsed by a complex web of information and knowledge management that is historically subject to a degree of asymmetry. This was made apparent in the interviews and previous studies across industrial sectors [31][32][33].
A combined understanding of the impact of qualitative and quantitative uncertainty on system performance will provide a holistic picture allowing for more informed and effective decisions leading to prosperous outcomes in maintenance, but this comes at a cost. Budgets can be set for this with the 'spend to save' approach or set aside lump sums for unforeseen circumstances. Ultimately, a trade-off is required.
This study can be extended in several ways for further research. First, a broader framework can be developed to identify contributing factors in a given system, define them as statistical or heuristic, identify acceptable uncertainty parameters for each element and combine the total subsystem uncertainties to gain a more holistic, quantitative picture. Second, the interrelationship between criteria can be incorporated and modelled through other quantitative and qualitative techniques such as the Analytic Network Process (ANP) [14] and PROMETHEE. Third is to develop analytical frameworks in order to better understand potential impacts of uncertainty and the ability to manage them should they arise.