Identification and Assessment of Cost and Effort Criteria for Prioritisation during Root Cause Analysis

Purpose: This study sought to provide a matrix by which to prioritise hypotheses in an Ishikawa diagram while considering a combination of the hypotheses' relation to the problem and the cost and effort to investigate because potential failure causes in an Ishikawa diagram are often prioritised for investigation using subjective methods, and not concrete criteria. Methodology/Approach: A survey was sent to organisations seeking to determine three levels of prioritisation for effort in hours to investigate a problem and costs to investigate. The results were then entered into a matrix that listed three levels of relationship between the hypothesis and the problem as well as three levels of costs and effort. A scenario was given to study participants in the industry to determine if participants with the hypothesis prioritisation matrix could find the correct hypotheses to investigate first, more often than participants without the matrix. Findings: The survey resulted in criteria for prioritisation of hypotheses to investigate. A study with employees in the industry found that participants with a hypothesis prioritisation matrix identified the correct hypotheses to investigate first more often than participants without the matrix. Research Limitation/implication: This paper provides concrete criteria for prioritising hypotheses to investigate during a root cause analysis. Originality/Value of paper: This paper provides concrete criteria for prioritising hypotheses.


INTRODUCTION
Quality failures can be costly for organisations, with failure costs accounting for 70% to 80% of an organisation's total quality costs (Rodchua, 2009).For example, one assembly at an automotive organisation has failure costs of 800 monetary units for every failed inlet camshaft assembly that fails in the organisation (Hirsch et al., 2020).An organisation may also incur warranty costs when the failure happens to the customer (Shang et al., 2022).
Failures that occur must be prevented from happening again, and a Root Cause Analysis (RCA) is performed to identify the causes of the failure (George et al., 2021).The cause of failure must be identified so that actions can be taken to remedy the problem and prevent it from happening again (Sharma et al., 2010).
The Ishikawa diagram is one of the most commonly used quality tools (McDermott et al., 2023a).The Ishikawa diagram is frequently used for problem-solving (Antony et al., 2021), such as finding the cause of a problem (Al-Hyari et al., 2019).
Although the hypotheses in an Ishikawa diagram have long been prioritised for investigation (Gryna, 2001) and a generic prioritisation matrix exists (McDermont et al., 2023b), there is no defined criteria for assigning priorities based on considerations of relation to the failure, together with cost and effort.Barsalou (2023) describes prioritising Ishikawa diagrams by considering both the cost and effort to investigate and how well the available evidence supports the hypotheses in the Ishikawa diagram.However, the lack of defined criteria leaves ratings for cost and effort subjective, with each person deciding on their own what could be considered a high, medium, or low rating.Criteria for prioritising based on the relation to the failure is available (Barsalou, 2022), but not for rating cost and effort or the interactions of relation to the failure, cost, and effort.This paper aims to establish criteria for prioritising hypotheses to investigate during an RCA based on considerations of cost and effort.

LITERATURE REVIEW
The Ishikawa diagram is frequently used during Six Sigma projects for the listing of potential failure causes (Uluskan, 2017) as illustrated by many case studies (Yadav et al., 2019;Uluskan and Oda, 2020;Solanki and Desi, 2021;Trakulsunti et al., 2022;Araman and Saleh, 2023).The use of an Ishikawa diagram to solve problems is well documented in the literature (Pyzdek and Maciulla, 1995;Sharma et al., 2010;Mahanti, 2014).However, research on prioritising hypotheses in the Ishikawa diagram using objective criteria is lacking.Solanki and Desai (2021) present a Six Sigma case study in which an Ishikawa diagram with 18 potential causes was created during a brainstorming session.The potential causes were prioritised using multi-voting, which often fails to consider the time and effort required for the investigation actions during an RCA where one specific cause may exist.
A case study by Yadav et al. (2019) describes the use of an Ishikawa diagram during a Six Sigma project.The Six Sigma team created the Ishikawa diagram, and then causes to investigate were selected using the nominal group technique, where participants rank the importance of the items under consideration, and the aggregate of the rankings is used for selection criteria (Srivastava et al., 2019).With the nominal group technique, expert opinion is used to derive individual rankings, not concrete criteria.Barsalou (2023) presented a case study in which hypotheses from an Ishikawa diagram were copied to a worksheet and prioritised with three levels of prioritisation; however, the priorities were low, medium, and high, but little guidance was given on how to prioritise other than suggesting to use three levels of prioritisation that consider cost and effort.However, criteria for cost and effort are not given.
A case study by Doshi et al. (2012) described an Ishikawa diagram for a problem with radiator fins opening.The Ishikawa diagram had 20 potential causes that were transferred to a table listing the top branch of the Ishikawa, the hypothesis, and the hypotheses' contribution to the problem rated as high, medium, and low based on a combination of the investigators' experience and knowledge of the product.The rating was based on experience and knowledge of the product, but time and effort were not considerations, which could lead to investigating potential causes that are more costly and time-consuming to investigate prior to investigating potential causes that are quick and low-cost to investigate.
Such methods fail to consider the cost and effort to investigate a hypothesised failure cause.Therefore, a study was conducted to identify costs and effort criteria to use as prioritisation criteria, which can provide organisations with concrete criteria for prioritising hypotheses to investigate.

METHODOLOGY
The literature does not provide criteria for prioritising hypotheses for investigation that consider combinations of links to the cause, costs, and efforts when using an Ishikawa diagram.For the criteria establishment, a survey was sent to organisations to define better criteria for rating costs and effort for investigating a hypothesis.A sample of organisations was selected so that the criteria would be representative of those that would use the criteria on the job.
The organisations were randomly selected from a list of ISO 9001-certified organisations from a region in Poland that was used in a previous study.One hundred organisations had originally been randomly selected from the list; however, three had since gone insolvent, resulting in 97 organisations receiving the survey.
The survey asked for demographic data such as the responding organisation's industry and size and the position of the person taking part in the survey.There were 47 respondents with a response rate of 48.5 percent; however, two failed to provide answers and were excluded from the analysis.One organisation stated that costs were not considered; one referred to standard labour costs without stating the costs and three respondents listed zero for minimal, moderate and extensive costs.These five respondents were excluded from the analysis, resulting in 41 responses for cost.
One respondent listed a range from 1,000 to 3,000 for moderate cost, which was converted into a mean of 2,000 for analysis purposes.Two respondents did not give a response for effort, and three listed zero for minimal, moderate, and extensive effort.They were excluded from the analysis, resulting in 42 responses for effort.
The respondents were in the automotive industry (23.4%), consumer goods (10.6%), and various non-identified industries (66.0%) and the organisations ranged in size from 1 to 10 employees to over 501 employees.The respondent's positions included engineers and managers.
For identifying cost ratings, the survey asked questions regarding costs with answers given in Polish złoty and questions regarding effort with answers given in hours.The questions are listed in Table 1.The survey results were used to identify criteria for prioritising, taking into consideration cost and effort.First, the median value of the high, moderate, and extensive ratings for both cost and effort were determined.Then, median values were used to calculate concrete prioritisation criteria.
The median values and ranges were calculated for both cost and effort using the procedure described in the methodology.Values ranged from zero to 65,000.Therefore, the median values were used to determine low, moderate, and high costs due to the wide range of results.Different industries and organisation sizes were represented, so the median is more representative of the data than the mean, which could be influenced by extreme values because the median is less sensitive to outliers (McShane-Vaughn, 2016).Also, the results must be general enough to apply across industries at organisations of different sizes; therefore, the median provides a more generalisable value.
For determining the maximum value for low cost, half of the difference between the median value for moderate and low was divided by two and added to the median of low and was calculated as The same procedure was used to determine the upper limit of low effort in manhours, where the difference between moderate effort and low effort was divided by two and subtracted from the median of moderate effort.This was calculated as The resulting evaluation table for cost and effort is shown in Table 3.An efficacy study was conducted to determine if using the hypothesis prioritisation matrix led to correctly identifying the hypotheses investigated first more often than not using a hypothesis prioritisation matrix.The study used employees in the industry to represent those who would actually apply the concept on the job.
A matrix was created to perform an efficacy study with cost and effort on the yaxis and cost on the x-axis.Literature was consulted to determine high, medium, and low prioritisation ratings.Smith (1998) once recommended selecting three to five hypotheses to investigate.More current case studies have used only two or three hypotheses for the first investigation.For example, Sarkar et al. (2013) selected two hypotheses to investigate, while Germanova-Krasteva and Petrov (2008), Chapman et al. (2011), Jayaprasad et al. (2016), and Shamsuzzaman et al. (2023) selected three hypotheses to investigate.Therefore, the prioritizations in the matrix were set so that most prioritizations would be medium, with only two of nine possibilities being classified as high.
Once clear criteria for cost and effort were established, the efficacy of the method needed to be evaluated.For performing it, volunteers were selected for the study.
The volunteers consisted of employees in industry working at manufacturing and production companies.Four university students completing internships at manufacturing organisations administered the surveys by asking for volunteers.
There were 30 efficacy study volunteers in positions ranging from technician to executive, with seven executives, two managers, five engineers, eight technicians, and one operator.Seven study participants did not identify a position.The volunteers were in four organisations.One organisation had 11 to 100 employees, two each had 101 to 500 employees, and the fourth had over 500 employees.There were four volunteers with less than two years of experience, three with over 20 years of experience, five with seven to ten years of experience, nine with three to six years of experience, and 11 with 11 to 20 years of experience.All but two study participants were in a quality role.
The volunteers were separated into two groups, and each group was given the matrix shown in Table 4.Although the matrix given to the volunteers listed złoty, the matrix has been updated to show the equivalent in euros.Hypothetical potential failure causes were also listed in Table 4.The hypotheses were then given additional details in relation to the cause, cost to investigate, and effort to investigate.The "relation to the cause" column describes how well evidence supports the hypothesis.The "Cost to investigate in złoty" column lists how much a hypothesis would cost, and the "Effort to investigate in man-hours" column lists how many hours it would take to investigate a hypothesis.The final column provides a place for study respondents to list a prioritisation, which will be explained later.
Only four sets of additional details would result in a high rating of the hypothesised failure causes.The volunteers in both groups were instructed to "Prioritise the hypothesis based on relation to cause, cost, and effort using a scale of High, Medium, and Low to determine which hypothesis should be investigated first."The first group was only given Table 4.The second group was also given Table 5, which was created as described at the beginning of this section.The study participants with Table 5 were also instructed to use the cost and effort versus relation to cause matrix to assign priorities.The hypothesis "shaft damaged" is greatly suspected to be a cause and has a cost of 850 złoty and the effort to investigate is 17 hours; "lever sticking" is also greatly suspected to be a cause and has a cost of 975 złoty and an effort of 16 hours."Sodium contamination" is also greatly suspected to be a cause and has a cost of 700 złoty and an effort of 17 hours, and "insufficient spring force" is greatly suspected of being a cause and has a cost of 1,100 and an effort of 12.These four are the only answers that would be rated as high according to the hypothesis prioritisation matrix, which was used as the evaluation criteria.
The main criteria for the efficacy study was the total number of mistakes made, with failing to correctly identify a cause as a high priority considered incorrect and incorrectly identifying a cause as a high priority also considered incorrect.The study results are shown in Table 6.The results were then evaluated using a hypothesis test of two proportions, which is used to determine if a statistically significant difference exists between two proportions using a given critical value (Laman, 2022).In this case, the critical value of 0.05 was selected.There are two possible p-values for the hypothesis test of two portions.One is the normal approximation, and the other is Fisher's exact, which should be used when there are less than five occurrences or non-occurrences of events (Barsalou and Smith, 2018).There are more than five events and nonoccurrences of events; therefore, the normal approximation can be used.
The comparison of correctly identified high ratings had a p-value less than 0.05, indicating a statistically significant difference.The volunteers with the prioritisation matrix correctly identified the causes to rate as high more often than the volunteers without a prioritisation matrix.The hypothesis test of two proportions for the number of causes incorrectly prioritised as high resulted in a pvalue less than 0.05; therefore, there is a statistically significant difference in the number of causes incorrectly rated as high.A hypothesis test of two proportions was also performed to determine if the total number of mistakes differed between the two groups.The resulting p-value was 0.05; therefore, there is a statistically significant difference between the total number of mistakes made.The results of the hypothesis tests of two portions are shown in Table 7.

DISCUSSION
There are many methods for prioritising potential causes in an Ishikawa diagram (George et al., 2021), when an Ishikawa diagram is used for RCA (Mahanti, 2014).Methods of prioritisation include multi-voting (Solanki and Desai, 2021), nominal group technique (Yadav et al., 2019), and a prioritisation matrix rating potential causes on a scale of one to ten (Araman and Saleh, 2023).Other methods of prioritising include using a worksheet using three levels of prioritisation based on the problem solver's experience and knowledge of the product (Doshi et al., 2012).These methods are all opinion-driven and fail to consider the cost or time required to investigate.
This study has produced a matrix of objective criteria for considering both the cost and time to investigate a hypothesis while considering how well the evidence supports a potential cause.The cost and effort rating could be used together with the strength of the evidence rating to prioritise hypotheses as high, medium, or low.
The cost and effort criteria were determined based on a survey of organisations in the industry.Using the established criteria, the efficacy study used volunteers in industry to compare the use of a matrix with effort, cost, and relation to the cause.The differences between the two groups were statistically significant when looking at the total number of causes that should be rated high, the total number of causes incorrectly rated as high, and the total number of mistakes.The difference between the two groups is clear; however, a limitation is that the majority of volunteers reported being in a quality-related role.An opportunity for future research would be to repeat the study with both volunteers in a quality-related role and volunteers not in a quality-related role.It is currently unclear if the matrix suits production operators without training to compensate for a lack of quality experience.
This study has some additional limitations.The sample was taken through a procedure close to convenience sampling, known for not representing the population (McShane-Vaughn, 2016); therefore, caution is needed when generalising the results.The ratings for cost and effort were identified through a survey of organisations in one region of one country.Although they can serve as a starting point, they may not be reflective of organisations in other countries with different economies.The survey also included small, medium, and large organisations, and smaller or larger organisations may have different values that they would consider high or low for costs.
The study has practical implications for managers.The results of these studies can be applied in organisations using an Ishikawa diagram when multiple explanatory hypotheses are available, and prioritisation is needed to determine which hypotheses should be investigated first.This would result in the first hypotheses investigated being those that have strong evidence in support of the hypotheses, or a low amount of time required for the investigation, or a combination of the two.
Weakly supported and high-effort hypotheses will be investigated later.Such an approach can lead to a more efficient identification of root causes.
The study also has implications for researchers.The survey results are from one country.Additional studies could be performed to identify cost criteria across highcost and low-cost economies.Alternatively, researchers could identify a correction factor.Organisations can convert the cost criteria into something more applicable to the organisation in consideration of organisation size, industry, and economy.

CONCLUSION
This paper sought to identify criteria for the evaluation of hypotheses from an Ishikawa diagram in consideration of the cost and effort involved in investigating them.Once identified, the criteria were then evaluated to determine if users could correctly identify the combinations of the strength of the evidence and the cost and effort to investigate.Criteria were derived from a survey, and the approach's efficacy was verified in a study.
A survey was used to determine what should count as high, medium, and low for costs, and a matrix was created using this information.The matrix was then validated through the use of a study to determine if more study participants using the matrix could find the four hypotheses with the strongest relation to the problem and the lowest costs and efforts versus a group of study participants that did not use the matrix, with those using the matrix correctly identifying the hypotheses to prioritise more often than those who did not use the matrix.
Categories for splitting effort and costs into three levels were gained through the use of a survey.The matrix with cost and effort values for prioritisation provides clear criteria for prioritising hypotheses to investigate during an RCA.
Organisations can use the results of the study for the prioritisation of hypotheses to investigate during an RCA in consideration of the cost and effort to investigate together with how likely a cause is believed to be the root cause.This approach would both provide concrete prioritisation criteria and lead to a more efficient prioritisation using three levels of prioritisation.
These levels can be adapted for specific use within an organisation; for example, a smaller company may view 1,000 monetary units for a test as expensive, while a large organisation may view such costs as low.Organisations should also convert the cost to their local currency.
A value modification for effort may also be needed if an organisation perceives a difference between the given values and what the organisation would consider low, moderate, and extensive.However, the current values can be used by an organisation as a starting point, with the values adjusted up or down as needed.

,
+    −    2 The lower limit for extensive was determined by subtracting the median value for moderate from the median value for extensive and dividing by two and was calculated m    −    −    2

,,
m    −    −    2 For determining the lower limit of extensive effort, the difference between the median of extensive effort and moderate effort was divided by two and subtracted from the median of extensive effort, calculated as m    −    −    2

Table 1 -Cost and effort questions Cost Questions Effort Questions When
planning to investigate a failure, what would you consider to be minimal costs?When planning to investigate a failure, what would you consider to be minimal effort?When planning to investigate a failure, what would you consider to be moderate costs?When planning to investigate a failure, what would you consider to be a moderate effort?When planning to investigate a failure, what would you consider to be extensive costs?When planning to investigate a failure, what would you consider to be an extensive effort?
Table 2 depicts the mean, minimum value, median, maximum value, and range of values for costs in monetary units and the same for effort estimates in hours.

Table 2 -
Results for costs in monetary units and effort in man-hours

Table 3 -
Ratings based on cost and effort

Table 4 -
Relation to cause, cost, and effort

Table 6 -
Efficacy study results

Table 7 -
Statistical tests of two proportions