Key Indicators for Organizational Performance Measurement

Each organization for assessing the amount of utility and desirability of their activities, especially in complex and dynamic environments, requires determining and ranking the vital performance indicators. Indicators provide essential links among strategy, execution and ultimate value creation. The aim of this paper is to develop a framework, which identifies and prioritizes Key Performance Indicators (KPIs) that a company should focus on them to define and measure progress towards organizational objectives. For this purpose, an applied research was conducted in 2013 in an Iranian telecommunication company. We first determined the objectives of the company with respect to four perspectives of BSC (Balanced Scorecard) framework. Next, performance indicators were listed and paired wise comparisons were accomplished by company's high-ranked employees through standard Analytic Hierarchy Process (AHP) questionnaires. This helped us establish the weight of each indicator and to rank them, accordingly.


Introduction
Organizations in today's worldwide competitive environment, must be able to evaluate their objectives such as unit cost, profit, subjective (e.g.quality, satisfaction) performances and setup appropriate strategies to reach their goals.A business process is a set of activities performed in order to reach common goals based on well-defined company objectives (Hammer & Champy, 1994;Keung & Kawalek, 1997).These processes contribute towards the achievement of aims and necessary objectives.Magretta and Stone (2002) stated that indicators and performance measurement were critical elements in translating an organization's mission or strategy, into reality.Indicators and strategy are tightly and inevitably linked to each other.Strategy without indicators is useless and also indicators without a strategy are meaningless.Edwards and Thomas (2005) argued that KPIs were compilations of information used to measure an assess performance.In addition, they indicate the final mark of a company's efficiency and effectiveness.Vucomanovic et al. (2010) stated that KPIs represent the basis for measuring business and project success.They enable the measurements of performance within firms, industry, and to initiate benchmarking.In addition, KPIs are implemented as a means of communication within stakeholders to inform them about improvement of endeavors constantly.Parmenterg (2007) noted that KPIs should have characteristics such as: nonfinancial measure, frequently measured, acted by the CEO (Chef Information Officer) and the senior management team, understood by all employees, ties responsibility to the individual or team, and significant and positive impact.Išoraite (2010) mentioned the best value performance indicators at five dimensions of: (1) Strategic objectives: why the service exists and what it seeks to achieve, (2) Costs/efficiency -the resources committed to a service: the efficiency with which they are turned into inputs, (3) Service delivery outcomes -how well the service is being operated in order to achieve the strategic objectives, (4) Quality -explicitly reflecting user's experience of services, (5) Fair accessrelating to case and equality of access to service.
Performance measurement in organizations is still largely concentrated on financial data for the purposes of coordination and control (Atkinson & McCrindell, 1997;Atkinson et al., 1997).Several researches investigated the effectiveness of balancing financial and non-financial measures on performance evaluation systems (Bremser & Barsky, 2004;Hudson et al., 2001;Kanji & Sa', 2002;Kerssens-van et al., 1999;Savioz & Blum, 2002).Making the connection among performance, strategy and organizational purpose is a challenging task.Taking into consideration the complexity of the phenomenon, many are in favor of using multiple perspectives and multiple measures of organizational performance (e.g., Barney, 2010;Chakravarthy, 1986;Venkatraman & Ramanujam, 1986;Bentes et al., 2012).The choice of important indicators has influenced on the operation and the direction of the organization.In addition, the identification of clear goals, matching the strategy and acceptance of those involved is required (Išoraitea, 2010).Measuring and improving performance also depends on the proper selection of effective performance models and indicators (Meng & Minoque, 2011).The previous studies indicated the positive relationship between managerial perception and the implementation of key performance indicators within a firm (e.g.Cox et al., 2003;Kaplan & Norton, 1993).Sawang (2011) explored the relationship between perceived importance and the actual use of financial and non-financial indicators capturing technical and administrative innovation performances among small and medium sized organizations in Thailand.Jovanovic and Krivokapic (2008) identified key performance indicators for the perspectives of balanced scorecard (BSC).Konsta and Plomaritou (2012) examined the applicability and usefulness of performance indicators in shipping management performance and evaluation.Delgado and Santiago (2014) introduced a set of KPIs considered essential to allow TETRA operators to be aware of whether or not provided services meet the quality of service requirements established by end users.
This study aims on defining a framework based on integrating two methodologies of BSC, a multiple perspective framework for performance assessment, and analytical hierarchy process (AHP) (Saaty, 1980(Saaty, , 1990)), a structured technique for organizing and analyzing complex decisions, to prioritize the KPIs in an organization.The paper is organized as follows.The Section 2 is devoted to research methodology.In Section 3, we demonstrate a hierarchical model for prioritization of performance measurement indicators.Section 4 includes results and discussion and section 5 summarizes the contribution of the paper.

Methodology
Multiple criteria can be developed and implemented for organizational performance assessment.The challenge is to design a structure for indicators (i.e., grouping them together) and to extract an overall sense of performance from them (i.e., being able to address the question of "Overall, how well are we doing?").Different approaches have been proposed for developing such an integrative system.The proposed study of this paper has applied BSC as presented by Kaplan and Norton (1992, 1996, 2001) and elaborated by others (e.g., Ittner & Larcker, 1998).BSC is a multi-dimensional framework that translates a company's strategy into specific measurable objectives.The role played in measuring the performance by BSC, which is the plain consideration of multiple performance perspectives rather than just a strictly financial standpoint, causes complexity to measure performance.It forces top management to recognize multiple activities carried out for cooperating success and management.Monitoring of these activities must be balanced and little attention has been devoted to this topic within the field of operations management.BSC considers several relevant dimensions of organizational performance but it does not formally explain how to weight their importance in a comprehensive framework.In practice, however, perspectives and their indicators do not often have equal importance.AHP (Saaty 1980) as a useful tool to prioritize and to consolidate performance indicators, based on multiple criteria, can be a promising mechanism to overcome the limitations of BSC.Pair-wise comparison is a method to measure the weights for criteria and indicators in AHP.In this method, criteria and indicators are compared with each other and the degree of importance for each criterion or indicator with respect to each other is specified.Saaty (1980) proposed a ratio scale between 1 and 9 with a value of 1 indicating no preference and a value of 9 indicating very strong preference.Harker (1998) noted that this scale has been derived by Saaty from insights of psychological science and has been defended on empirical grounds.Deployment of AHP in real-life decision making involves successive comparisons between every two alternatives, criterion by criterion, according to a 9-point scale as presented in Fig. 1.If an alternative A i is preferable to an alternative A j , then the value of the comparison scale indicates the intensity of relative importance of A i over A j , assigned by the decision maker.The scale helps, in a pair wise comparison, the investigator establish which alternative is more suitable.Higher values of a ij indicate stronger preference of alternative A i over A j .The comparison of one pair of alternatives at a time for each decision criterion (instead of a simultaneous comparison involving all alternatives and criteria) reflects the assumption of a decision maker and reveal the preferred alternative by analyzing one object at a time (Voronin, 2007).By considering all possible pair wise comparisons between alternatives, a matrix A results that can represent the relative importance a ij of each element over each other.Given that an element is as important as itself and taking into account the theorem of reciprocity, if i=j then a ij =1 and if i≠j then a ij = 1/a ji .The calculation of weights relies on an iterative process in which matrix A is successively multiplied by itself, resulting in normalized weights, w i .The process halts when the difference of weights between successive iterations is smaller than a given halt criterion.In this framework, w i represents the importance of alternative A i relative to all other alternatives.Note that the normalization process entails that weight components total 1.0.The judgment of decision makers, in pair wise comparisons, may present inconsistencies when taking into consideration all alternatives simultaneously.In order to the comparison matrix be consistent, it should be a ik =a ij .ajk .However, decision-makers, when comparing alternatives, often violate this relationship.Sharma and Bhagwat (2007) noted that the consistency index (CI) and the consistency ratio (CR) measure the degree to which judgments are not coherent.If CR<10%, then the degree of consistency is satisfactory (Saaty, 1990).

A hierarchical model for prioritization of key performance indicators
The aim of this paper is to define a methodology to improve the quality of prioritization of organizational key performance indicators.For this reason, we planned a research and investigated the relative performance indicators in an Iranian telecommunication company.In order to select performance indicators and to guide comparative judgments, 27 employees were selected to participate.We selected respondents among those who have the appropriate information on indicators and also preferentially associated with them.The 27 employees were chiefs and telecommunication experts who have long experience in the company.Since managers from different levels may offer particular images, we tried to choose 27 managers from different managerial levels (operational, middle and senior).We first used a library method counting books, articles, journals, research projects, and online databases to establish primary indicators associated with each balanced scorecard's perspective based on the company's strategic plans and goals.Then a meeting was arranged with the interviewees.The participants were asked to choose the most important indicators with respect to the company goals based on their experiences and opinions.All participants were allowed to choose indicators from four perspectives of BSC.Among the indicators, those were proposed jointly by all the participants were selected and classified into four perspectives.Then a discussion with senior executives of a group has been arranged about the adequacy and completeness of the indicators that reflect company's key objectives.Finally, nineteen indicators were determined as final key indicators, including four indicators in financial perspective, seven indicators in processes perspective, five indicators in customer perspective and three indicators in learning and growth perspective.
In the next stage, a standard AHP questionnaire was designed based on the proposed indicators, i.e. the matrix of paired comparisons was used to answer the questions.The questionnaire is composed of two parts: i) comparison of four balanced scorecard perspectives relating to the company, ii) comparison of the indicators relating to each perspective.After formulation of questionnaire, a preliminary survey is performed on it.The purpose of this survey is to resolve the potential problems.Therefore, we distributed the questionnaires among the research sample and then they were asked about clarity or ambiguity of the questions.Based on responses, some questions were revised and the final questionnaire was adjusted.The chosen method, integrating BSC and AHP, requires a hierarchical structure to yield a result.Criteria in the model are considered as financial perspective, internal processes perspective, customer perspective, and learning and growth perspective.Financial perspective is characterized by four subcriteria: Reducing the establishing costs of each phone line and ADSL system (A), reducing the maintenance costs of each phone line and ADSL (B), reducing the percentage of uncollectable (C), increased monthly revenue per fixed line and ADSL (D).Internal processes perspective is characterized with seven sub-criteria: The number of sets of fixed telephone and ADSL switches (E), sets of fixed telephone and ADSL (F), the number of data ports transfer (G), the number of network ports (H), cities have access to the data network (I), telephone and ADSL fault clearing time (J), percentage failure of Telephone and ADSL (K).Customer perspective is characterized with five subcriteria: Penetration co-efficient of fixed telephone and ADSL (L), the success rate of calls (M), waiting time for fixed telephone and ADSL (N), pay and benefits of employee performance (O), education and promotion (P).Learning and growth perspective is characterized with three subcriteria: Time management training (Q), time employee training (R), and the number of offers (S).
The AHP hierarchical structure of this study appears in Fig 2 .The overall goal is the top issue and perspectives and indicators are in the next levels.

Fig. 2. Hierarchical structure of criteria
In this study, Expert Choice software is used for data analysis.One advantage of pair wise comparison of this software is that priorities or weights do not enter as you wish (contractors) but these values are obtained from the numerical verbal or graphic judgments.This method for deriving priorities is an authentic knowledge, which formed from the foundation of mathematics (Nikmardan, 2007).According to the theoretical basis of the AHP technique, the credibility of information is determined based on the inconsistency rate for the paired comparisons.If inconsistency rate of paired comparisons matrix is less than 0.1, it will be valid questionnaires otherwise, software helps us to find and fix inconsistent data.After collecting the questionnaires, the data tables imported into the computer and the inconsistency rate is calculated by the software.
Forman and Peniwati (1998) discussed two methods for aggregating individual responses in terms of a group response.One method is simple averaging across the assessments (of the values of priorities and of the performance level of each alternative in each indicator) produced independently by the evaluating judges, and another method is an agreement-building approach whereby evaluating judges reach some consensus about the value of priorities and of performance levels.While some researchers employ the averaging approach (e.g., Chou et al., 2004;Javalgi et al., 1989), others prefer the agreement-building approach (e.g., Fletcher & Smith, 2004;Shahin & Mahbod, 2007;Kumar & Bhagwat, 2007), who employ the opinions of the majority of the interviewees.This study uses simple averaging across the assessments.Then, we applied geometric mean of 27 questionnaires using Excel software, and the resulted data analyzed by Expert Choice software.Geometric mean is used to combine individual comparisons to group comparisons and form paired comparisons matrixes.Since paired comparisons will create the ratio data, the geometric mean is the best for them.Moreover, the inverse matrix of comparison justifies the use of this mean more than anything (Azar & Memariani, 1994).

Results and Discussion
Nineteen performance indicators according to four criteria were identified and weighted by experts, and prioritized through AHP.The weight of each criterion determines the importance of the criterion against each other, leading to the attainment of the company goals.Table 2 shows raw and normalized weights, as well as the consistency ratio of the 4×4 matrix.With regard to the results of AHP method, the financial perspective is much more important than any other performance perspectives.As the results show, customer and internal processes have almost equally important.At the second step of the analysis, indicators of each perspective were compared with each other to form pairs.After calculating the indicators' normalized weights, the global weight of each performance indicator that shows its contribution to the overall objective, was determined.Calculating weights in AHP consists of two parts: relative weights and global weights.Relative weights are product of paired comparison matrices; while the global weight of each indicator in a hierarchical view is the product of performance indicator local weight times the respective BSC perspective local weight (see Table 3).For example, the global weight of indicator A is 0.044*0.366=0.016.Table 3 presents normalized local and global weights of performance indicators in the financial perspective.The consistency ratio (CR) is below the threshold of 0.1, which is acceptable (Saaty, 1990).

Table 3
Relative importance normalized local, global weights and consistency ratios at the financial perspective indicators.
After necessary mathematical calculations to get the eigenvector of each matrix, we obtain in the financial perspective, indicators D (Increased monthly revenue per fixed line and ADSL) and C (Reducing the percentage of uncollectable) occupied the top priority among other criteria with 51.7%, and 26.8% respectively.Therefore, the company should increase monthly revenue by providing value-added services, encourage customers to more use the phone, transfer phone to applicants with more calls (commercial customer, office customer, etc.).In addition, to reduce the percentage of uncollectable, it should use incentive methods to pay debts (payment by installment, reduce the percentage of debt, etc.).This procedure continued for indicators belong to other perspectives and prioritizing and ranking have been set for them and the result is illustrated as the following: In the processes perspective, indicator J (Telephone and ADSL fault clearing time) with 31.3% and indicator K (Percentage failure of Telephone and ADSL) with 23.5% occupied the top priority among other criteria (see Table 4).Therefore, in order to reduce fault clearing time, company should inform consumers on how to wiring and use a modem and telephone and train MDF personnel and fixing broken officers.In addition, periodic visits to different parts of the network and renovations, according to visits can be effective.In order to increase the number of set fixed line and ADSL, systems and cable networks should be developed proportional to needs of applicants in each region or be replaced with new communications systems that enable provides service for voice and video, and data simultaneously for consumers.

Table 4
Relative importance normalized local, global weights and consistency ratios of the internal processes perspective indicators.
In the customer perspective, indicator M (The success rate of calls) with 50% and indicator L (Penetration co-efficient of fixed telephone and ADSL) with 37% occupied the top priority among other criteria (see Table 5).Therefore, the company can increase the success calls with check and maintain systems periodically, resolve problems and improve and modernize them if necessary, use of private call center systems in offices and large organizations, use of special services, use of fax systems and answering machine.In the learning and growth perspective, indicator Q (Time management training) with 49.3% occupied the top priority among other criteria (see Table 6).Theoretical and practical training of management in the fields will have a great impact on guidance the subsets to present the quality of services and enhance productivity.Then its relative importance compared to other indicators seems reasonable.(BSC perspectives).Although this method leads to finding the final global performance indicators, it seems to preserve against individual bias of decision-makers by a check of convergent validity.Thus as a final step, the center head rank-ordered the nineteen performance indicators and their values, based on the list (stated in the original units, not in the nine point scale).His results were the same as that obtained by the AHP method.As a description, the center head explained that increased monthly revenue per fixed line and ADSL was more accessible capacity than other indicators with increasing the quantity and quality of services as well as providing value-added services.Relative importance of the financial perspective compared to others can be justified based on several important reasons.One of the most basic reasons is privately-held telecommunications company and its presence in stock and expected shareholders and stakeholders for high EPS (Earning per Share).Obviously, the presence and survival of the company depends on the financial situation.Although, the possibility of offering new services, enhancing existing services, customer satisfaction and preserving them are resulted from good financial status.

Conclusions
This article investigated an integrated method for determining and ranking key performance indicators within an Iranian telecommunications company.This approach provides a general framework and helps managers in selecting performance indicators.Indicators enable managers and workers to assess and control the performance of the resources in which they are responsible.They connect managers to internal personnel for purposes of control, and to external stakeholders for other purposes as well.Many times stakeholders and users of indicators do not understand the workings and processes of a firm, nor do they need to.Well-designed indicators provide the users a sense of knowing the needs to be accomplished without necessarily requiring understanding the intricacies of related processes.Indicators with poor development or implementation can lead to frustration, conflict, and confusion.In addition, indicators identify gaps between performance and expectation that ideally point the way for recuperation.The size and the direction of the gap (positive or negative) provide information and feedback that can be used to identify productive process adjustments or other actions.Performance indicators influence employee's behavior and inappropriate indicators may lead to non-functional behaviors of employees.Employees who improve just their own performing indicators may make decisions that are conflict with improving performance of their department or demands of managers.This leads to damage to other parts or even the overall performance of the organization.The proposed method considers a variety of performance indicators to cover important aspects for succession the organization.It appropriately focuses on short and long term results, various functions (cost, quality, delivery ...) and aspects (customers, stakeholders, innovation …) and different levels of organization (overall parts of company).By applying BSC and AHP methodologies, managers must employ a complicated framework for ranking of performance indicators.The particular numerical values of weights for perspectives and indicators are specific to the case studied.This paper not only shows the company's features, but also telecommunication industry features, country environment and time-based moment.However, the procedure for concluding relative weights and rankings of them which presented here is sufficiently general to be used in other firms, regardless of industry or country.

Fig. 1 .
Fig. 1.Scale of comparisons of the AHP Table 1 illustrates criteria (BSC perspectives) and sub-criteria (indicators) of the research model.

Table 2
Weights and consistency ratio at the BSC perspectives

Table 5
Relative importance normalized local, global weights and consistency ratios of the customer perspective indicators.

Table 6
Relative importance, normalized local, global weights and consistency ratios of the Learning & growth perspective indicators.

Table 7
shows the prioritized key performance indicators with relation to different criterion

Table 7
Weight and rank of criteria