Modified assessment tools for measuring innovativeness level of companies

Abstract The innovation role in Indonesia as economic growth and prosperity driver has not been fully optimized. Indonesia ranked 88th from 128 countries that included in the list of world’s most innovative countries. It shows that Indonesia must develop and improve itself within several indicators of innovation. One of the assessing aspects is related to the role of industry or company in innovation areas. The companies’ role can be assessed more extensively by assessing the innovative excellence of organizations using assessment tools. By identifying the level of innovativeness maintenance and improvement related to better performance are constantly done. Existing assessment tools are modified in order to result in a framework that is easily understandable and suitable with the background of Indonesian companies. Methods used for modifying assessment tools are Delphi method to validate the research sub-variables, Analytical Hierarchy Process (AHP) method to determine the importance weight of each sub-variable, and in-depth interview method to collect the assessment data. The modified assessment tools can be utilized to analyze the strengths and weaknesses of the company evaluated from the innovation excellence aspects so that the company could then determine the maintenance, improvements and the refinements needed. The score of 1,000 is the maximum value that can be obtained.

Abstract: The innovation role in Indonesia as economic growth and prosperity driver has not been fully optimized. Indonesia ranked 88th from 128 countries that included in the list of world's most innovative countries. It shows that Indonesia must develop and improve itself within several indicators of innovation. One of the assessing aspects is related to the role of industry or company in innovation areas. The companies' role can be assessed more extensively by assessing the innovative excellence of organizations using assessment tools. By identifying the level of innovativeness maintenance and improvement related to better performance are constantly done. Existing assessment tools are modified in order to result in a framework that is easily understandable and suitable with the background of Indonesian companies. Methods used for modifying assessment tools are Delphi method to validate the research sub-variables, Analytical Hierarchy Process (AHP) method to determine the importance weight of each sub-variable, and in-depth interview method to collect the assessment data. The modified assessment tools can be utilized to analyze the strengths and weaknesses of the company evaluated from the innovation excellence aspects so that the company could then determine Bambang Purwanggono

ABOUT THE AUTHOR
The study aims to identify the variables system related to innovativeness of companies, determine the importance and contribution level of variables and sub-variables to the innovativeness of company, and develop assessment tools for measuring innovativeness level of companies. This paper is a part of continued research to formulate an innovativeness assessment tool to be used or implemented by Indonesian companies. This modified assessment tool is directed to suit the need of assessing innovation activities done in Indonesia companies to improve the country's innovation position globally. Accommodating local situations is one main factor of modification. A framework of Indonesiaspecific assessment tool for innovation is the objective of the continuing research. The ultimate goal of this research and more research to follow is to come to a national level assessment tool for national companies to result in the position of their innovativeness levels after knowing the criteria or sub-factors of the scoring. Continuous improvement on innovation can be assessed from time to time.

PUBLIC INTEREST STATEMENT
Innovation has been and will be the driver of a nation's state of wealth. It is now an innovation-driven economy. The position in the global innovation ranking of a country or economy will depend on the innovativeness of its companies and enterprises working on product innovation. This will comprise the nation's index of innovation which leads to the indications on how good the nation is the development process.
Indonesia is still a developing country that needs more and more innovative companies within the country to always improve the economy of the people. Companies need to be assessed on how good on or level of their innovativeness to make improvement needed accordingly. Up to now, assessment tool available is coming from overseas not yet to adopt local or national factors. Modifications are made to more adaptive to what innovation is about in the national companies.

Introduction
Role of innovation in a company can be measured using some particular assessments or measurements methods or tools. One of the innovativeness assessments related to company's innovation excellence is Global Innovation Index (GII). GII is jointly published by Cornell University, INSEAD, and World Intellectual Property Organization (WIPO). It has been recognized globally. This innovativeness level assessment can show how active companies are on innovation activities as input factors of GII and how companies creatively produce knowledge and technology as output factors of GII. The company's assessment in GII demonstrates that the innovation performance of companies in a country will influence country's innovation index, because GII is a country innovativeness measurement considering the average value of innovation index and innovation efficiency ratio which is obtained from innovation input sub-index and innovation output sub-index.
In 2016, GII studied 128 countries' profile representing 92.8% of the global population and 97.9% of gross domestic product (GDP). At that time, Indonesia ranked 88th in GII. Comparing within classified region by United Nation, Indonesia ranks 13 out of 14 countries inSEAO region (South East Asia, East Asia, and Oceania). Indonesia only succeeded in winning over one country, Cambodia, which is ranked 95th.
Indonesia's achievement can be analyzed by analyzing each assessment result of criteria in GII. The analysis is conducted to find out Indonesia's position compared to other countries and Indonesia's strengths and weaknesses on the overall aspects of the assessment in GII. Innovation achievement of a country influenced by the role of the company shows the need of attention and consideration to develop the company's innovation activities, in terms of input, process, and, output. The role of innovation in a company can be measured more extensively by assessing the innovativeness level of the company. It is also said by Lakiza and Deschamps (2018) that is lack of appropriate systems in place especially in the developing countries.
According to Dervitsiotis (2010), innovativeness level is defined as the overall measurement of innovation achievement called innovation excellence. The overall measurement is derived from a combination assessment of innovation capability and innovation results within a company. Research on the assessment of innovation in a company has been done by Dervitsiotis (2010). The research produced a framework for the assessment of an organization's innovation excellence that will be modified as a purpose of this study. Modification is intended to result in the assessment tools which are suitable with Indonesian companies' situation and condition. Modifications are conducted as following: validating sub-variables contained in every variable, determining importance weight between sub-variables, and formulating the modified assessment tools. This study uses Delphi method to validate research sub-variables, Analytical Hierarchy Process (AHP) method to determine the importance weight between sub-variables, and in-depth interview method to collect research data. The study aims to identify the variables system related to innovativeness of company, determine the importance and contribution level of variables and sub-variables to the innovativeness of company, and develop assessment tools for measuring innovativeness level of companies.

Literature study
The literature used in this study is literature review that helps solving existing problems, such as innovation framework for the assessment of an organization's innovation excellence, European Foundation for Quality Management (EFQM) excellence model, dan baldridge excellence framework.

Innovation
According to Leonard (1995) innovation can be defined as a continuous way to build and develop the organization through the introduction of new technologies, the introduction of new applications in the form of products or services, the new market development, and the introduction of new forms of organization as fusion of various aspects of innovation which then forming innovation area in certain time. In order to be innovative, the company strives to optimize the invention and the design of new value propositions in the form of new products, new processes, or new business ways. Thus innovativeness can be achieved based on visionary leadership, employee creativity, customer participation, suppliers participation, and other partners' participation (De Geus, 1997).
Transition to knowledge-based economy-the most important process that takes place in these decades-has as a defining element the amplification of creativity and innovation, in dimensions never seen before, which generates strategic knowledge, ultimately decisive in achieving competitiveness (Popa, 2011). (2014) identified elements that can lead to strategies aimed at the promotion of innovation and technological development, emphasizing strategies directed to the formation of researcher-driven innovation and technology, promotion of publications and creation of magazines related to science and technology; strategies that link the academic and business sectors, as well as the use of patents generated in order to boost competitiveness and development in Mexico.

Framework for the assessment of an organization's innovation excellence
Innovativeness can be defined as overall measurement of innovation achievement called innovation excellence (Dervitsiotis, 2010). The overall measurement can be obtained from the combined assessment of both innovation capability and innovation result. Innovation excellence requires a high level of innovation capability to create a sustained stream of successful innovations and a new stream of cash revenues connected with the significant tangible and intangible benefits that are offered to stakeholders, such as customers, employees, owners, suppliers, and community.
In the development and formulation of a framework to assess innovation excellence, specifications are as following: Innovation capability as the measure of effectiveness of the innovation system.
Innovation results as the benefits realized from innovation project for a company's key stakeholder.
Innovation excellence as the overall measurement of innovation achievement from the combined assessment of both capability and results.
Company's innovation capability is strongly related to system variables, including organization culture, leadership for innovation, internal and external resources, customer participation, employee participation, and supplier participation (Dervitsiotis, 2010). While total innovation efforts that refer to impact realized by stakeholders in the organization specified as customer impacts, employee impacts, organizational impacts, and total performance impacts. Innovation excellence can be determined by combining the evaluation scores for the variables system. Variables system is drawn in framework providing an overall measurement of innovation achievement toward the pursuit of innovation excellence. See Figure 1.

EFQM excellence model
EFQM excellence model enables a company to understand and know the cause-effect relation between organization's efforts and organization's achievement. This excellent model implements eight fundamental concepts, such as adding value for customers, creating a sustainable future, developing organizational capability, harnessing creativity and innovation, leading with vision, inspiration and integrity, managing with agility, succeeding through the talent of people, and sustaining outstanding results (EFQM, 2013).
Nine assessment criteria in Baldridge Excellence Framework are as following: (1) Leadership (2) Strategy This framework uses two kinds of matrix assessment, which are enabler matrix assessment and results matrix assessment. The difference between two matrices can be seen in the utilization of elements of assessment. Enabler matrix assessment considers three elements to include approach, deployment, and assessment & refinement. While results matrix assessment consists of two elements which are relevance & usability and performance. Every element of assessment has some criteria.
Furthermore, the criteria in assessment element will be assessed for every enablers or results. Matrix scoring for every enablers or results is conducted based on two aspects, those are scoring capability of demonstrating the criteria in the element of assessment and scoring overall measurement for the enabler or the results. Overall scoring can be conducted within scoring range between 0% and 100%. The assessment using EFQM's matrix assessment is performed for every subcriterion contained in EFQM Excellence Model. After completing assessment, overall measurement for every subcriterion can be processed to get the final score. The final score that an organization can obtain is by applying the EFQM Excellence Model to range from 0 to 600.

Baldridge excellence framework
Baldridge Excellence Framework is a framework that helps the organization to measure and evaluate the performances of management (BPEP, 2017). This framework promotes a system perspective which means managing all the components of organization as a unified whole to achieve ongoing success. Baldridge Excellence Framework consists of seven criteria divided into two categories, those are process category and results category. The seven criteria of Baldridge Excellence Framework are as following: (1) Leadership (2) Strategic planning (3) Customer focus (4) Measurement, analysis, and knowledge management (5) Workforce focus (6) Operation focus (7) Results The scoring for every criterion is conducted based on two dimensions of evaluation, those are process dimension and results dimension. The scoring for process category considers four evaluation factors, such as approach, deployment, learning, and integration (ADLI). While scoring for results category considers four evaluation factors, such as levels, trends, comparisons, and integration (LeTCI). The final score that an organization can obtain in applying the Baldridge Excellence Framework ranges from 0 to 1,000. The final scoring obtained by an organization will show the performance level of the organization. The categories of organizations based on scoring results from using Baldridge Excellence Framework are world leader, benchmark leader, industry leader, emerging industry leader, good performance, early improvement, early result, and early development.

Delphi method
Delphi Method is a modification of brainwriting and survey techniques (Powell, 2003). This method was developed in the early 1950 to gain expert opinion. In this method, panels are used in the communication movement through several written questionnaires. The purpose of this method is to obtain the most reliable consensus from an experts' group.
According to Linstone and Turrof (2002), Delphi's procedures have features such as ignoring names, controlled iterations, controlled feedback, and statistical group responds. Usually, Delphi questionnaires conduct in 3-5 rounds, depending on the degree of conformity and the amount of additional information applicable. The first-round of Delphi questionnaire asks the individual to respond to the questions generally. Each Delphi questionnaire in the second and subsequent rounds is built on the response of the predecessor questionnaire.

AHP method
AHP is a method developed by a mathematician named Thomas L. Saaty in 1971Saaty in -1975 when he was in Wharton Business School. Basically, AHP is a measurement method using pairwise comparisons which depend on expert judgments to produce a priority scale (Saaty, 2008). This method is used to make effective decisions on complex issues. AHP simplifies and speeds up the decisionmaking process by setting various priorities. Prioritized process is done with the experts' consideration to the data obtained in the field. This process depends on the imagination, experience, and knowledge to construct a problem's hierarchy and rely on the logic and experience of giving consideration (Shega, Rahmawati, & Yasin, 2010).

Respondents' profile
The requirements to participants as the experts are at least to have a bachelor degree, at least to have been working in middle-level management, and to have been experiencing more than 8 years of total working period. There are two kinds of questionnaires in this study. First questionnaire is a Delphi questionnaire for gathering information about validation of sub-variables. While the second questionnaire is pairwise comparisons questionnaire for gathering information on the determination of the priority scale between sub-variables. This study is conducted in three steps including validating subvariable, determining importance weight of sub-variables, and formulating assessment tools.

Validating sub-variables results
Variable system that will be used in this study is determined by literature study from some references, such as the framework of Dervitsiotis (2010) and EFQM Excellence Model. Variable system of organization's innovation excellence consists of two dimensions, which are the dimension of innovation capability and dimension of innovation results. Dimensions of innovation capability are composed by eight variables, which are innovation leadership, organization culture, innovation strategy, innovation resources & partnership, customer feedback process, employee participation process, supplier participation process, and innovation process. While dimension of innovation result are composed by six variables, which are customer impact, employee impact, organizational impact, economic result, market performance result, and environmental footprint. Each variable is composed by a number of sub-variables. The determination of sub-variables is conducted by the literature study. The result of literature study shows that innovativeness level of companies has 43 sub-variables as assessment criteria which are divided as follows, 26 sub-variables are included into dimension of innovation capability and 17 sub-variables are included into dimension of innovation results.
After the determination of variable system, the result has to be validated. The validation is conducted by the Delphi method. Validation is conducted only for sub-variables. Delphi method is applied to get experts' consensus about valid sub-variable that can be used in assessment tools. The consensus is obtained from data processing of Delphi questionnaire which previously been filled by experts. Data collection and processing using the Delphi method is done in two rounds. The final result of Delphi Method is innovativeness level of companies has 44 sub variables as assessment criteria which are divided as follows, 27 sub-variables included into dimension of innovation capabilities and 17 sub-variables included into dimension of innovation results. See Appendix 1.

Determining importance weight of sub-variables result
Determination of sub-variables' importance weight is conducted by AHP method using Expert Choice software. Data processing by AHP method is conducted based on paired comparison questionnaire filled by experts. The questionnaires only compare the importance between subvariables. While data of importance comparison between dimension and between variables have been known based on adjustment of data input software to get weight dimensions and weight variables approaching Dervitsiotis research (2010). It is intended that this study still uses the reference importance weight of dimensions and variables from research of Dervitsiotis (2010).
The weight calculation between sub-variables can be conducted after consistency test. Consistency test is conducted to determine whether the questionnaire is consistent or not. It is intended that the question of importance comparison between sub-variables in the questionnaire is not filled randomly by the respondent. Based on the result of consistency test, it is known that the result of data processing for each respondent and the result of data processing as a whole fulfilling the consistency requirement by having inconsistency value ≤0.1. The final result of AHP method can be seen in Appendix 1.

Formulating assessment tools result
Formulation of assessment tools for measuring innovativeness level of companies is conducted by modification framework of Dervitsiotis (2010) and using references from the assessment of excellences such as EFQM (2012) and BPEP (2017). Modification is applied in formulation sub-variables for each variable, determination importance weight for each sub-variable, formulation sub-variables scoring, and determination the classified rank for the value of innovation excellence.
The two steps of modification have been conducted before. The next step is the formulation of subvariables' scoring system adopting the scoring system of EFQM (2012). Scoring system is divided into two categories, which are scoring system for dimension of innovation capability and scoring system for dimension of innovation results. The differentiation between both scoring systems happens because the difference of evaluation factors that cannot be similar. Scoring matrix for dimension of innovation capability consists of three elements including approach, deployment, and assessment & refinement. While scoring matrix for dimension of innovation results consists of two elements including relevance & usability and performance. Each scoring element has some composed criteria. One scoring matrix is used to assess one sub-variable. The total number of scoring matrix that will be used in assessment tools of innovativeness level of companies is the same as the total number of sub-variables in assessment tools. Hence, there will be 44 scoring matrices specified 27 scoring matrices for dimension of innovation capability and 17 scoring matrices for dimension of innovation results.
Matrix scoring for both dimensions is conducted based on two aspects, those are scoring capability of demonstrating the criteria in element of assessment and scoring overall measurement for the sub-variables. Demonstration capability is differentiated to five categories, which are unable to demonstrate, limited ability to demonstrate, able to demonstrate, fully able to demonstrate, and recognized as Global Role Model. Scoring overall measurement for the subvariable can be conducted within scoring range of 0% to 100%. Score 0%, 5%, and 10% can conclude that the company as overall is unable to demonstrate the criteria of assessment. Score 15%, 20%, 25%, and 30% can conclude that the company as overall has limited ability to demonstrate the criteria of assessment. Score 40%, 45%, 50%, 55%, and 60% can conclude that the company as overall is able to demonstrate the criteria of assessment. Score 65%, 70%, 75%, 80%, and 85% can conclude that the company as overall is fully able to demonstrate the criteria of assessment. Score 90%, 95%, and 100% can conclude that the company as overall is recognized as Global Role Model to demonstrate the criteria of assessment.
After the determination of scoring system, tabulation of scoring results is determined based on research by Dervitsiotis (2010). Processing of scoring result is started with calculating each subvariable score. Sub-variable score is achieved by multiplying the sub-variable scoring with sub-variable weight and maximum value of a certain variable. Then variable score is achieved by summing up the composed sub-variables. Calculating variable score is conducted eight times as much as number of variables included in the dimension of innovation capability and six times as much as number of variables included in dimension of innovation results. The next process is calculating the dimension score. Calculating dimension score is conducted for both dimensions so that could be two-dimension score achieved. Score of innovation capability is achieved by summing up the score of composed variables. And also score of innovation results is achieved by adding up the score of composed variables. After the dimensions score is known, the next step is calculating the innovation excellence score by summing up the score of innovation capability and the score of innovation results. Maximum score for innovation excellence is 1,000. The last step is determining the innovativeness level of company based on classification adopted from Baldrige Performance Excellence Program [BPEP] (2017). The classification shows that there are eight levels to differentiate the achievement of innovative performance.
In order to simplify the assessment process, five steps are applied as following: (1) Scoring each sub-variable (2) Calculating sub-variables score and variables score (3) Calculating dimensions score (4) Calculating innovation excellence score (5) Determining the innovativeness level of company based on classification The results of assessment tools' weights for measuring innovativeness level of companies can be seen in Appendix 1.
Appendix 2 is to show maximum score each variable can be achieved.

Conclusion and future research
Formulation of assessment tools for measuring innovativeness level of companies is conducted by modifying framework to the assessment of an organization's innovation excellence proposed by Dervitsiotis (2010) which is also synthesized with other references such as EFQM (2013) and BPEP (2017). Modification is intended to formulate the assessment tools which suitable with Indonesian companies' situation and condition. Variable system related to companies' innovativeness is divided into two dimensions, those are dimension of innovation capability and dimension of innovation results. Innovation capability is composed by eight variables. While innovation results are composed by six variables. Innovativeness level of companies has 44 sub-variables as assessment criteria which are divided as 27 sub-variables included into innovation capabilities and 17 sub-variables included into innovation results. The importance and contribution level for each variables of assessment tools used in this study is derived from the research by Dervitsiotis (2010). While the importance and contribution level for each sub-variables of assessment tools is determined by AHP method.
One limitation of this research is that the system variables are seen as a hierarchical structure rather than a network structure. Since the system variables viewed as assessment criteria are related and connected to each other, for future research it is recommended to apply analytic network process (ANP) method to determine the importance and contribution level between variables and also between sub-variables. Scale of each sub-variable is not determined yet as further study is underway.
Tohidi and Jabbari (2012) made conclusions in their paper that it is the need to view the process of innovation as changes in a complete system of not only hardware but also marker environment, production facilities & knowledge, & the social contexts of the innovating organization.
As it is stated by Dewangan and Godse (2014) future research can be directed more on the distinction between the measurement of inventions and exploitation that can be used as taxonomy for innovation key performance indicators. Dziallas and Blind (2018) suggested to accommodate soft indicators which refer to nontechnological and technological innovations as well as a broader spectrum of industries (including service industries) and a wider timeframe.