Research Article Research on a Comprehensive Index System for Regional Patent Evaluation

the With the rapidly increasing number of patent applications in China, each applicant tends to accumulate the quantity of patent, but not the improvement of patent quality. Therefore, the State Intellectual Property Office (SIPO) has recently issued a package of notification, which aims to reverse the status quo. However, existing methods of regional patent evaluation still emphasize on quantity rather than quality indices, which leads to biased evaluation results. In this paper, we establish a comprehensive index system for regional patent evaluation. The indicators of the system consist of four dimensions, including quantity, quality, structure, and coordination. This aims to provide a path of high-quality development for regional patent work. Based on the patent data of each city in Sichuan Province in 2020, this paper validated the effectiveness of the comprehensive evaluation system and put forward countermeasures and suggestions.


Introduction
Since the Chinese government enacted the "Outline of the National Intellectual Property Strategy" in 2008, the number of patent applications has made a great increase [1]. Until 2020, the annual number of patent applications has consecutively ranked rst in the world for 10 years. ough the number of patents increases, the patent quality worsens [2,3]. According to the statistics, the majority of patent applications consists of utility model patents and design patents, which are in low quality compared with invention patents. e antecedents of the situation comprise two folds: rst, the government departments attach great importance to the patent numbers [4], so they provide great number of subsidies to encourage patent applications [5,6], while the quality of patents is neglected; second, in order to easily gain patent subsides from the government, enterprises tend to apply for utility model patents and design patents, which are easily granted but useless for innovation [7]. e existence of low-quality patents results in two negative e ects [8,9]. On the one hand, enterprises cannot yield economic pro ts from patent activities. On the other hand, the government cannot make a great progress in innovation though it has input lots of resources in patent activities.
Nowadays, in the context of transition economy, the aim of economic development has transformed into high-quality growth. Hence, the government has paid more attention on the high-quality patents that can improve innovation ability. Correspondingly, the State Intellectual Property O ce (SIPO) has issued policy documents, such as "Implementation Measures for Improving Patent Quality Projects" in 2017. Meanwhile, the SIPO urges regional governments that they should properly guide and support patent applicants to focus more on the quality of patents and establish a comprehensive index system to better accomplish the regional patent work. However, the existing index system used by regional governments has drawbacks in three aspects: rst, the evaluation indicators are incomplete, hence cannot fully re ect the regional patent work [10,11]; second, the index system is more quantity-oriented [12], thus leads to plenty of low-quality patents; third, the calculation of index system is simple so that cannot distinguish the importance of different types of patents.
Based on the requirements of high-quality development of regional patent work, this research proposed a comprehensive index system for regional patent evaluation. First, our index system includes more evaluation indicators, which consist of four first-level indicators and 15 second-level indicators, so it makes the index system more complete. Second, we develop our index system from four dimensions, which comprise quantity, structure, quality, and coordination indices, which correspondingly make the index system more comprehensive. Finally, we use analytic hierarchy process (AHP) to build the index system, thus making the calculation process more scientific. Our research has the following contributions: on the one hand, this research has filled the gap of lacking research on the index system for regional patent evaluation. On the other hand, the newly developed index system can effectively and comprehensively evaluate the regional patent work and guide patent applicants to attach more importance on the patent quality rather than quantity. e paper proceeds as follows: Section 2 introduces the extant patent evaluation research studies; Section 3 describes the principles and structure of our index system; Section 4 presents the application of the index system based on the regional-level patent data of Sichuan Province in China; Section 5 discusses implications and limitations of this evaluation system and concludes the paper.

Review of Patent Evaluation Research
ere are a large number of previous studies regarding patent evaluation indicators. As innovation becomes more and more important in firm competence, the research of patent evaluation started to bloom. Scholars back in 1990s tended to establish the evaluation system using single indicators. However, these single-indicator-based methodologies are soon to be replaced by more comprehensive evaluation systems due to better data availability. Next, we will introduce some key indicators that are not only widely discussed in the extant literature but also the pillars of our evaluation system.
First of all, the number of patent holding by firms can be thought of as an indicator for the successful outcomes from research and development process and reflects in the market value of the firm [13]. However, some scholars argued that solely using patent statistics can be a mirage appearing to provide a great number of objective and reliable proxies for innovation [14].
Second, after a patent has been granted, the length of duration for the maintenance of patent rights can be thought of as an indicator of the business value of a patent. Lanjouw and Schankerman [15] showed that the period of patent maintenance is a powerful indicator of patent value. However, the duration of patent right maintenance can only be obtained after a long period. erefore, other evaluation systems generally exclude this indicator due to timeliness.
ird, some scholars evaluate the value of a patent through patent length and breadth. In other words, the more technology classes a patent can cover, the more valuable it will be. Kuhn and ompson [16] proposed an easy-to-use measure of patent scope based on patent law and practices of patent attorneys. In the meantime, because the classification can be sometimes subjective, some scholars did not consider patent scope as a valuable indicator.
Fourth, the number of citations of a patent is a widely recognized indicator of patent values. Harhoff et al. [17] believe that the number of patent citation can be used to assess the economic quality of a patent. However, for this indicator, the cumulative nature of forward citations needs a lengthy time period to be useful, and if there exists highly citing group for certain prior patents, there could be an upward bias.
As mentioned above, the existing research on the patent evaluation using single indicator has been relatively mature. Afterwards, scholars pay more attention on comprehensive patent evaluation systems. In 1994, Narin as the founder of the CHI Research proposed the concept of patent measurement and then proposed the CHI patent evaluation system [18].
e system mainly consists of three basic indexes and 6 advanced indexes. Lanjouw and Schankerman [15] then established a comprehensive index for patent quality.
is quality index is calculated using four indices, namely, the number of patent citations, claims, patent references, and patent family members. Also, they point out that adopting several indicators to evaluate patent quality will significantly reduce the measurement variance. Moreover, Mariani and Romanelli [19] learn from this model, employing the number of patent citations, claims, and patent family members to set an evaluation system for patent quality. e analysis index system of patent value launched by the China Technology Exchange in December 2012 comprehensively evaluates patent value from the legal status, technical intensity, and economic value.
e index system includes three firstlevel indicators and 18 supporting indicators, and the Patent Value Degree (PVD) will be formed after the scores of experts are weighted.

Establishment of Principles and
Comprehensive Index System for Regional Patent Evaluation

Principles of Evaluation.
In order to establish a comprehensive index system for regional patent evaluation, we believe that quantity is the basis, quality is the key, and coordinated development is necessary. e following will explain each principle specifically.

Quality-Oriented.
e focus of the current patent management is taking measures to improve the patent quality under the new situation. e idea that "quantity is the key" should be reversed, and the quality should be made to be an important standard in regional patent evaluation. Emphasize the quality in the index system and highlight the central position of the patent quality in patent development for guiding innovators to pay more attention to the patent quality and advanced technology.

Quantity-Considered.
e number of patents is the basis of patent management and the most evident reflection of the achievement of regional patents. ere may be a few low-value patents and some high-value patents and core patents among regional patents. erefore, a comprehensive evaluation method to distinguish those high-value patents is also necessary. e total number of patents can reflect the overall innovation ability of a region and the government's attitudes towards patent management. e number of highvalue patents can reflect whether the development of the quantity coordinates with the quality and whether there is a deviation in the quantity. Taking quantity into account includes not only the assessment of the total number but also the regional layout of quantity, highlighting the number of high-value patents and core patents in the region.

Systematic.
e comprehensive strength of regional patents is a complex system composed of multiple levels and elements. In addition to the quantity and quality of patents in a region, the regional patent layout, the coordinative development of economy, society, and technology, and other factors should also be taken into consideration. erefore, when we deliberate evaluation indicators, it is necessary to consider as many relevant key factors as possible. However, the evaluation index system should be logical and systematic rather than a simple combination of each index. erefore, our study evaluates regional patents from quantity, structure, quality, and coordination, forming a clear analysis framework. ese indicators are independent but interrelated, aiming to cover the most concerned key factors when evaluating patents.

Feasibility.
In theory, as many evaluation indicators as possible should be taken into consideration for establishing a comprehensive regional patent evaluation system, so as to assess the strength of regional patents comprehensively. However, in practice, the feasibility of indicators and the availability of data cannot be ignored, as well as the interference of human factors requires to be minimized. Besides, it would be better to employ objective quantitative indicators, which are available in the existing database or by simple searching procedures.

Establishment of Comprehensive Index System.
Following the principles mentioned above, collecting the quantifiable indicators of patent evaluation at home and abroad, referring to the establishment of evaluation system for scientific and technological competitiveness, regional innovation capabilities, and scientific research performance of universities, and discussing opinions with experts, this study will divide the comprehensive index system into three levels. e first-level index leads to a composite index. e second-level index consists of four dimensions, namely, quantity, structure, quality, and coordination. e thirdlevel supporting index includes 15 independent but interrelated evaluation indicators. Consequently, the comprehensive index system for regional patent is formed by 4 evaluation dimensions (the second-level index) and 15 evaluation indicators (the third-level index) through selecting indicators to estimate the patent composite index (the first-level index). e details of comprehensive index system for regional patents are shown in Table 1. Table 1, the patent composite index is a criterion for the overall evaluation of regional patents. It includes both quality and quantity evaluation. It is formed on the basis of evaluating various indexes of regional patents. erefore, it can objectively reflect whether the regional patent management meets the requirements of high-quality development and the needs of the country's innovation-driven development.

Quantity Index.
Quantity index is an important second-level index; this is a quantitative characterization of patent management, which reflects the government's attitudes towards patent management and the innovators' attitudes towards patent application and maintenance. As for indicator selection, the total number of patent applications per year is used to reflect the overall volume of regional patents; the number of Patent Cooperation Treaty (PCT) applications and invention patents granted per year is regarded as the proxy for the number of high-value patents; the valid invention patent index can be used to evaluate the maintenance of regional patents.

Structural Index.
We also include the structure as a second-level index. e patent structural index is used to analyze the proportions of different types of patents in the region and evaluate whether there are structural problems in regional patent management. Invention patents, valid invention patents, and PCT patents can to some extent represent high-quality patents.
ey are selected to be the evaluation object of the structure index, reflecting whether the region is focused on the high-quality development of patent and whether there are a large number of low-quality patents in the patent application.

Quality
Index. Patent quality index is a comprehensive evaluation of regional patent quality, which evaluates patent quality from technology, law, and economy. In the selection of indicators, the patent maintenance rate is a more evident indicator to reflect the value of regional patents and the patent maintenance time is used as the index for evaluation. Moreover, the number of citations and claims are also relatively recognized as evaluation indicators of technological dimensions. Also, patent licensing, pledge, and transfer are the reflection of the activeness of patent economic activities.

Coordination Index.
e last second-level index is coordination. Regional patent management must be coordinated with the development of society, economy, and technology so as to make more contribution to regional economic development.
e function of the patent coordination index is to reflect the coordinative degree of the development between patents and society, economy, and technology. e number of valid invention patents per 10,000 people, invention patents granted per ten billion of GDP, and invention patents per 10,000 researchers can comprehensively measure the quality of regional scientific research and the market application.

Calculation Method.
In this study, Analytic Hierarchy Process (AHP) was employed to measure the weights of indicators at all levels, and the weight of each indicator in the evaluation index system is determined by its importance. e importance of each indicator is compared in pairs referring to the scores given by the experts and then design the judgment matrix. In this study, we select 10 experts that are from different areas but all are related to patent work. ose experts consist of three scholars from universities whose research interests are patent and innovation, four R&D engineers with master degrees from state-owned manufacturing firms, and three R&D engineers with master degrees from privately owned manufacturing firms. After calculating the weight vector, the accuracy and reliability are tested, checking whether they are logical. e application and analysis steps of the calculating method are as follows.

Designing the Judgment Matrix.
e indicators of the third-level index are compared in pairs referring to the scores given by the experts. e values are assigned by 1-9, and the pairwise comparison judgment matrix is formed as shown in Table 2.
e judgment matrix can be simplified as follows:

Weight
(1) Calculate M i , the product of each row in the judgment matrix: (2) Calculate the fourth root of M i :

Test of the Accuracy and Reliability.
First, the consistency of the matrix is tested through the following steps. e maximum eigenvalue λ max � 4 j�1 (AV) j /4V j � 4.1545 and the consistency index CI � 0.0515, so the average consistency index is CR � 0.04153 < 0.1. It demonstrates that the second-level index judgment matrix A is consistent, and each weight is logical. erefore, the weight of the second-level index is (0.2771, 0.1215, 0.4799, 0.1215).
Similarly, the judgment matrix of each level is designed and calculated. e weight and composite weight of index at each level are shown in Table 3.

Calculation of Composite Index.
Based on the comprehensive evaluation index system, by standardizing the indicators at three different levels, the patent composite index becomes a dimensionless value. e index value is multiplied by the index weight to calculate the quantity index, structural index, quality index, and coordination index. Similarly, composite index can also be calculated using the following method: (1) e dimension of the third-level index is removed with the standard deviation method to calculate the evaluation value. e formula is as follows: where X ij is the dimensional evaluation value of the third-level index, in which i denotes the second-level index and j denotes the third-level index. d ij is the dimensionless value of the third-level index, and its value ranges from 0 to 100. (2) e evaluation value of the second-level index is a weighted dimensionless value of the third-level index.
where W j is the weight of the third-level index and n i is the number of the third-level index under the second-level index i. (3) e composite index is weighted and integrated by the second-level index, and its calculation formula is as follows: where W i is the weight of the second-level index, n is the total number of the second-level index, and d i is the second-level index.

Calculation of the Patent Composite Index of Each City in Sichuan.
To verify the validation of our comprehensive regional patent evaluation index system, we used the patent data of each city in Sichuan Province in 2020. Our data comprise two parts, including patent data and economic data. e patent data are retrieved from China National Intellectual Property Administration (CNIPA, https://www. cnipa.gov.cn/) and IncoPat (https://www.incopat.com/). CNIPA contains the most authoritative patent statistics and is well recognized in previous study [3]. IncoPat is a thirdparty patent service provider for complement. As for economic data related to population, economy growth, technology, and other aspects in the coordination index, they are retrieved from the national statistical yearbooks (http:// www.stats.gov.cn/) and local government websites for public statistics of each city in Sichuan. In 2020, the number of provincial patent applications increases by 167676, reaching a year-on-year rise of 27.49%.
ere are 42676 invention patents, accounting for only 25%, while the rest are utility model patents and design patents.
ere are 108386 new patents granted, with a year-on-year growth of 32.08%, among which 14187 are invention patents. e number of provincial PCT patent applications increases by 530, and there are 8.41 invention patents owned per 10,000 people. By calculating the composite patent index of each city in 2020 (as shown in Table 4), the results can reveal underlying problems on patent quality behind the fast-developing figures.

Analysis of Evaluation Results
. By comparing the composite index, there is significantly uneven development between the overall strength of patents in these cities.  Coordination index  Quantity index  1  3  1/3  3  Structural index  1/3  1  1/3  1  Quality index  3  3  1  3  Coordination index  1/3  1  1/3  1 Chengdu has highest composite score, 64.33, while Ziyang has the lowest one, 11.20. Obviously, polarization is demonstrated. e average composite index of Sichuan Province is 21.79, with only 7 cities reaching the figure, so most cities need to improve their comprehensive strength of patents. Although the number of patent applications in Mianyang, Yibin, and Deyang soars in 2020, the deficiency of overall quality and the coordination with the rapid economic and social development of the city makes them lag slightly behind. By analyzing the second-level index, the development of each city is quite different. In terms of quantity index, Chengdu is prominent, with a huge number of patents and innovators, forming a "No. 1" in the figure. Because of the great gap between Chengdu and other cities, there may not be cities can surpass it in a short period. In terms of structural index, patent layout of Panzhihua is relatively scientific, focusing on the invention of high-value and highquality patents. However, the patent structure of Bazhong and Dazhou is not as appropriate for the overhigh proportion of low-tech patents. In terms of quality index, the overall score is relatively lower than the other three indicators, indicating that the quality is the most serious problem in the patent development in Sichuan. Measures should be taken to improve the overall quality, such as issuing policy and assessment instructions. In terms of coordination index, the development of patents and cities in Chengdu, Panzhihua, and Mianyang are relatively coordinated. Also, the degree of coordination is much higher than that of other cities, becoming a height of coordinated development.
e low coordination index of Guangyuan, Guang'an, and Dazhou shows that the local government  ere is no patent in Ganzi in 2020, so the quantity index of Ganzi is 0.
does not pay enough attention to the development of intellectual property rights, so that the strategic position of patent work is not significant, the development of patents is obviously lagging behind the development of other cities, the relationship between economic growth and innovation activities is not close, and that the achievements of patents are inadequate. Based on the above analysis, it turns out that the main problem of patent work in the cities of Sichuan is "quantity first or quality first." Taking Chengdu as an example, the comprehensive patent strength ranks first in these cities, and the quantitative index (100) takes the lead, but the quality index (30.90) lags behind. is reflects that its overall patent strength is "large but not strong" and the development of quantity and quality is imbalance. In the economically developed area of Sichuan Province, the rank of the quantity index is obviously higher than the quality index, reflecting that the local government is inclined to pursue the rapid increase in the number of patents rather than the quality in the patent work.

Conclusion
is paper establishes a comprehensive index system for regional patent evaluation. In order to evaluate regional patent comprehensively, we set 4 evaluation dimensions (the second-level index) and 15 evaluation indicators (the thirdlevel index) in this system. e established principles of the index system include quality-oriented, quantity-considered, systematic, and feasibility. In China, the quantity of patents has increased rapidly while the quality has not kept pace.
In our study, we emphasize the quality of patent in the evaluation system, which aims at reversing the present idea of "quantity-oriented" in patent work and regarding quality improvement as the core of patent development. We try to maintain objective and rigorous during the process of comprehensive regional patent evaluation, and most of the data are retrieved from public database. erefore, the results should be relatively convincing. We use the patent data of Sichuan Province in China and calculate the scores for each index and each indicator. Accordingly, the composite index of each area of Sichuan Province is measured. By comparing the composite index, we can evaluate the comprehensive quality of regional patents. Moreover, we can provide advice to the development of regional patent through the index system. is comprehensive regional patent evaluation index system is currently built on the data that can be mostly available by public. Our study provides regional and local government an easy-to-use and low-cost method to evaluate their patent work. However, this index system also has its limitations. For instance, we cannot estimate the commercial value of a patent, which is an important quality indicator. erefore, as the technology grows, we will be working on developing more scientific indicators using new technologies, such as big data, and machine learning to design and build a more inclusive and comprehensive evaluation system, which will better serve the national intellectual property strategy.

Data Availability
is paper evaluates the patent data of each city in Sichuan Province in 2020. e patent data in the study mainly come from the SIPO and IncoPat. Also, the statistics related to population, economy, technology, and other aspects in the coordination index come from the statistical yearbooks and public data of each city in Sichuan.

Conflicts of Interest
e authors declare that they have no conflicts of interest.