Integrating science and technology metrics into a competitive technology intelligence methodology

For years, the appropriate interpretation and application of metrics have enabled scientists to assess science and technology dynamics. Consequently, diverse disciplines have emerged, such as bibliometrics, scientometrics and patentometrics, offering important theoretical and methodological contributions. However, the current accelerated technological advances require researchers to implement a superior approach to detect continuous changes in the external environment identifying opportunities and vulnerabilities to strengthen the decision-making process regarding R&D and innovation. In this context, competitive technology intelligence (CTI) offers a strategic approach based on a continuous cycle where information is transformed into an actionable result. This research provides a broader scope to science and technology metrics, incorporating them into a CTI global methodology of eight steps. Metrics add value throughout the entire CTI process, from project planning to decision-making stages, having the most significant role in the information analysis stage, mainly to process information from sources such as scientific documents, patents, and social networks. Particularly, this approach considers recent studies in CTI in which quantitative tools such as patentometrics and scientometrics were successfully used. This proposal can be applied to predict upcoming technologies, movements of competitors, disrupting activities, market changes, and future trends. Accordingly, this research adds value to the assessment of science and technology dynamics, aiming to improve the decision-making process of R&D and innovation.


INTRODUCTION
In a global world, technological advances embody new opportunities to be assessed by organizations. Good decision-making requires correctly identifying the technologies to implement, the products to develop and the standards to use. The new forms of collecting, storing, transforming, and processing data have made the information more accessible than ever. Information about the external environment enables leaders to manage their businesses more effectively; however, information is not enough.
An adequate assessment of science and technology is fundamental to impact present and future R&D and innovation decisions; therefore, building metrics is essential to obtain accurate results. To facilitate science and technology understanding, diverse disciplines based on metrics analysis have emerged, including scientometrics, patentometrics, and altmetrics. They offer fundamental theoretical and methodological contributions involving the use of traditional and non-traditional metrics.
Scientometrics is a discipline based on mathematical methods to quantify scientific research literature to reveal the process of scientific development (Qiu et al. 2017). It enables researchers to identify the actors and processes involved in scientific activities, such as authors, research groups, institutions, countries, and their scientific production, to determine the structure, relationships, and research dynamics (Michán L. 2013). Scientometrics deals with scientific information analysis mainly from scientific documents; on the other hand, patentometrics focuses on the analysis of patents. Patents protect inventions developed by companies, institutions, or individuals, and can be interpreted as indicators of invention, and it is possible to create scientific and technological scenarios of countries, industries, and research institutes by analyzing them.
Unlike scientific literature, patents have a legal framework that supports them, and the information they contain has a uniform structure, allowing the easy extraction of the information desired. Economic indicators have also been associated with patents, addressing connections between technology and trade. While traditional metrics focus on the data mining of scientific and technological outputs (e.g., scientific papers, patents), nontraditional metrics-also known as alternative metrics or altmetrics (Priem et al. 2012)-are oriented to measure scholarly activities on social networks, blogs, newspaper articles, and web sites, among others (European Union 2017). The consolidation between traditional and non-traditional metrics offers a complementary view to evaluate the dynamism of science and technology, which requires more than one single metric approach (Staudt et al. 2018).
Although metrics analyze different aspects of science and technology, they are not enough to continuously monitor the external environment to support strategic R&D and innovation decision-making. By systematically analyzing the external environment, organizations can increase their advantages (Luu, 2015), identify movements of competitors (Rothberg and Erickson 2017), and detect opportunities for growth (Zeid 2014), which represents a crucial factor to survive under the current and future industry global competition (Shaitura et al. 2018). In this context, CTI adds fundamental value since it involves a continuous process based on collecting and analyzing external information transformed into a strategic result (Dou et al. 2019) to anticipate changes in the market and to identify relevant opportunities, supporting the decision-making process for innovation (Du Toit 2015, Rodriguez-Salvador andLopez-Martinez 2000). Based on studies that apply CTI for science and technology assessment, a global methodology that incorporates metrics into a CTI process is proposed. First, a description of the general context of CTI is presented, then recent CTI studies are identified, and finally, a CTI methodology is proposed revealing the incorporation of metrics.

COMPETITIVE TECHNOLOGY INTELLIGENCE
R&D fosters knowledge aiming to provide answers to questions from different fields. For this activity to be enhanced, it is necessary to transform information into intelligence and provide conditions to facilitate a continuous flow of knowledge (Amidon Rogers, 1996). Therefore, it is crucial to analyze external data in a timely and proper way to develop business insights and become more competitive (Rodriguez-Salvador 2006, Zeid 2014. Competitive intelligence emerges as an approach to support strategic decision-making. While Fitzpatrick and Burke (2003) define competitive intelligence as a process to collect and analyze external information to increase the advantages and position of an organization, Dou et al. (2019) define it as the ability of an organization to understand its environment effectively and drive strategies accordingly. This ability is sustained by the process of collection, analysis, and dissemination of actionable knowledge.
When competitive intelligence is applied to science and technology research, the term CTI arises. CTI pursues timely awareness of scientific and technological events to stay ahead, identifying collaboration prospects, technology knowledge landscapes, and in general, valuable findings to improve R&D and innovation processes (Rodriguez-Salvador et al. 2002). Achieving this involves a methodology which in general starts with a planning stage aligned to the main objectives of the study, continuing with the identification of information sources, aiming to identify the best alternatives to collect the most relevant information on science and technology from different sources, such as experts in the field, documents, patents, and social networks. The next step consists of information collection, where the main goal is not to get the most significant amount of information but to get the most meaningful information possible. This information is processed and prepared for further analysis, where different techniques can be applied (e.g., scientometrics, patentometrics, road-mapping). Dissemination and decision-making comprised the last stages.

CTI METHODOLOGY
To understand the impacts of metrics on scientific and technological research, an indepth analysis of studies that examine metrics and apply CTI for science and technology assessment was conducted. Wilsdon et al. (2015) explored the effects of the growing use of metrics to evaluate research, proposing their responsible application and establishing that an appropriate research assessment should include elements such as robustness, transparency, reflexivity, humility, and diversity. Staudt et al. (2018) proposed a set of text-and citation-based metrics to identify high-impact and transformative works. These metrics are categorized into seven types: radical-generative, radical-destructive, risky, multidisciplinary, wide impact, growing impact, and impact (overall).
Based on an analysis of the accuracy of 39 metrics, Bollen et al. (2009) showed that a multi-dimensional view is required to measure the impact of scientific research effectively. Finally, Cronin and Sugimoto (2014) emphasized that the web leads to new tools to assess scholar productivity, revealing behaviors and impact that were previously invisible, such as number of mentions, acknowledgments, endorsements, number of downloads, recommendations, blog posts, tweets, and a variety of other metrics that can be utilized. Despite the profuse research on metrics, few studies have explored metrics under a strategic view. Recent investigations incorporate scientometrics, patentometrics, and other quantitative tools into a CTI approach using data mining to make science and technology assessment more comprehensible, but they mainly address the analysis of secondary information, such as scientific documents and patents. As an example, Table 1 shows some recent efforts ranging from the analysis of specific fields to those focused on analytic tools. In the first group are new 3D bioprinting applications: optical tissue phantoms, emerging dentistry applications of 3D bioprinting, additive manufacturing for hand orthoses, technology forecasting on dyesensitized solar cells (DSSCs), competitiondriven figures of merit (automotive industry), 3D bioprinting in Latin America, keyword network mapping analysis on 3D bioprinting research, knowledge landscape of 3D bioprinting and forecasting on big data research. In these studies, scientific and technological trends were identified by analyzing documents and patents, including data mining of some of their elements, such as titles, keywords, authors, and affiliations. This revealed the dynamics of intellectual outputs in terms of metrics such as number, evolution, and impact of publications and/or patents by authors, journals, institutions, countries, and areas.
On the other hand, Table 1 also exhibits tools to facilitate analysis tasks, particularly data analytics for technology and engineering management and technology road-mapping for competitive technical intelligence where metrics combine qualitative and quantitative data and external actors are participating.
Approaches from Porter (2019) were considered to develop a new CTI methodology where, unlike current studies, 1) primary and secondary information are considered, 2) quantitative and qualitative metrics are applied and 3) experts participate. This CTI eight-step methodology comprises interdependent phases, receiving continuous feedback. 2. Identification of data sources. This represents the input for further analysis. In this stage, metrics can facilitate the selection of the best information sources. There are two basic types of data sources: primary and secondary. The former is based on the insight of experts, where metrics contribute to identifying experts for feedback purposes, determining their presence in a field in terms of indicators such as the number of paper citations, the number of patents, the ranking of publications, areas of specialization, and network collaborations through affiliation analysis. Commonly, secondary sources include scientific and technical documents, as shown in the studies in Table 1; however, this approach also incorporates strategic information such as industry and market reports. Additionally, social networks, which are rapidly evolving, are considered. In this case, metrics can help identify the quality of sources in terms of features such as their completeness, impact, and prestige.
3. Search strategy design. Establishing a plan and a strategy to retrieve information is essential. This activity should be aligned to the study focus as well as to the characteristics of the specific data sources previously identified. Tools like Delphi studies, focus-groups, and interviews may be considered, each one with specific metrics. For secondary information, particularly from the internet and databases, it is essential to identify the most relevant terms to feed searching queries. For this aim, an in-depth literature review should be executed, not only from scientific publications, technical reports, and patents but also from industry and market reports, among others. Moreover, different queries should be designed to maximize the efficiency of further data collection.
4. Data collection. This focuses on the previous primary and secondary sources identified. Database management systems would be required to access and manipulate large sets of information. This stage also includes normalization and preparation of the information to be processed and analyzed in the following step.
5. Information analysis. While traditional studies are typically focused on only on solving the questions "what?" and "how?" and on the analysis of scientific and technological documents, this stage aims to answer fundamental questions such as "what?" (to develop, incorporate, cancel, allocate), "how?" (human and material resources), "when?", "where?", "why?", "with whom?" and involves industry and market reports in combination with expert views. Scientific literature can be evaluated using metrics such as the number of publications, the growth rate of publications, impact factor of journals, the number of citations, author affiliation, collaboration networks, countries-and institutions-predominance, and areas of specialization by journal, author, institution or country. On the other hand, patents can be analyzed based on metrics such as patent production, patent categorization according to the International Patent Classification (IPC), IPC distribution of inventors, assignees and/or countries, the number of patent families (PFs), patent distribution of assignees and patent legal status (assigned, granted or inactive). To analyze social networks and websites alternative metrics can be incorporated, considering statistics comprising the number of mentions, number of downloads of the documents, and social network interactions. For scientific papers and patents, specialized software, data mining, and algorithms for text mining can be used to apply co-occurrence analysis, and keyword or term clustering to determine behaviors by the output of authors, countries, institutions, journals, and areas. Industry and market reports can be analyzed in terms of well-known metrics such as market share, competitiveness level, consumer behavior, and distribution rate.
6. Feedback from experts. Recommendations from experts constitute a great asset. Compared with other studies where expert participation is scarce or does not exist, this research suggests their participation across the entire CTI process. In this case, metrics can be established to get an expert evaluation of results obtained, particularly for the analysis stage.
Interviews and questionnaires throughout the entire CTI process are suggested. Experts can also participate in Delphi studies and focus groups.

Validation and delivery of final results.
It is crucial to zoom in to examine the accuracy of results obtained; validation should be developed from the initial to the final stages of the methodology. This stage represents the last verification to make final adjustments as needed. Results are then consolidated and prepared for their delivery through a report that can be communicated to the decision-maker and other stakeholders. This report should be aligned to the objectives previously established, adding value to the decision-making process. It is suggested that the content displays quantitative results; for example, figures that show the evolution of the industry, market, technology, product or process, emerging areas, readiness level, its performance, the predominance of areas, and position by authors, countries, companies, collaboration networks, market share, economic feasibility and distribution. Additionally, it is suggested to include qualitative results such as scenarios for industry, market, technology, product, or process. Furthermore, metrics can help to systematically monitor research activities at international, national, regional, and industry levels.
It is also fundamental that this report considers user preferences in terms of several aspects such as presentation style, content, and delivery time.
8. Decision making. This step represents the execution of results obtained, where decisions are taken by the people in charge of R&D and innovation. The result obtained previously is transformed into a specific action, and it is an input to decide what should be monitored constantly, which is also a differentiator of other approaches. It is imperative to stimulate debate and discussion, looking for competitive advantages.

DISCUSSION
The methodology proposed in this paper promotes the integration of metrics through a CTI eight-step process. Like those of Table 1, current studies are mainly focused on a statistical analysis of secondary information, principally through data mining of scientific and patent information applying techniques such as scientometrics and patentometrics. A gap still exists in the application of metrics under a strategic perspective, such as that of CTI, where primary and secondary information can be combined, and quantitative and qualitative metrics are integrated with the insight of experts. Such insight is required to support and validate metrics to ensure the production of reliable outcomes. Figure 1 illustrates how metrics could be integrated into the CTI methodology previously presented, where the principal contribution is for four steps. The second step is to identify the best information sources. The fifth step is where metrics acquire the most critical role of the CTI process, being possible to reveal strategic elements for R&D and innovation as early warnings, technology-and market-lifecycles, potential collaborators, technology impact and so on. The sixth step is where the insight of experts can help to refine results and broaden the scope of the study. And the seventh step involves validation and the elaboration of an executive report, integrating the most relevant insights.
Through this CTI global methodology, we aim to fill the gap regarding the current use of metrics. This can be an essential guideline for assessing science and technology evolution supporting strategic planning for R&D and innovation initiatives in technology-based organizations. Leaders should set new directions toward strategic changes to achieve competitive advantages (Verlander 2012). This effort may represent an important alternative to give a better overview of science and technology, giving the possibility of detecting opportunities and threats on time to gain competitive advantages through R&D and innovation.

CONCLUSIONS
In a globalized world where competition is becoming more complex, professionals should adapt, change, and develop competitive advantages by assessing the evolution of science and technology research. Metrics analysis represents a strong contribution for the research activities to be more Figure 1 Competitive technology intelligence (CTI) eight-step methodology incorporating metrics.
understandable. However, they do not offer a complete solution to anticipate and detect continuous changes in the external environment as CTI does. While current research focuses mainly on analyzing secondary information through quantitative metrics, this proposal goes further by considering 1) primary and secondary information, 2) quantitative and qualitative metrics, and 3) insight of experts into a CTI eight-step methodology, with an emphasis on the stages of identification of data sources, information analysis, feedback from experts, and validation and delivery of final results.
The proposed methodology can be applied to disclose the dynamics of an industry, market, and a scientific and technological field, predicting new technologies, movements of competitors, disrupting activities, market changes, and future trends. Moreover, this approach can enable the early detection of scientific and technological opportunities or threats by monitoring the competitive environment continuously and supporting the strategic planning of R&D and innovation.
Finally, it is relevant to state that the implementation of this global approach requires identifying specific metrics to incorporate according to the objectives to pursue, the industry, and the project context. Furthermore, novel metrics and automation processes to interpret information are continuously emerging. Consequently, it is crucial to keep abreast and include them in future research.