Measurement framework for assessing disruptive innovations

Assessing potential disruptiveness of innovations is an important but challenging task for incumbents. However, the extant literature focuses only on technological and marketplace aspects, and most of the documented methods tend to be case specific. In this study, we present a multidimensional measurement framework to assess the disruptive potential of product innovations. The framework is designed based on the concept that the nature of disruptive innovations is multidimensional. Three aspects are considered, i.e., technological features, marketplace dynamics and external environment. Ten indicators of the three categories are proposed and then connected based on the conceptual and literature analysis. Three innovations, namely, WeChat (successful), Modularised Mobile Phone (failed) and Virtual Reality/Augmented Reality (ongoing), are selected as case studies. A panel of industrial experts with PhD degree in engineering is surveyed. The survey results are calculated and analysed according to the framework and then compared against the developments of the innovations. We also check the robustness of this framework by surveying other groups of people, and the results are nearly identical to the previous findings. This study enables a systematic assessment of disruptive potential of innovations using the framework, providing insights for decisions in product launch and resource allocation.


Introduction
Determining whether an innovation (product or service) is disruptive or not is critical, because a disruptive innovation can radically unsettle the market status quo by overturning incumbents or creating new markets (Bower and Christensen, 1995). On one hand, the consequences of ignoring a potentially disruptive innovation can be catastrophic: losing market share and net profit or even bankruptcy (Bower and Christensen, 1995;Lucas and Goh, 2009). On the other hand, by embracing disruptive innovations, new firms can seize market share (Christensen, 1997a), and incumbents can maintain their positions (Christensen et al., 2015). Despite facing heavy criticisms such as being based on shaky foundation and lacking applicability (King and Baatartogtokh, 2015;Lepore, 2014), the disruptive innovation theory is continuously attracting attention from academics and business practitioners. One common belief is that potential marketplace disruption can be turned into a real business opportunity, provided that potential disruptiveness can be identified (Nagy et al., 2016).
Since the introduction of 'disruptive innovation' (Christensen, 1997b;Christensen et al., 2002), the theory is a research hotspot for the past two decades. 'Disruptive innovation' originally focused on technological innovations in terms of products or services (Christensen, 1997b), and it has then been extended to social innovation (van der Have and Rubalcaba, 2016). Christensen and Raynor (2003) listed a series of disruptive innovations: discount department stores; low-price, point-to-point airlines; cheap and mass-market products like power tools, copiers and motorcycles, and online merchants. Distinct innovations arise from different ways, exert varying competitive effects and require different responses; they should be treated as non-identical phenomena (Markides, 2006). Intensive efforts have been invested to identify the impacts of disruptive innovation on companies (Christensen, 2006;Christensen et al., 2002;Danneels, 2004), industries (Momeni and Rost, 2016;Rayna and Striukova, 2016;Ruan et al., 2014), markets (Adner and Zemsky, 2005;Markides, 2012;Vecchiato, 2017), administration (van den Broek and van Veenstra, 2018) and society (Christensen and Raynor, 2003;Feder, 2018). The same can be said as on identifying the settings of developing and adopting disruptive innovations (see Gao et al., 2017;Mahto et al., 2017;Pandit et al., 2018;Pérez and Ponce, 2015;Pulkki-Brännström and Stoneman, 2013;Roy, 2018;Roy and Cohen, 2015;Ruan et al., T 2014;Wan et al., 2015). Compared to the aforementioned extensive research on ex-post case analysis, evaluations on the disruptive potential of emerging innovations are limited (Hang et al., 2011;Klenner et al., 2013); the terms of 'disruptive innovation' and 'emerging technology' are scarcely co-occurred (Li et al., 2018). This gap can be attributed to the lack of research on characteristics of disruptive innovations (Danneels, 2004;Govindarajan and Kopalle, 2006a), probably resulting from the vagueness and/or misapplication of disruptive innovations (Christensen et al., 2015;Yu and Hang, 2010). Although Christensen et al. (2015) clarified the definition of disruptive innovation, lacking quantitative measurement to assess the disruptive potential of innovations remains a persistent problem (Nagy et al., 2016). This problem hinders various innovation-related decisions like capital investment, product development and policy formulation, and thereby it becomes a source of the attacks on disruptive innovation theory (King and Baatartogtokh, 2015;Lepore, 2014).
To address the above knowledge gap, we propose a measurement framework to assess the innovations' disruptive potential per se. The proposed framework allows indicators to be developed from the three categories: (a) technological features, (b) marketplace dynamics and (c) external environment. The potential connectivity of indicators is explored, and the weights of indicators are assigned according to their connectivity with others. A measurement space is hereby formed within the framework. Three innovations, namely, WeChat (successful), modularised mobile phone (failed) and Virtual Reality (VR)/ Augmented Reality (AR) (ongoing), are selected to illustrate the effectiveness of the proposed framework. We further verify the robustness of this framework by surveying other groups of people based on the same indicators, cases and procedure.
The contribution of our work is threefold. First, we provide a quantitative and more comprehensive measurement framework to assess the disruptive potential of innovations from the aspects of technological features, marketplace dynamics and external environment, whereas the documented assessments tend to focus only on technological and marketing (Gatignon et al., 2002;Govindarajan and Kopalle, 2006a). Second, we exploit the links between different features of innovations to facilitate the assessment of their disruptive potential, instead of simply adding up the scores of indicators (Hang et al., 2011). Third, we apply our measurement framework to three cases, i.e. We-Chat, Modularised Mobile Phone and VR/AR, and explain their success/ failure by comparing their survey scores against their actual developments. Rather than discussing the disruptiveness of innovations from a firm perspective (Govindarajan and Kopalle, 2006a), we explore the inherent characteristics of innovations to explain the likelihood to be successfully disruptive. The proposed framework facilitates the decisions on whether an innovation is disruptive and has the potential to succeed, through which plenty of managerial recommendations can be offered to stakeholders. For example, based on the assessment results, incumbents may be proactively prepared for all the sequential impacts, and investors will be aware of the strengths and weaknesses of emerging innovations as well as their chance of being successfully disruptive. This study offers implications for solving the 'innovator's dilemma' (Christensen, 1997b), as the case study (and the robustness check) shows a good agreement with the actual developments of the innovations.
The rest of the paper is organised as follows. The extant disruptive innovation literature that focuses on conceptual investigations and case studies is reviewed in Section 2. The proposed measurement framework is illustrated in Section 3, including the construct of the framework and the procedure of assessing disruptive innovations. Case study is presented in Section 4 to demonstrate the applicability and effectiveness of the proposed measurement framework, with WeChat (successful), modularised mobile phone (failed) and VR/AR (ongoing) selected as case innovations. Using the same cases and the same procedure, the robustness of the framework is checked and summarised in Section 5. Implications and concluding remarks are given in Section 6. Limitations and future research directions are discussed in Section 7.

Defining disruptive innovations
Defining disruptive innovations is of vital importance, given that such innovations modify development trajectory (Bower and Christensen, 1995), change technological paradigm (Momeni and Rost, 2016) and pose opportunities as well as challenges to business practitioners (Bower and Christensen, 1995;Christensen, 1997a;Lucas and Goh, 2009). In the early literature (Christensen, 1997b;Christensen and Bower, 1996), disruptive innovations are defined as the technologies that enable a new set of product features different from those associated with mainstream technologies and are initially inferior to the latter in certain attributes ('mainstream features') most valued by mainstream customers. During the early stage, the disruptiveness of an innovation is often so subtle that even top managers cannot perceive (Henderson, 2006), possibly attributing to insufficient training in technology management (Christensen and Raynor, 2003). Over time, the performance of disruptive technologies surpasses that of the dominant technologies and eventually 'invade' the mainstream markets. Disruptive innovation is not an event but a process (Christensen et al., 2015).
In general, two different types of disruptive innovations exist: (a) new market innovations that create a new demand for novel technologies and related products, and (b) low-end innovations provide technologies with similar characteristics to existing technologies but at a lower cost. Recent literature on disruptive innovation theory has tend to include both types of innovations, as Christensen et al. (2015) stated, 'disruptive innovations originate in low-end or new-market footholds'. Typical disruptive process innovations can also be labelled as low-end disruptive innovations, and their disruptive potential is usually fulfilled through products (Bower and Christensen, 1995). 3D printing is a typical example of disruptive process innovation, realising its disruption to business models through home-made products fabricated via 3D printers (Rayna and Striukova, 2016). The definitions of social innovations remain vague, ambiguous and diverse; nonetheless, the area is receiving increasing attention from academics (van der Have and Rubalcaba, 2016). In this work, social innovations have been excluded owing to the vagueness and uncertainty in their definitions.
Disruptive innovations cannot be defined by unidimensional characteristics. For example, as the literature (Christensen, 1997a;Christensen, 1997b) suggests, the disruption process of potentially disruptive innovations is likely to begin from low-end segments. However, Sood and Tellis (2011) examined 36 technologies and reached the opposite conclusion: the technologies that adopt a low attack are unlikely to disrupt incumbents. The definition of disruptive innovations must be multidimensional, and we summarise a few of the relevant works in Table 1. The definitions given by Govindarajan and Kopalle (2006a) and Hardman et al. (2013) follow the classic theory: the disruptive innovations initially seize new or low-end markets and then promote performance and capture the mainstream market (Christensen et al., 2015). The authors view the disruptive innovations as a process, whereas Thomond and Lettice (2002) and Nagy et al. (2016) focus on the static features. Despite the differences in descriptions, all the definitions agree that disruptive innovations are expected to have performance and market entry that are heterogeneous to those of incumbents, as Christensen et al. (2015) suggested. In this sense, Uber is not considered a disruptive innovation although it possesses explicit disruptive features (Cramer and Krueger, 2016) because its entrance points and service quality are essentially equal to the incumbent taxis (Christensen et al., 2015).
To conclude, disruptive innovations must possess distinctive characteristics in terms of technological features and marketplace dynamics. Considering that disruptive innovations are a process (Christensen et al., 2015;Christensen and Raynor, 2003), their different business models and/or ownerships can be affected by the changes in external environment. How mainstream customers value the traits of disruptive innovations are also under the influences of external environment, for example, increasing environmental concerns and rising fuel prices adding value to electric vehicles (Hardman et al., 2013). Hence, we believe that the nature of an innovation's disruptive potential is multidimensional, as technological features, marketplace dynamics and impacts of external environment are comingled and interconnected.

Assessing disruptive innovations
Identifying the disruptive potential of an innovation at its early stage can prevent the possible failure of incumbents, though no certain law of 'disrupt or being disrupted' exists (Christensen et al., 2015). Rafii and Kampas (2002) argued that decision-supporting tools are needed to assess emerging technologies, and the disruption triggered by these innovations may not be inevitable. Considering the accusation on the disruptive innovation theory of relying only on selective ex-post analysis (Lepore, 2014), such assessment tools are urged.
The approaches assessing the disruptiveness of innovations can be grouped into three main categories (Klenner et al., 2013): (a) scoring and analysis models, (b) economic models and (c) scenario and situation analysis. Among the three categories, scoring and analysis models are the most frequently used approaches. To the best of our knowledge, most of the documented scoring and analysis models are case specific. Combining publications, interviews and market reports, Hüsig et al. (2005) predicted the disruptiveness of wireless local area network (W-LAN) technology using a method of guided interviews, and pointed out that W-LAN is unlikely to become a disruptive technology. From the viewpoint of industrial practitioners, Sainio and Puumalainen (2007) evaluated the disruptiveness of four technologies: Bluetooth, WLAN, grid computing and mobile peer-to-peer (P2P) paradigm; bluetooth and WLAN are not necessarily 'disruptive', whereas grid computing and mobile P2P paradigm have higher susceptibility. Focusing on the technological performance, Keller and Hüsig (2009) used a list of innovation criteria and trajectory maps to study the potential disruption of Google's web-based office applications. They pointed out that the disruptiveness of Google applications may be compromised in the main market entry phase due to lack of compatibility and high switching costs (Keller and Hüsig, 2009). Hang et al. (2011) proposed an assessment framework for disruptive innovation, consisting of questions on three aspects: market positioning, technology and other favourable drivers. Hardman et al. (2013) used the three-part criterion to examine the potential disruptiveness of fuel cell and battery electric vehicles to the internal combustion engine (ICE) vehicles, and suggested that the fuel cell vehicles are still insufficient to disrupt the incumbents of the automobile market. Klenner et al. (2013) proposed a theoretical framework for evaluating disruptive susceptibility based on 14 conceptual propositions, and built a construct from the framework. Adopting the Disrupt-O-Meter tool proposed by Anthony et al. (2008), Hahn et al. (2014) linked the business traction of 3D printing technology start-ups to the degree of disruptiveness. Based on the four-regime-based typology of market evolution (Dijk et al., 2015), Dijk et al. (2016) suggested that full-electric vehicles are currently insufficient to displace the ICE vehicles. Hung and Lee (2016) proposed a proactive technology selection model for evaluating, selecting and improving emerging technology, and they applied the model to the 3D Integrated Circuit-Through Silicon Via (IC-TSV) technology. Roy (2018) discussed the role and characteristics of lead user in fulfilling the disruptive potential of innovation. Reinhardt and Gurtner (2018) defined 'embeddedness' as a degree to measure the position of a product in the social, market and technological systems valued by the user, yet this parameter is qualitative.
According to Section 2.1, the characteristics that define disruptive innovations are multidimensional, therefore assessing the disruptive potential of innovations should be based on multidimensional measures. The literature review suggests that the current assessments focus primarily on the technological aspect; a few of them have included the market aspect (Dijk et al., 2016;Hahn et al., 2014;Klenner et al., 2013), and external environment receives even less attention. External environment plays an important role in realising disruptive innovation (van den Broek and van Veenstra, 2018) and should be included (Li et al., 2018). Ruan et al. (2014) argued that the impact of government can be significant, as industrial policies are quite effective in cultivating disruptive innovation. In fact, the effects of such policies, laws and regulations on the disruptive potential of innovations can be either positive or negative. Yet, only positive legislations are considered (Dijk et al., 2016;Hang et al., 2011;Hardman et al., 2013). Wan et al. (2015) found that disruptive innovations are likely to arise and to be realised in emerging economies like China. Although the existing studies have identified that innovations somehow impart propelling effects on macroeconomics, such as economic growth (Hasan and Tucci, 2010;Wu et al., 2017), productivity (Feder, 2018) and employment (Frey and Osborne, 2017), the impacts of macroeconomics on disruptive innovations have still been excluded. In this study, we confine the external environment into policy and macroeconomics, as other Table 1 Definitions of disruptive innovations through their multidimensional characteristics.
Reference Definition Thomond and Lettice, 2002 Disruptive innovations are supposed to have the three characteristics that change marketplaces: (a) radical functionality, (b) discontinuous technical standards, and (c) an innovation's ownership. Radical functionality provides a user the ability to undertake a new task that is impossible before the coming of the innovation, and it disrupts markets by creating new markets. Discontinuous techniques utilise new materials or new processes. Ownership affects the development and adoption of an innovation. Govindarajan and Kopalle, 2006a Disruptive innovations have five characteristics: (a) the innovation underperforms on some attributes that mainstream customers value; (b) the new features offered by the innovation are not valued by the mainstream customers, only attract customers from an emerging or niche market; (c) the innovation tends to be simpler and cheaper; (d) the innovation initially appeals to a low-end, price-sensitive customer segment; and (e) subsequent developments improve the performance on the attributes that mainstream customers value to a level where the innovation begins to occupy more shares of the mainstream market. Hardman et al., 2013 Based on analysing successful samples like digital cameras, automobiles, hydraulic excavators, quartz watches, steam ships, eReaders and iPod, the seven characteristics are proposed to define disruptive innovations: (a) the threat of disruptive technologies is not often recognised by current market leaders; (b) disruptive technologies are initially more expensive than the incumbents; (c) the quality of disruptive technologies initially is often worse than that of the ones they seek to replace; (d) the technologies have some forms of 'adding value' to the consumers; (e) the disruptive technologies fill niches markets first, then they spread to other niches at the meso level, and eventually reach the macro level of the market; (f) the incumbent technologies are never wiped out altogether, as they might be applied in niche markets; and (g) socio-technical systems are ever evolving. Furthermore, the disruptive technologies require different manufacturers and infrastructures and are used differently. Nagy et al., 2016 An innovation that changes the performance metrics or consumer expectations of a market by providing radically new functionality, discontinuous technical standards, or new forms of ownership. Radical innovations and discontinuous innovations are corresponding to new market innovations and low-end innovations, respectively. environmental factors impart their impacts via these parameters. For instance, improved environmental awareness facilitates the industrial policy that promotes the adoption of electric vehicles (Hardman et al., 2013).

Measurement framework
In this study, the proposed measurement framework is essentially a scoring and analysis model, as the measurements of disruptive innovations are built on the basis of the ratings or scores given by surveyed personnel. Fig. 1 shows a framework of assessing disruptive innovations proposed based on the identified multidimensionality of potential disruptiveness: technological features, marketplace dynamics and external environment. The selection of these categories is in accordance with the literature review and discussion presented in Section 2. Indicators of each category are developed based on analysis of the disruptive innovation literature, particularly the works on the frameworks for assessing potential disruptiveness (Hang et al., 2011;Klenner et al., 2013). The detailed selection of these indicators is elaborated in the following subsection. The data source of the proposed measurement originates from the rating results of surveyed experts, and the rating items that form the questionnaire are based on these indicators. Similar to the assessment framework proposed by Hang et al. (2011), this framework is kept short and concise for adapting different types of disruptive innovations. Moreover, based on our previous survey experience (Gu et al., 2017a), a short and concise questionnaire can facilitate in achieving a highly effective completion rate. Table 2 summarises the definitions and explanations that justify the selection of these indicators (see Fig. 1). Apart from the contents in Table 2, several additional points must be illustrated. For the technological aspect, one commonly used technological indicator, namely, technological advance (Govindarajan and Kopalle, 2006b;Hüsig et al., 2005), is excluded, because the judgement can be highly subjective as it limited to one's background and epistemic level. For the marketplace category, the mainstream market is excluded as well as in the framework proposed by Hang et al. (2011), because linking technological capacity with marketplace feature is difficult (Gambardella and Giarratana, 2013). For the environmental category, instead of discussing the impacts of legislation as the previous literature did (Dijk et al., 2016;Hang et al., 2011;van den Broek and van Veenstra, 2018), both the external environmental indicators measure the magnitudes of possible changes associated with the external impacts, that is, the susceptibility to be affected by external environment. The proposed measurement framework enables an explicit assessment on exogenous shocks (Klenner et al., 2013), through the use of the two chosen factors.

Selection of indicators
As the literature review (see Section 2) suggests, the focus of the extant research is narrow, only technological characteristics (Govindarajan and Kopalle, 2006b;Keller and Hüsig, 2009) or market diffusion (Schmidt and Druehl, 2008). The external environment has not received sufficient attention. This proposed framework enables a holistic and quantitative measurement to assess the potential disruptiveness of innovations. Its three categories of technological features, marketplace dynamics and external environment represent the three major aspects of disruptive innovations. Table 1 implies that there could be connections between the indicators. For example, the Leadership indicator measures the potential to foster related markets and is therefore related to the Value Network indicator. The supposed connectivity of the indicators is depicted in the following subsection. Table 3 summarises the possible connections between the indicators from the categories of technological features and marketplace dynamics, as well as the relevant explanations. These connections are rather potential than factual, showing speculated relationships between  Integration Degree of the innovation merges with existing paradigms, i.e., higher level of integration means a more sophisticated deed of the innovation An innovation with a higher Integration rating means the innovation can be more easily introduced or adopted. For example, online shopping is essentially a combination of information technologies, logistics and different business modes, representing an innovation of high integration level. A higher Integration rating also means less future development is required. 'Built on existing technological skills and knowledge, or experience' is also included in the assessing measures proposed by Govindarajan and Kopalle (2006b). Leadership

Connectivity of indicators
Potential of leading related technological developments, deployments and applications The Leadership indicator measures not only the potential of adopting related technologies, but also the possibility of fostering related markets. Innovation plays a key role in cultivating a business ecosystem or an innovation ecosystem (de Vasconcelos Gomes et al., 2016), and a business ecosystem is usually considered as a consequence of a knowledge ecosystem (Clarysse et al., 2014). In the other way around, disruptive innovations are increasingly developed and commercialized by innovation ecosystems (Walrave et al., 2017).

Maturity
Maturity and reliability of the supporting technologies or the related infrastructures, especially during the early introduction of the innovation The Maturity indicator is a measure of the timing of introducing the innovation. The supporting technologies and the related infrastructures are crucial in adoption of innovations, for example, lack of infrastructure is thought to compromise the disruptive potential of electric vehicles (Dijk et al., 2016;Hardman et al., 2013). Diffusivity Easiness of diffusion of the innovation among its target audience The Diffusivity indicator evaluates the foothold of the innovation in its target market, as the innovation spreads, the foothold would become stronger. A strong foothold in the market is one of the typical characteristics of disruptive innovations (Hang et al., 2011;Yu and Hang, 2010).

Simplification
Realising certain functions that improve the satisfaction of clients through simplification of technologies.
The Simplification indicator refers to the technological replacement, where the desirable functionalities are no longer requiring some complicated operations. Simpler products are usually in favour of customers (Govindarajan and Kopalle, 2006a;Keller and Hüsig, 2009). For example, conventional film cameras are replaced by digital cameras, as the later ones are more convenient in operation, and this led to the failure of Kodak (Lucas and Goh, 2009). Marketplace dynamics

Niche market Introduction of the innovation via occupying the new niche markets
Seizing new markets is one of the typical characteristics of disruptive innovations, as well as adding value to the stakeholders (Hardman et al., 2013;Yu and Hang, 2010). The success of Tesla could be attributed to the occupation of a limited, luxury niche market where high-price sport electric vehicles are acceptable to customers (Dijk et al., 2016). Value network Profitability of upstream, downstream and all other collaborative firms associated with the innovation de Vasconcelos Gomes et al. (2016) pointed out that innovation ecosystem is about to create value. The Value Network indicator is hereby proposed to evaluate the profitability from the innovation rather than to assess its attack on the established value networks. The capacity of constructing a value network is also valued in the assessment framework proposed by Klenner et al. (2013).

Cost reduction
Reducing the cost of acquiring certain functions, services or products, that is, introducing the innovation through the low-end markets.
A typical type of disruptive innovations are low-end innovations, which possess similar characteristics to the existing technologies but at a lower cost. In the classic theory of disruptive innovations (Christensen, 1997a;Christensen, 1997b;Christensen et al., 2015), the term of 'disruptive innovation' generally refers to the low-end innovations. The mainstream customers would favour new low-end products, provided that these low-end products offer enough quality (Schmidt and Druehl, 2008). External environment Policy Scale of policy-related impact on development and adoption of the innovation, both positive and negative The positive effects of policy have been included in the framework proposed by Hang et al. (2011) in term of 'helpful legislation', while the negative impacts are generally neglected. van den Broek and van Veenstra (2018) discussed the regulatory impacts on big data collaboration and recommended hierarchical governance arrangement for realising the disruptive potential. Since subsidies are more effective in stimulating technological developments than loans (Huergo and Moreno, 2017), uncertainties and changes in related industrial policies could also impart negative impacts on innovations and compromise their disruptiveness (Dijk et al., 2016), especially considering that the role played by government in cultivating disruptive innovation is highly dynamic (Ruan et al., 2014).

Macroeconomics
Influence of macroeconomic situation on the development and adoption of the innovation According to classical endogenous growth models, technological developments are positively correlated to economic growth. Ulku (2005) found that the correlation might be unilateral, as macroeconomic status affect innovation in a more significant fashion. Besides, for various products, their price elasticity of demand could also be affected by macroeconomic factors (Tellis, 1988). the indicators. Apart from presenting the potential inherent logic of the framework and its components (indicators), we purpose this connectivity to provide a weighting function for the rating items corresponding to the indicators. Individual indicators are insufficient to offer an overall quantified measurement of candidate innovations' disruptive potential. Adopting the work of Freeman (1978), the number of indicators connected to one single indicator is denoted as 'degree', measuring the involvement of the indicator in the network. In the assessment, the degree is used as the weight of the scoring item that corresponds to the indicator. External environment indicators, i.e. the Policy and Macroeconomics indicators, exert their impacts on the overall performance of the connected network. Both the indicators measure the magnitude of the corresponding variations of innovations and their adoption caused by external environmental factors. For innovations with high Policy and/or Macroeconomics ratings, external environments can either significantly promote or hinder the fulfilment of their disruptive potential. Considering that disruptive innovations are dynamic processes (Christensen et al., 2015;Govindarajan and Kopalle, 2006a;Hardman et al., 2013) and even industrial policies are occasionally counterproductive (Dijk et al., 2016), high immunity to exterior impacts is highly desirable. Accordingly, the inverse production of the ratings of the external environmental scoring items that correspond to the indicators of Policy and Macroeconomics are employed as the multiplier.
Based on Table 3, we construct a network of the selected indicators by exploiting their intrinsic connectivity (see Fig. 2).
Overall, this framework is supposed to be competent in assessing the disruptive potential of innovations, because its indicators not only cover the most important characteristics of disruptive innovations, but also explain the inherent relationships among the characteristics through the network of indicators. Although the technological and marketplace aspects are highly interrelated in the selected case studies of the extant literature, such as electric vehicles (Dijk et al., 2016;Hardman et al., 2013) and 3D printing (Hahn et al., 2014), no explicit depiction is provided to illustrate the nexus. Based on a structured scoring and analysis method, Hung and Lee (2016) constructed a causeand-effect relationship plot among the improvement target factors and causal factors. However, the cause-and-effect plot tends to be highly case specific, given that its design is relied on the surveying results of a target case. The proposed framework is generalised, as it not being limited to providing a quantified ex-ante analysis but also extended to unveiling certain intrinsic traits of disruptiveness via the topological features of the indicators, that is, connectivity of the indicators.

Procedure of assessing innovations
In Appendix A, we prove that an appropriate measurement space Table 3 Possible connections between indicators and their explanations.

Possible connection Explanation
Integration -leadership According to the given definition (see Table 1), the Integration indicator is relevant to the degree of combining with existing paradigms, thereby it seems to be naturally related to the Leadership indicator.

Integration -simplification
As several technologies or business models merged, it would be much more convenient to achieve certain functionalities and incumbent is thereby likely to be replaced by an entrant, e.g. online shopping enables surfing and purchasing simultaneously and results in decline of conventional stores (Worstall, 2015). In this sense, the Integration indicator could be associated with the Cost Reduction indicator. Integration -value network Similar to the Integration -Leadership connection, as the Value Network indicator is also concerning about the coordination of stakeholders; the Integration indicator is likely to be associated with the Value Network indicator. Integration -cost reduction Similar to the Integration -Simplification connection, combining extant technological paradigms saves future investment for research and development, as well as the cost for customers to adopt such innovations. Affordable future investment (Hang et al., 2011) and appealing adoption cost (Christensen, 1997a;Christensen, 1997b;Christensen et al., 2015) are considered as the possible triggers of disruption. Hence, there could be some degree of connection between the Integration indicator and the Cost Reduction indicator. Leadership -diffusivity According to Clarysse et al. (2014), more connections promote the speed of innovation diffusion. Since the Leadership indicator measures the potential of leading related technological developments, deployments and applications, as defined in Table 1, it is believed that there could be links between the Leadership indicator and the Diffusivity indicator. Leadership -value network Through coordinated innovation ecosystem, the disruptive innovation has higher possibility to create value for stakeholders (Walrave et al., 2017), that is, the Leadership indicator is likely to be relevant to the Value Network indicator. Leadership -cost reduction Typically, substantial cost reduction is achieved via development, deployment and application of a series of related technologies, e.g., marine renewable energy (MacGillivray et al., 2014) and electric vehicle (Dijk et al., 2016). In addition, a reduction in costs can be considered as a consequence of a learning-by-doing effect which triggered by technological closeness (Andergassen et al., 2017). Technological closeness is also positively correlated with economic development, and isolated innovation is insufficient to oust monopolistic incumbents (Andergassen et al., 2017). Based on the above discussion, it can be deduced that the Leadership indicator might be related to the Cost Reduction indicator.

Maturity -cost reduction
Similar to the discussion of the Leadership -Cost Reduction connection, readiness of supporting technologies and infrastructures also has significant impact on adoption of new innovation, e.g. the case of electric vehicle (Dijk et al., 2016;Hardman et al., 2013). Thus, the Maturity indicator is supposed to have some linkage with the Cost Reduction indicator.

Diffusivity -simplification
Low-cost disruptive products attract customers for their easiness of use and lower prices (Adner, 2002;Christensen, 1997b;Schmidt and Druehl, 2008). On the basis of this commonly-recognised statement and the survey conducted by Reinhardt and Gurtner (2018), it can be deduced that the Diffusivity indicator could somehow be linked to the Simplification indicator.

Diffusivity -value network
Since reconfiguration of value chain is a basic characteristic of disruptive innovations (Markides, 2006) and the invasions of disruptive innovations usually take place in low-end markets (Govindarajan and Kopalle, 2006a), the Diffusivity indicator could be related to the Value Network indicator. Moreover, the easiness for customers in value networks to switch preference determines the likelihood of adopting new innovations, that is, innovations of higher disruptive potential are more likely to be adopted (Klenner et al., 2013).

Diffusivity -cost reduction
Referring to the Diffusivity -Value Network connection, as the hypothesis of 'low-end offers in value networks increase disruptive susceptibility/tested in the work of Klenner et al. (2013). Further, based on the explanation on the Diffusivity -Simplification connection, the Diffusivity indicator and the Cost Reduction indicator are thought to be somehow linked.

Simplification -cost reduction
The Simplification indicator and the Cost Reduction indicator are considered to be naturally linked, as the aforementioned discussion, particularly the explanations of the Integration -Simplification connection and the Integration -Cost Reduction connection, implies. Niche market -value network In the early stage of the lifecycle, disruptive innovations dwell in niche markets where their value networks are built (Vecchiato, 2017). In this sense, the Niche Market indicator is linked with the Value Network indicator. Niche market -cost reduction As there could be overlapping between niche and low-end markets (Govindarajan and Kopalle, 2006a), we speculate that the Niche Market indicator can be associated with the Value Network indicator at some certain degree. Value network -cost reduction According to Klenner et al. (2013), value networks play an important role in determining the disruptive susceptibility of an innovation; its size and other features like customer loyalty and lobbying power are found to be related to an innovation's disruptive potential. These features impart their impacts through altering the required cost of adopting such an innovation. Thus, the Value Network indicator and the Cost Reduction indicator could be connected. enables all the necessary arithmetic operations can be constructed within the framework. Therefore, each attribute can be quantified to perform a quantitative evaluation on a candidate innovation. The assessment procedure consists of four steps: constructing eigenspaces, conducting surveys, calculating results and interpreting the findings.

Constructing eigenspace
The initial step of assessing disruptive innovation is to construct the eigenspace. The eigenvectors are the concretisations of technological, marketplace, and external environment factors. In this step, the eigenspace and eigensubspaces are constructed according to the categories and their indicators shown in Fig. 1.

Conducting surveys
A questionnaire is designed, and all the indicators of the three categories are included in the form of rating items. Respondents of this survey are industrial experts holding PhD degree in engineering and have over five years of working experience. To survey the desirable respondents, the questionnaires are distributed through online social professional networks because these networks enable straightforward information access (Brandão and Moro, 2017). The experts rate every scoring item, and the returned questionnaires are collected for further calculation and analysis.

Calculating results
According to the mathematical proof shown in Appendix A, the measurement framework is a proper measure space that enables desirable calculations like adding and multiplying. Based on Fig. 2 and related discussion in Section 3.1, the calculation procedure consists of two steps. First, the average ratings of the scoring items that are based on the indicators in the categories of technological features and marketplace dynamics are multiplied by their corresponding degrees. Second, the sum of the weighted ratings is multiplied by the inverse ratings of the external environmental scoring items. The calculation can be summarised in the following equation: where DII refers to 'Disruptive Innovation Index' (DII), which is a quantitative measure of intrinsic disruptive potential of innovations; a i refers to the rating of the scoring item based on the indicators in the technological and marketplace categories; d i is their corresponding degree, which is defined as the weight of the indicator (defined in Section 3.1.2) and b j refers to the rating of the scoring item based on the external environment indicators, i.e. the Policy and Macroeconomics indicators.

Interpreting results
According to Fig. 2 and the calculation process, the result consists of three segments: (a) Overall performance, which refers to DII computed using Eq. (1) (b) Techno-market performance, which refers to ∑a i d i (c) Immunity to external environment, which refers to ∏ b 1 j In this framework, high overall and techno-market performance indicates a high degree of disruptive potential. The immunity to external environment can be very tricky. Although high immunity is valued in this assessment framework, the innovations with low immunity cannot be considered as non-disruptive, because environment factors like industrial policy, will play a significant role in fostering these innovations. With proper external supports, the innovations with lower immunity can be more potentially disruptive than those with higher immunity. Thus, a proper analysis of external environment is critical, particularly for the vulnerable ones. Analysing the ratings corresponding to the indicators is also required, providing implications such as changing marketing focus and reallocating product functionality.

Selection of cases
In this study, we select three cases: WeChat (denotes successful innovations), Modularised Mobile Phone (denotes failed innovations) and VR/AR (denotes ongoing innovations).

(a) WeChat
WeChat is a social networking mobile application software with integrated services, including instant messaging, social network, online commerce and payment services. The software was developed and launched by a small group of developers in the email branch of a Chinese internet giant -Tencent -in 2011, with the original functionality of providing instant messaging service for mobile phone users. WeChat enables text and voice communications at a lower cost than the similar services offered by traditional telecommunications operators; only electric energy is consumed, and no other fees are charged. Over time, the performance of WeChat improves, and more functions are included as well. Currently, it is one of the world's largest standalone messaging applications; by the first quarter of 2018, it has one billion monthly active users (Statista, 2018). Over 70 million WeChat users are outside of China, posing a real threat to other popular messaging apps such as Messenger, WhatsApp, KakaoTalk and Line (Business Insider, 2016). E-commerce and payment services of WeChat also enjoy rapid growth. According to Wang et al. (2017), in 2016, almost a third of WeChat's users made online purchases on WeChat stores. To a certain extent, the development of WeChat fits the description of disruptive innovations given by Christensen et al. (2015). The WeChat case thereby represents a successful case.

(b) Modularised mobile phone
Modularisation is a concept and a design approach that divides a system into smaller segments called modules which are independently created and used in different systems. The purpose of developing modularised mobile phones is twofold: (i) to realise mass customisation, that is, manufacturing customised goods at the cost of mass production (Gu et al., 2018) and (ii) to provide a possible solution to the ewaste disassembly problem, particularly to lower the life-cycle cost of mobile phones. The first modularised mobile phone was designed and launched by Modu, an Israel company, in 2008 (Wong, 2008). Closed Loop Emotionally Valuable E-waste Recovery (Clever), a UK-based project aims at eliminating mobile phone waste, has developed a prototype modularised mobile phone on the basis of a 'skeleton' made of a plastic/cellulose composite, which can be dissolved into sugars in the presence of engineered bacteria. Components such as battery, screen, motherboard and memory are attached to the 'skeleton' as 'organs' (Scott, 2014). Google briefly launched a modularised mobile phone project in modularised mobile phones-Alphabet's Project Ara; however, this project was cancelled in 2016 (Morris, 2016). The Modularised Mobile Phone case represents a failed innovation.
(c) VR/AR VR is the synthesis of 'reality' as a mean to create an intuitive method for human computer interaction via simulating sensations (Lv et al., 2017). AR is primarily derived from VR, and it develops combined environment where the virtual objects are integrated into a real scene in real time (Zhao et al., 2017). The major advantages of VR/AR include convenience, economy, good interactivity and security (Baus and Bouchard, 2014). Although both technologies have been introduced since the 1990s (Baus and Bouchard, 2014), they have become a recent hot zone of investment (Digi-Capital, 2017). This is possible due to the rapid development of information and communication technologies, particularly the smart and wearable equipment. The applications of the VR/AR technologies can be found across various sectors, including military, medical, manufacturing, education, construction, and transportation (Baus and Bouchard, 2014;Palmarini et al., 2018;Wang et al., 2013). However, most of these applications are still in their pilot stages. The VR/AR case represents an ongoing innovation.
Although the fates of a few innovations are well known, e.g., the success of WeChat and the failure of Modularised Mobile Phone, the purpose of selecting these cases is to prove that our framework can unveil their intrinsic characteristics that affect their disruptive potentials. The ex-post analysis of innovations with sealed fates is credible, given that the outcomes of the ex-ante analysis on ongoing technology cannot be verified promptly. This is an innate weakness of any technology assessment method.

Surveying experts
A questionnaire that contains 10 scoring items correspond to the selected indicators is formulated and shown in Fig. 1. Its detailed design is presented in Table A1 in the Appendix. Through online social professional networks, mainly the personal connections of the authors, the questionnaires are dispatched to a panel of industrial experts with PhD degree in engineering and over five years of working experience. Characterised by high speed, convenience and high efficiency (Zhang et al., 2017), this online survey was conducted from 1st Jan to 30th April 2017. To ensure the authenticity of the survey results, the following measures are adopted: (a) Each respondent's Internet protocol is recorded, and duplicate responses from the same IP addresses are excluded. (b) The time spent on filling out the questionnaire is recorded as well.
According to the reading habits (Kong et al., 2018), a questionnaire with a timespan of less than 15 s is considered invalid.
A total of 59 qualified experts are surveyed, and 55 validated returned questionnaires are received. The availability rate was 93.2%, with an average completion time of 218.4 s. The original results are presented in the supporting information (SI).

Verification of connectivity
Before the computation of the DII values, the Pearson correlation coefficients (PCC) between the ratings of the connected indicators (see Fig. 2) are calculated to verify the connectivity in the proposed network. PCC is a measure of the linear correlation between two arrays and is calculated using Eq. (2) (Rodgers and Nicewander, 1988). The PCC  Table 4 in which the linkages of the other pairs of the indicators are also presented with their PCC values. (2) Table 4 shows that all the PCC values of the speculated connections are greater than or equal to 0.4, whereas the PCC values of the other pairs of indicators are no more than 0.43, only the PCC values of the three pairs are greater than 0.4. This result can be employed as partial evidence that supports the connectivity of the proposed network (see Table 3 and Fig. 2). Hence, the degree of the indicator (d i ) can be validly used as the weight of the corresponding scoring item. Table 5 shows the calculated DII values (overall performance of assessed innovations) and the detailed ratings of scoring items that correspond to the indicators, along with the results of the two segments of Eq. (1), i.e., techno-market performance and immunity to external environment. According to Table 5, WeChat has the highest DII value as well as the highest techno-market performance and immunity to external environment. For each indicator in the categories of technological features and marketplace dynamics, WeChat has the highest rating in each and every corresponding scoring item. The success of WeChat is compatible with the survey results, and the failure of Modularised Mobile Phone is also reflected by its ratings. VR/AR has a higher techno-market performance than Modularised Mobile Phone. However, the lower immunity compromises its overall performance and consequently makes its DII value the lowest among the three innovations, even lower than that of Modularised Mobile Phone. Yet, we cannot affirm that the fate of VR/AR is doomed.

Analysis of results
According to Section 3.2, a simple calculation of survey results is neither convincing in explaining the success or the failure of innovations, nor reasonably sufficient in performing an ex-ante evaluation. A detailed analysis on the result of each and every selected indicator is performed, as additional evidence is gathered and presented.
WeChat is basically a combination of instant messaging, social network, online commerce and payment services; hence it obtains the highest score in the Integration indicator. By contrast, developing Modularised Mobile Phone requires a redesign of the structure as well as all the modules, whereas most mobile phones on the market are integrated (Dodbiba et al., 2016). Modularized Mobile Phone receives the lowest rating in the Integration indicator. VR/AR combines hardware (e.g. sensors) and software (e.g. image-processing programme), but their patterns are highly diversified (Palmarini et al., 2018). Consequently, the innovation gains a moderate score in the Integration indicator. WeChat provides an e-commerce environment and represents a typical example of social commerce (Sun et al., 2016). In addition, WeChat offers a platform for anyone to build embedded apps called 'mini programmes', which grant direct accesses to multiple businesses and services like ordering food, booking cinema or train tickets and renting cars (Millward, 2017). Moreover, the availability of mini programmes facilitates consumer-developed innovations, which can only diffuse to a limited extent in conventional channels (de Jong et al., 2015). Undisputedly, WeChat obtains the highest rating in the Leadership indicator. Modularised Mobile Phone has few relations with other technological developments other than modularisation, whereas VR/AR has the potential to lead the development of multiple related sectors such as user interface design, hardware manufacturing and the gaming industry (Digi-Capital, 2017). Consequently, the ratings in the Leadership indicator of Modularised Mobile Phone and VR/AR come at the third and the second places, respectively. WeChat receives the highest rating in the Maturity indicator because its primary supporting technologies are smartphones and the Internet, which are mature and extensively adopted. For both Modularised Mobile Phone and VR/AR, their supporting technologies or the related infrastructures are quite uncertain; thus, lower ratings in the Maturity indicator are assigned to them. Launching of WeChat fills the vacancy in the instant messaging mobile applications back in 2011, and using red packet to popularise the payment service of WeChat has been recognised as a brilliant piece of marketing strategy (Williams, 2016). In the Diffusivity indicator, WeChat consequently obtains the highest score. To the best of our knowledge, Modularised Mobile Phone has no specific market preference as it must compete with other mobile phones from dominating incumbents like Apple and Samsung. The Diffusivity rating of Modularised Mobile Phone is the lowest among the three cases. Although VR/AR has tremendous implications in many fields like construction (Wang et al., 2013) and surgery (Baus and Bouchard, 2014), this type of innovations is not yet widely adopted in these fields. Moreover, the potential applications of VR/AR are scattered across various sectors as investors are fuelling up their own different start-ups (Digi-Capital, 2017). Given the above reasons, the Diffusivity indicator of VR/AR is rated in the midst of WeChat and Modularised Mobile Phone. As WeChat combines a series of different functionalities without introducing any extra requirements, it fits the definition of Simplification, as the highest score indicates. By contrast, VR/AR is a highly complicated cyber-physical system consisting of elements, components and sub-systems of multiple layers to achieve human-machine interactions (Baus and Bouchard, 2014;Wang et al., 2013). As a result, the innovation obtains the lowest rating in the Simplification indicator. Modularised Mobile Phone offers no convenience other than easiness of assembly and disassembly as its medium rating of the Simplification indicator implies.
Before the launch of WeChat, another Tencent product-OICQ-was dominating the market of instant messaging in China (Ju and Tao, 2017), and e-commerce market was dominated by Taobao and JD. WeChat successfully survives in the fierce market competitions; a rapid increase in the number of monthly active users (Statista, 2018) confirms the exceptional performance of WeChat in the Niche Market indicator. Owing to its diversified implications, VR/AR shows certain potential in capturing specified markets; therefore, it is ranked in the second place of the Niche Market indicator. Modularised Mobile Phone comes at the bottom in the Niche Market indicator because this innovation has never captured a proper market share, even development was renounced (Morris, 2016). WeChat has earlier established a business ecosystem via online social network (Sun et al., 2016) and the mini programmes (Millward, 2017). Thus, it earns the highest score in the Value Network indicator. Modularised Mobile Phone receives the lowest rating in the Value Network indicator, given that the innovation has little prospect in forming its own value network.
Considering the wide range of applications and the complexity in the structure, VR/AR has the potential to set up a vast business and innovation ecosystem and thus holds the second place in the Value Network indicator. Nowadays, WeChat not only provides a combination of services but also enables online commerce for individuals; a series of costs (e.g. trading fees) are substantially reduced owing to disintermediation. Hence, WeChat once again gains the highest score in the Cost Reduction indicator. Modularised Mobile Phone requires resignation of the structure for modularisation; thus, its score in the Cost Reduction indicator comes at second. Introduction of VR/AR requires various components and subsystems as well as tasks like data collection, training, deployment and maintenance. Although the innovation is supposed to be cheap, safe and convenient in its application scenarios (e.g. being adopted as a training tool) (Baus and Bouchard, 2014), its effects in cost reduction still await full recognition; it gets the lowest rating in the Cost Reduction indicator.
In the Policy indicator, Modularised Mobile Phone receives the highest score, given that one of the initiatives of launching such projects is to facilitate mobile phone recycling (Scott, 2014). Both WeChat and VR/AR are less likely to be affected by legislation issues, and they require little support from industrial policy. Therefore, both innovations gain lower ratings in the Policy indicator. As discussed previously, VR/AR represents a complicated cyber-physical system, which requires highly sophisticated hardware and software as well as welleducated operation personnel. As a result, this innovation gains the highest score in the Macroeconomics indicator. For only requiring smartphones and access to the Internet, WeChat gains the lowest rating in the Macroeconomics indicator. Modularised Mobile Phone is rated at the second place in the Macroeconomics indicator, as the innovation has implicit demand for improved manufacturing paradigms, e.g. mass customisation.
Based on the detailed analysis on each indicator, we summarise several observations and implications regarding the disruptive potential of the three innovations. The successful adoption of WeChat can be possibly attributed to its exceptional techno-market performance and high immunity to external environment. Combining several functions, WeChat succeeds in creating its own business ecosystem as well as in reducing costs for its users. Brilliant marketing also facilitated WeChat in capturing a high market penetration rate of its derivative service-payment. The technical nature of WeChat is an integration of existing technologies; not only that massive investment in technological development is no longer required, it also shows an effective resistance to possible external impacts. The Modularised Mobile Phone's failure can be deduced from its weak purposefulness as the innovation offers no clear benefits to its stakeholders other than the convenience in assembly and disassembly. The capacity to facilitate recycling is indeed a sustainable practice. However, a brief analysis shows that disassembly is not the current bottleneck of mobile phone recycling, but rather the collection rate is the most critical issue (Gu et al., 2017b). Currently, the collection rate of spent mobile phones is fairly low, even for countries with well-established waste management systems (e.g. Switzerland) (Duygan and Meylan, 2015). For the concerns, such as fear of privacy leakage and no knowledge of recycling facilities, a large proportion of spent mobile phones remain idle (Gu et al., 2017a). Attention focuses on addressing the collection problem (Gu et al., 2017b) rather than recycling. The failure of Modularised Mobile Phone is explained by both its intrinsic weaknesses and adversary external environment. For the ongoing innovation of VR/AR, the analysis shows that it is insufficiently disruptive at its present stage, particularly considering its complexity and high demands for its adoption. A key issue in increasing the disruptive potential of VR/AR is to identify and seize a suitable niche market. Currently, investment on this innovation is dispersed in multiple sectors (Digi-Capital, 2017). However, VR/AR poses a high standard on equipment, infrastructure, software and operators, and its application paradigms still have to be established. Consequently, full-scale adoption of this innovation is not solidly plausible at the current stage. Successful adoptions of disruptive innovations require clear target incumbents or markets, e.g., digital cameras replacing film cameras (Lucas and Goh, 2009) and Tesla seizing the market for sports electric cars (Dijk et al., 2016). Despite VR/AR's extensive range of potential applications (Baus and Bouchard, 2014;Digi-Capital, 2017;Wang et al., 2013), neither specific incumbents nor occupied niche markets have yet been reported. Pragmatically, a breakthrough in penetrating a certain specific market can boost the confidence of the developers, investors and administrators. Focusing investment and research on a few niche points expected to break through is highly desirable for VR/AR, and the good news is big money is continuously flowing into this field (Digi-Capital, 2017).

Comparison of methods
We briefly compare the methods that aim at assessing the disruptive potential of innovations. Our method and a few of the reviewed methods in Section 2.2 and a few case-specific evaluations are neglected due to the obvious flaws. The comparison is essentially made from three dimensions: (a) quantitative or qualitative, (b) inclusion of the impacts of external environment and (c) consideration of the interrelations among scoring items. Table 6 summarises the descriptive results of this comparison, wherein a few merits of our proposed multidimensional measurement framework over the extant assessment methods are demonstrated.

Robustness check
Considering that the outcomes are actually based on the surveying opinions of a small group of engineering experts, the robustness of the proposed measurement framework should be checked. Given that the selection of such experts is a highly demanding task, an important question requires an answer: does this framework solely rely on experienced experts with flamboyant academic degrees? To address this question, the same questionnaires as the ones used in Section 4.2 are distributed to two groups of survey subjects: Group (1) consists of people holding master's degrees and Group (2) includes people holding bachelor's degrees or below. 100 questionnaires have been dispatched to these two groups, and 88 validated returned questionnaires are received. 44 validated returns exist for each group, as the quality control measures are the same as the previous survey. The original results of these two groups are also presented in the SI.
(2), the PCC values of the two groups are calculated and presented in Table 7.  Table 4, and the robustness of the connectivity is proven to an extent.
Based on the returned questionnaires from the two groups, Table 8 Table 6 Comparison of methods for assessing disruptive potential of innovations. shows the calculated DII values of the selected innovations. In general, the results in Table 8 prove the robustness of the proposed framework in assessing the disruptive potential of innovations; a pattern that is similar to the initial results in Table 5 is observed. In the results of both the two groups, WeChat possesses the highest DII value and the highest techno-market performance, and the scores for the techno-market performance are ranked in the following order: WeChat, VR/AR and Modularised Mobile Phone, being consistent with the results shown in Table 5. In the results from Group (1), despite the poorest technomarket performance, Modularised Mobile Phone remains at the second place in terms of the DII value, and VR/AR holds the last place owing to its vulnerability to external environment. In the results of Group (2), WeChat has the highest immunity to external environment, followed by Modularised Mobile Phone, and VR/AR comes last. The observations are similar to those of Table 5, indicating that the proposed framework is robust in terms of its generic outcomes. The proposed question is thereby partially answered. The framework does not only rely on experts with rich experience and high degrees. Similar outputs can be derived via querying other groups of people, provided that the framework is used. However, certain differences are noted between the initial and robustness check results. According to the results of Group (1) shown in Table 8, Modularised Mobile Phone has the highest immunity, whereas innovation takes the second place in the immunity score in Table 5. In the results of Group (2), VR/AR takes the second place in terms of the DII values, a position that formerly belongs to Modularised Mobile Phone (see Table 5). The existence of these major discrepancies implies that the differentiated ability of the different groups of respondents in recognising the characteristics of innovations can be associated with the different levels of academic training, as Heidenreich (2009) pointed out that employment of doctorate holders can be a crucial factor in promoting knowledge transfer and innovation. This finding provides side evidence to support the disproof of the claim 'too many PhDs' (Santos et al., 2016).

Conclusions and discussion
This study develops a quantitative measurement framework to assess disruptive innovations with regard to their multidimensional characteristics. This study intends to add value to the ongoing discussion on the ex-ante approaches aim at identifying disruptive innovations (Hang et al., 2011;Keller and Hüsig, 2009;Klenner et al., 2013). Based on the analysis of the literature and concepts, ten indicators are developed from three categories of multidimensionality, namely, technological features, marketplace dynamics and external environment, to represent the multidimensional traits of innovations. Some of the indicators are then connected, and the weight of the indicator is assigned according to its degree of connectivity. The indicators are developed into scoring items that form a survey questionnaire (see Table A1). The assessing procedure is illustrated using a case study of three innovations: WeChat (successful), Modularised Mobile Phone (failed) and VR/ AR (ongoing). Based on the survey results from a panel of industrial experts holding PhD degree in engineering and over five years' working experience, the three innovations are thoroughly evaluated according to every indicator. As a combination of existing technologies and paradigms, WeChat obtains the highest overall score as it represents an excellent example of integration, value network construction and cost reduction as well as good market penetration capacity. With limited advantages and questionable purposefulness that can be provided, the failure of Modularised Mobile Phone seems inevitable as its overall rating implies. For VR/AR, its overall performance is rated in between WeChat and Modularised Mobile Phone, its current stage is insufficiently disruptive; seizing a few niche markets can be the key to fulfilling its potential disruptiveness. The results of robustness check suggest that the measurement framework does not only rely on wellselected experts; similar outcomes can be obtained by surveying other groups of people with low academic degrees.
The novelty of the proposed framework is threefold. First, the vulnerability to external environment is initially included. The existing literature focuses primarily on technological and market perspectives (Gatignon et al., 2002;Govindarajan and Kopalle, 2006a;Hang et al., 2011), whereas external impacts receive little attention. Considering that the institutional factors have earlier been considered in the  research on development and adoption of innovations (see Dijk et al., 2015;Gao et al., 2017;Pérez and Ponce, 2015;Pulkki-Brännström and Stoneman, 2013;Roy and Cohen, 2015;Ruan et al., 2014;Wan et al., 2015), the immunity to these factors shall be included in assessing the disruptive potential of innovations. In the proposed framework, the magnitudes of the possible variations in development and adoption of innovations by external impacts are included and surveyed in the form of the external environment indicators. Two indicators are selected, namely, the Policy and Macroeconomics indicators as inspired by the previous study on the effects of subsidies on innovations (Huergo and Moreno, 2017) and productivity changes related to innovations (Feder, 2018). The immunity is expressed using the inverses of the ratings that correspond to the external environmental indicators. To include external impacts, the framework is comprehensive in assessing the intrinsic disruptive potential of innovations. Notably, the actual institutional factors should be combined to conduct a precise analysis as suggested by Reinhardt and Gurtner (2018). Second, a mathematical formulation of intrinsic disruptive potential (Eq. (1)) is proposed from a discussion regarding the definitions of disruptive innovation and the proposed measurement framework. The framework itself is constructed as a measurement space, where the internal connectivity of the selected indicators can be quantitatively presented and the ratings can be validly calculated. Two segments exist in the mathematical formulation of disruptive potential, namely, the techno-market performance and the immunity to external environment. The former segment consists of a network of connected indicators built upon the literature and conceptual analysis, and the latter segment measures the vulnerability to external impacts. Surveying results of the case study confirm the connectivity and verify the applicability of the measurement framework. Furthermore, the connectivity passes the robustness check, exhibiting the persistence in the designated relations between the indicators.
Third, this framework can be added into the literature of the ongoing discussion on the ex-ante approaches to assess potentially disruptive innovation (see Danneels, 2004;Dijk et al., 2016;Govindarajan and Kopalle, 2006a;Hang et al., 2011;Hüsig et al., 2005;Keller and Hüsig, 2009;Klenner et al., 2013). However, conducting a rigorous exante prediction on the disruptiveness of an innovation is nearly impossible. The proposed measurement framework constitutes a new set of indicators, which are developed based on literature and conceptual analysis. In the case study, the indicators and their network can effectively reflect the intrinsic characteristics of disruptive innovations, because the causes of successes and failures of the selected innovations are well explained. By combining the functionalities for adding values to its users and value networks, WeChat has been successfully introduced and has gained a dominant market position. For Modularised Mobile Phone, the lack of offerings and its easiness to be affected externally sealed its fate of being disruptive. The differences between the successful and failed cases are embodied in the indicators within the framework. As an ongoing innovation, the ratings of the indicators show that VR/AR is insufficiently disruptive because it is still struggling to penetrate various markets; big money continuously flows into this area (Digi-Capital, 2017). Furthermore, suggestions are provided to fulfil the disruptive potential of VR/AR, including focusing development and seizing niche markets. The findings are compatible with market threshold theory (Klenner et al., 2013), given that gaining a market share is critical to ensure the disruptive potential of innovations.
The study has tremendous implications for both theoretical and managerial perspectives. From a theoretical perspective, a reconsideration of disruptive characteristics has been suggested by the framework and the findings derived from the case study. Several factors that previously evaluated the disruptiveness of innovations, such as performance overshooting (Christensen, 1997a;Hang et al., 2011;Yu and Hang, 2010) and radical functionality (Dahlin and Behrens, 2005;Thomond and Lettice, 2002), are excluded. According to the case study, the framework can still explain the developments of the selected cases without these factors. Given that WeChat is a combination of functionalities and therefore offers no radical ones than the incumbents, performance overshooting also fails to work well with the case. For both Modularised Mobile Phone and VR/AR, single radical functionality is unlikely to ensure the realisation of their disruptive potential. This finding is compatible with the analysis on gas turbines and automotive power-train industries (Bergek et al., 2013). For an innovation with a vast range of potential applications, like VR/AR, identifying the performance overshooting is a challenging task. The importance of performance overshooting is further weakened because it is no longer considered an essential characteristic for disruptive innovation (Klenner et al., 2013). Instead of analysing the legislative impacts (Dijk et al., 2016;Hang et al., 2011), the study focuses on the innovations themselves by assessing their immunity to the external shocks. In general, the indicators in the proposed framework are proven to be explanatory in unveiling the intrinsic characteristics of assessed innovations as well as the inherent connections between these characteristics. Focusing on the nature of disruptive innovations, from the technological configuration to the magnitudes of responses to external influences, this study provides insights into research on this particular field. In addition, the results of robustness test open a unique perspective in the ongoing discussion on the value of education in technological development and management (Herrera and Nieto, 2016;Müller et al., 2018;Santos et al., 2016).
From a managerial perspective, an inclusive and quantitative measurement framework to identify the disruptive potential of innovations is helpful for any incumbent or potential investor. The proposed framework is a case-independent multidimensional measurement tool that merges technological traits and business model, as Markides (2006) suggested, thereby suggesting its potential extensive applications. If the external environment is highly volatile and unpredictable, then measuring the immunity of technologies can create a feasible ex-ante prediction in identifying potentially disruptive innovations. Innovations with high immunity to external impacts are supposed to have a great chance of becoming successfully disruptive. The indicators, the analysis of the selected cases and their surveyed results also provide guidance for the development of technologies and relevant products. The importance of proper marketing strategy is underlined as well.

Limitations and future research
Our proposed framework is proven effective in explaining the possible reasons underlying the success or failure of the selected cases. However, several limitations exist in both the methodology and case study. First, the framework is working well on the basis of the surveyed experts, although other groups of respondents yield slight differences in the outcomes. The ratings of the scoring items that correspond to the indicators are assigned by respondents; therefore, the framework suffers the same drawbacks as any other scoring and analysis models. Although the stringent selection of experts in this study is supposed to possess high trustworthiness in the field of engineering, personal knowledge and experiences remain limited. Moreover, subjectivity in the survey results cannot be ruled out. History tells us that experts, even the most experienced and educated ones, can also incur fatal mistakes in predicting the future (Strohmeyer, 2008). Surveying a panel of experts is also costly.
Second, the evolutions of the selected innovations have not been fully analysed, and detailed environmental changes during their development and adoption are also neglected. The case study focuses primarily on presenting evidence and analysis to explain the ratings of the indicators and subsequently demonstrate the effectiveness of this measurement framework. Technological evolutions can contribute a significant role in the adoption of disruptive innovations as well as the changes in the market behaviours (Dijk et al., 2015). However, in the extant literature, including this study, the dynamic and progressive nature of the disruptive innovations, that is, technological evolutions in innovations, still await sufficient attention. Considering that disruptive innovation is essentially a process (Christensen et al., 2015), reviewing the developmental trends of innovations is highly desirable.
Third, the cases selected in our case study are rather simple and well known. Thus, such qualities can partly explain why all the three surveyed groups provided similar ratings. When encountering other complicated innovations, such as autonomous driving, autonomous drones and block-chain technologies, huge discrepancies may occur in the perceptions of different groups of people. It can pose an extremely difficult challenge to all the scoring and analysis models that assess the disruptive potential of innovations; thus, a new method may be required.
A series of minor limitations are also observed in this study, including the narrow focus of survey respondents, the inadequate linking of external environmental indicators to techno-market performance and inadequate comparisons between the entrants and the incumbents in the case study.
To address the limitations identified above, future research can be carried out in the following directions: (a) To identify a few appropriate, objective and generic datasets for measuring disruptive innovations and replace scores without being limited to anyone's particular personal knowledge and experience, that is, free from any form of subjectivity; (b) To clarify the external environmental factors, a series of tasks can be carried out: identifying and sorting the factors, and unveiling the nature of the impacts associated with the factors and their inherent conceptual connections with technological features or marketplace dynamics; (c) To propose tools for sensitivity analysis aimed at assessing the possibility that an innovation can fulfil its disruptive potential with regard to heterogeneous combinations of external environmental factors, that is, decisions under uncertainties; (d) To measure the technological evolutions of the innovations according to different lifecycle stages and important timings, and compare the disruptive innovations with the incumbents from a techno-market performance perspective, as suggested in the work of Dijk et al. (2015) regarding the market share analysis on transportation.
It is worth noting that the operator Φ is comonotonic subadditive on the lattice Ξ, that is, Appendix B. Design of questionnaire Table A1 The questionnaire.