1 Introduction

“We categorize as we do because we have the brains and bodies we have and because we interact in the world as we do.” (Lakoff and Johnson 1999, p. 18).

The categorization of objects, also known as classification, is a fundamental cognitive process. Thereby, humans classify objects to simplify their understanding of the world and avoid being overwhelmed by the sheer diversity of objects (Lakoff 1987; De Langhe and Fernbach 2019). Taxonomies help humans classify objects according to similarities and differences, and thus enable researchers and practitioners to describe, understand, and analyze phenomena of interest (Nickerson et al. 2013). Taxonomies have a long history in the natural and social sciences (Bailey 1994; Eldredge and Cracraft 1980; Sokal and Sneath 1963) and play a central role in the information systems (IS) discipline, especially given the speed of socio-technical progress that requires continuous efforts of understanding.

To support IS researchers in rigorously designing taxonomies, Nickerson et al. (2013) proposed a method for developing taxonomies in a systematic, transparent and replicable manner. Since then, their method has been widely applied, as evidenced by the fact that since 2013 about two-thirds of the taxonomies published in IS outlets – and analyzed in this study – follow Nickerson et al.’s (2013) method. Examples stem from domains as diverse as mobile health (Varshney 2014), digital finance (Gomber et al. 2017), and virtual assistance (Janssen et al. 2020).

Apart from being artefacts for describing and classifying phenomena, taxonomies have gained attention in IS because they can also serve as a foundation for sense-making (Gregor and Hevner 2013) and theory building (Doty and Glick 1994). According to Gregor's (2006) theory types, taxonomies can be theories for analyzing, which is the most basic form of theory (so-called taxonomic theory, Varshney et al. 2017). Taxonomic theories can be used for the development of other theories (e.g., explanatory, predictive or design theories) by constituting their fundamental constructs and relationships (Gregor 2006; Varshney et al. 2015; Whetten 1989).

Despite the growing interest in taxonomies, as reflected in the increasing number of taxonomies published, IS researchers face two related challenges: Taxonomy building and taxonomy evaluation.Footnote 1 With regard to taxonomy building, we observed that taxonomy articles adopt existing methods inconsistently and often report the taxonomy building in a non-transparent manner. At the same time researchers ask for more guidance on how to implement the individual steps of existing taxonomy development methods (e.g., Mwilu et al. 2015). Regarding evaluation, we found that taxonomies are rarely evaluated and researchers call for further support on how to evaluate them (e.g., Kazan et al. 2018; Krieger and Drews 2018). From these observations, we conclude that taxonomy designers would highly benefit from additional guidance supporting a rigorous design of taxonomies. A need that has also been emphasized by the taxonomy experts whom we interviewed for this study. They found themselves often asking questions such as “what are good practices to build my taxonomy?” or “how shall I evaluate my taxonomy?”. Against this backdrop, we address the following research question: How should taxonomies be built and evaluated?

Based on the systematic analysis of how taxonomies have been built and evaluated in IS research since the publication of Nickerson et al.’s (2013) method, we provide an update for taxonomy designers for methodologically augmenting taxonomy design. This update is anchored in the design science research (DSR) paradigm and takes into account previous efforts to advance taxonomy design in the IS discipline and beyond. As with the original method, also our advanced and extended guidance on taxonomy design is not limited to the IS discipline. Instead, IS-specific peculiarities arise from the phenomena that the IS discipline is typically concerned with. Such phenomena are changing and emerging at an unprecedented speed and scale. Moreover, they are complex and require constant consideration of which (parts of) phenomena are “entirely new and unique, a significant variation […], or just a retread of what we already have” (Nickerson et al. 2013, p. 337). Given that digital transformation often drives the change and the emergence of phenomena, there is a continuous need for coordination between research and practice. For researchers in the IS discipline to pick up on such developments, the identification and structuring of characteristics and dimensions using taxonomies as well as the transparent reporting of their taxonomy design are important.

Our main contribution consists of two parts: First, we advance existing methodological guidance on taxonomy building. Second, we extend this guidance regarding the evaluation of taxonomies. As result, we present an extended taxonomy design process (ETDP) together with 26 operational taxonomy design recommendations (TDR). In sum, we seek to refine and complement the seminal work of Nickerson et al. (2013), offering prescriptive guidance that facilitates a more rigorous building and evaluating of taxonomies.

2 Research Background

2.1 Taxonomy Development in Information Systems and Adjacent Fields

Taxonomies are fundamental in research and practice, which is why taxonomy design has been investigated in various disciplines such as natural sciences (Sokal and Sneath 1963), social sciences (Bailey 1994), organizational science (McKelvey 1975), and strategic management (Hambrick 1984). As a well-accepted method in terms of citations, Bailey (1994) provided a sound overview of and guidance on how to develop taxonomies in social science. Bailey differentiates between the conceptual (i.e., deducing taxonomic structure from a theoretical foundation), the empirical (i.e., grouping inductively via statistical methods), and the operational (i.e., mapping both conceptual and empirical levels) approach. In computer science, Bayona-Oré et al. (2014) proposed a method with 24 activities along five phases (i.e., planning, identification and extraction of information, design and construction, testing and validation, and deployment) to develop taxonomies in software engineering. Thereby, they highlighted demands for clearly formulating a user problem that should be addressed by means of a taxonomy as well as for testing and using a taxonomy to obtain insights concerning its practical usefulness. Usman et al. (2017) refined Bayona-Oré et al. (2014) method and proposed 13 activities ranging from the selection of a classification procedure type to the identification of top-level dimensions and validation through benchmarking. Next to these general activities, other researchers present more specific guidelines for testing taxonomies with conceptual and analytical models (Doty and Glick 1994) or for typical taxonomy building issues (Hambrick 1984). Additional guidance focuses on fields such as requirements-driven taxonomies (Notheisen et al. 2019) or business reporting taxonomies (Ojala et al. 2018). A detailed overview of related guidance on taxonomy design can be found in Appendix 1 (available online via http://link.springer.com).

While other disciplines extensively reflected on taxonomy design, taxonomies in IS research have often been built rather intuitively, for instance by applying methods such as clustering (Posey et al. 2017), qualitative content analysis (Goo et al. 2000), or deductive reasoning (Al-Debei and Avison 2010). Nickerson et al. (2013) provided the first – and so far the only – well-conceived taxonomy development method for the IS discipline. This is important as IS research is concerned with emerging and rapidly evolving technologies, and thus needs a foundation for understanding and analyzing socio-technical phenomena. Nickerson et al. (2013) method comprises seven steps, combining both an inductive and a deductive approach to build taxonomies iteratively. It starts with the determination of a meta-characteristic that is derived from the purpose and target users of the taxonomy. Next, the objective (i.e., taxonomy is valid) and subjective (i.e., taxonomy is useful) ending conditions help to determine when the iterative method can be terminated. Afterward, researchers have to decide on the approach to be adopted for the first/next iteration. If data about real-world objects are available, researchers are advised to follow an inductive approach (i.e., empirical-to-conceptual). If a significant understanding of the phenomenon in focus already exists, a deductive approach (i.e., conceptual-to-empirical) should be employed. The taxonomy development process continues with the next iteration until all ending conditions are met (find the original method in Appendix 1).

Despite being the de-facto standard in the IS discipline, some researchers have adapted Nickerson et al. (2013) method. For instance, Mwilu et al. (2015) argued that the ending conditions are “often difficult to apply in practice” (p. 3) and highlighted the need for advancing steps such as the specification of the object sample. In another example, Land et al. (2013) have extended the original method by introducing a library of case studies (i.e., real-life objects) and a testing phase to validate a taxonomy version using (new) cases and stakeholder feedback. In this vein, Sarkintudu et al. (2018) have provided an extension that seeks to refine a taxonomy based on insights gained during the taxonomy’s usage. In some instances, researchers have explicitly raised the need for specific further guidance regarding taxonomy evaluation (e.g., Kazan et al. 2018; Krieger and Drews 2018), which is apparent in statements such as “the main development cycle [for taxonomies] does not include the evaluation step, leaving it open for researchers” (Chasin et al. 2018, p. 299).

Even though first attempts have started to advance and complement existing methodological guidance, Nickerson et al.’s (2013) method is by far the most referenced method in IS that has been served as a blueprint for numerous of taxonomy projects (see status quo in Sect. 4). Moreover, this method is grounded in the literature on taxonomy design in IS and adjacent fields (see seminal work such as Bailey 1994; Doty and Glick 1994) and serves as a starting point for refined and adapted methodological guidance (e.g., Land et al. 2013; Sarkintudu et al. 2018). Against this backdrop, we have decided to build this study primarily upon the seminal work of Nickerson et al. (2013), while considering additional guidance also beyond IS research to advance and extend the taxonomy design process.

2.2 Taxonomies as Design Science Research Artefacts

Taxonomies help to identify and structure characteristics and dimensions towards describing, understanding, and analyzing phenomena. This applies to both, DSR and non-DSR (such as behavioral science that seeks to develop and test theories, Doty and Glick 1994). This study builds upon previous work by Nickerson and colleagues positioning taxonomies and their design in the DSR paradigm (Nickerson et al. 2013). Given that DSR has the potential to contribute “both practical relevance (via its emphasis on useful artefacts) and scientific rigor (via the formulation of design theories)” (Baskerville et al. 2018, p. 358) this study relies on DSR for the advancement and extension of methodological guidance for taxonomy design.

DSR contributions come in forms as diverse as design artefacts, design theories, and design processes (Baskerville et al. 2018). We see taxonomies as artefacts and consider them – in contrast to real-world phenomena – as artificial objects that solve practical problems (Simon 1996; Hevner et al. 2004) (here identifying and structuring dimensions and characteristics of phenomena). In accordance with the DSR paradigm, this study seeks to provide useful artefacts and not, as common in non-DSR research, to achieve truth or truthlikeness (Hevner and Chatterjee 2010; Goldkuhl 2004) depending on the underlying epistemological and ontological assumptions (Baskerville et al. 2015; Niehaves 2007; Frank 2006).

Generally, four basic artefact types are differentiated in DSR (March and Smith 1995), namely constructs (i.e., concepts and vocabulary of a domain), models (i.e., constructs and their relationship to represent a phenomenon), methods (i.e., processes to perform tasks and achieve goals), and instantiations (i.e., situated implementations that operationalize constructs, models, and methods). Following previous research examining DSR artefacts (e.g., Offermann et al. 2010; Sangupamba et al. 2014), we consider taxonomies as structure-giving artefacts in the form of models for four reasons: First, models “can be viewed simply as a description that is a representation of how things are” (March and Smith 1995, p. 256), which also applies to taxonomies that capture what things are out there (e.g., Iivari 2007). Second, models provide “certain concepts [of a domain] and relationships among them” (March and Smith 1995, p. 256) for understanding phenomena. Taxonomies help researchers to organize knowledge by representing relevant dimensions and corresponding characteristics (i.e., constructs) and thereby reflect relationships among dimensions and characteristics. These relationships are visualized, for instance, through hierarchical structures (e.g., Prat et al. 2015) or multi-layer structures encompassing abstract layers, dimensions, and characteristics (e.g., Janssen et al. 2020). Third, models share three essential properties (borrowed from general model theory as presented by Stachowiak 1973) that are also addressed by taxonomies: Representation – taxonomies represent existing or future, and natural or artificial objects; reduction – taxonomies capture not all attributes of phenomena and instead focus on those attributes that are relevant to a taxonomy’s purpose and target user group; and pragmatism – taxonomies can fulfil various purposes such as describing or analyzing phenomena. Fourth, Nickerson et al. (2013) and other IS taxonomy designers have already positioned their research outcomes as models (e.g., Oberländer et al. 2018; Yang and Varshney 2017).

Positioning taxonomies as models allows taxonomy designers to draw from the rich body of DSR knowledge. As a consequence, the DSR knowledge provides foundations and methodological guidance that help researchers to motivate an artefact’s objective (Peffers et al. 2007) and thereby anchor artefacts in the problem and solution space (vom Brocke et al. 2020). DSR generally differentiates two main activities for building and evaluating artefacts, commonly referred to as the build-evaluate pattern (e.g., Hevner et al. 2004; March and Smith 1995; Sonnenberg and vom Brocke 2012). A need for evaluation arises not only from DSR in general (e.g., Prat et al. 2015; Venable et al. 2016) but also from the application of taxonomies in particular, as taxonomies need to be useful for certain goals such understanding phenomena or making decisions (e.g., De Langhe and Fernbach 2019; Morana et al. 2020). Transferring the build-evaluate pattern to taxonomy design, we observed that further guidance with regards to the evaluation of taxonomies would be helpful, which has been emphasized by Nickerson et al. (2013) who stated that “the resulting taxonomy needs to be evaluated for its usefulness” (p. 346). Rigorously built and evaluated artefacts allow researchers to generalize knowledge (Baskerville and Pries Heje 2019). This is an important aim of DSR projects − including those that intend to design taxonomies – to add design knowledge to the DSR knowledge base (Hevner et al. 2004; Niehaves 2007). Depending on the scope and size, the design of a taxonomy can be a stand-alone DSR project or part of a larger DSR project. Moreover, anchoring taxonomy design under the DSR paradigm allows us to promote the iterative nature of artefact design as well as to organize the design process (ETDP) and the design recommendations (TDR) in a structured way, for example, following the steps of the DSR methodology proposed by Peffers et al. (2007) (see Sect. 5).

3 Research Method

In this study, we aim to answer the research question of how taxonomies should be built and evaluated. To this end, we first assess relevant articles building and/or evaluating a taxonomy. Given the insights (e.g., good practices and challenges) from this status quo analysis, we then advance existing methodological guidance and extend this guidance with regards to taxonomy evaluation.

3.1 Status Quo: Data Collection and Coding

To assess the status quo of taxonomy design in IS, we followed the recommendations of Templier and Paré (2018) in a two-phased approach. In the first phase, we aimed at gaining insights into the operationalization of taxonomy design, presentation and evaluation. Given this aim, we searched and screened relevant literature (see Table 1). In the second phase, we analyzed our sample of identified taxonomy articles and extracted relevant data (using coding attributes such as taxonomy design method, presentation form, evaluation method and criteria). On this empirical basis, we synthesized findings and identified ‘good practices’ as transparent and comprehensible operationalizations of taxonomy design steps. We present a summary of our coding results in Table 2 and methodological details on the status quo analysis in Appendix 2.

Table 1 Identification and analysis of relevant taxonomy articles
Full size table
Table 2 Status quo analysis of taxonomy design
Full size table

3.2 Extended Taxonomy Design Process and Design Recommendations

For deriving the ETDP and the TDR, we particularly drew on examples of transparent and comprehensible taxonomy design, adherence to good practice, and methodological gaps collected during the coding of the sample. Following inductive reasoning (e.g., Hempel 1966), we critically discussed the collected codes (i.e., attributes such as good practices and challenges) to select the most potent attributes in terms of guiding future taxonomy designers. Based on this selection, we advanced and extended the steps of the method proposed by Nickerson et al. (2013) to create the ETDP, formulated TDR, and assigned these recommendations to the steps of the ETDP. The ETDP and the TDR were iteratively (re-)formulated by the author team. Specifically, with regards to further guidance on the evaluation of taxonomies (particularly evaluation goals and ex post evaluation), we built on our sample of taxonomy articles that evaluate taxonomies (56 unique articles) following analytical and abductive reasoning principles (e.g., Gregory and Muntermann 2011; Van de Ven and Johnson 2006). Further, we anchor our guidance in well-established DSR knowledge on artefact evaluation, among others, following the structuring questions of ‘why’, ‘how’, and ‘what’ to evaluate (e.g., March and Smith 1995; Prat et al. 2015; Venable et al. 2016) (see Table 6).

For the evaluation of the ETDP and the TDR, we conducted semi-structured expert interviews (Myers and Newman 2007) that can be used “to confirm what is already known whilst at the same time providing the opportunity for learning” (Recker 2013, p. 91). Our evaluation's goal was to verify the understandability and expected usefulness of the ETDP and the TDR. We first tested our interview procedure and conducted four pre-tests with IS researchers who have already published at least one taxonomy in the conference proceedings of ICIS or ECIS. Once the interview procedure was robust, we started to recruit experts (i.e., authors who have published taxonomy articles in journals listed in the AIS Senior Scholars’ Basket). The participants were provided with the ETDP and the TDR beforehand. Each interview consisted of four parts, i.e., (1) motivation and problem awareness related to taxonomy design, discussions on (2) the ETDP and (3) the TDR, and (4) additional feedback (e.g., assessment of understandability and usefulness). During the interview, one co-author took the role of the interviewer. At least one other co-author was responsible for asking clarifying questions and documenting the interviewee’s responses and questions in the form of a protocol. We analyzed the feedback and revised both the ETDP and the TDR (see Appendix 4 for details on the taxonomy experts and their feedback).

Moreover, to validate our result’s coverage and ensure alignment with the existing body of knowledge, we cross-checked the ETDP and TDR with additional taxonomy design guidance in and beyond IS (see Appendix 1 for a detailed overview of related guidance).

4 Status Quo: Taxonomy Design in IS

The analysis of taxonomy articles revealed two major insights: First, taxonomy building (i.e., research process and product) often remains unclear to readers and, second, a substantial part of taxonomy projects refrains from evaluating the taxonomies. In Table 2, we summarize the data relevant to illustrate our two major insights, while Appendix 2 presents a detailed overview of the descriptive statistics.

Regarding taxonomy building, we found 160 articles in which researchers built a taxonomy. Despite the fact that in 77% of these articles the taxonomies are built with reference to the method of Nickerson et al. (2013), only 22% provide a transparent description of their taxonomy building process.Footnote 2 Even among the articles citing Nickerson et al. (2013), a substantial proportion does not report on elementary design decisions (e.g., no dedicated method section). Such lack of transparency and consistency may complicate the replication of the research process, weaken credibility, limit robustness, and hinder future extensions as well as the taxonomies’ adoption. Further, about two-thirds of the articles present their taxonomy only vaguely.Footnote 3 If a taxonomy is insufficiently presented, it is more difficult to assess its quality, use it for the classification of objects, and extend or revise it in future research.

Regarding taxonomy evaluation, we found that taxonomies are evaluated in around one-third of all analyzed articles (see Table 1). In these articles, we observed ten different types of evaluation methods (see Appendix 2). The most commonly used method (58%) involves the illustration of a scenario with real-world objects (e.g., description and/or classification of real-world objects using a taxonomy) and is followed by the illustration with existing research (15%) and expert interviews (9%). Further, we found 41 different evaluation criteria spearheaded by usefulness (59%), comprehensiveness (47%), and applicability (39%). There are a few predominant evaluation criteria and many with low-frequency use (66% of the evaluation criteria are mentioned in three or fewer taxonomy articles; see Appendix 2).

In summary, our analysis of taxonomy articles reveals that existing methodology for taxonomy building is cited by a large proportion of articles. However, this guidance is adopted inconsistently, the process of taxonomy building often remains unclear (i.e., relevant design decisions in the research process are only partially reported), and taxonomies are hardly evaluated. We conclude that some of these issues may be attributable to operationalization problems, where researchers ask for more guidance on how to implement the individual steps of existing methods (e.g., Mwilu et al. 2015). For instance, Nickerson et al. (2013) provide support for the ex-ante evaluation of taxonomies (i.e., during the development) in the form of objective and subjective ending conditions (Nickerson et al. 2013; Szopinski 2019b). However, the authors do not provide detailed guidance with regards to an ex post evaluation (i.e., after the development) stating that they “are not able [..] to give sufficient conditions other than to say that a taxonomy is useful if others use it” (Nickerson et al. 2013, p. 342). From these insights, we infer a demand for further guidance with regards to the advancement and extension of the existing methodology. More specifically, we argue that an extended taxonomy design process with design recommendations for taxonomy building and ex post evaluation is needed.

5 Results

In response to the outlined demands for further methodological guidance for the design of taxonomies, we developed the ETDP (see Fig. 1) and the TDR (see Table 3). Both, the ETDP and the TDR reflect the results from the systematic analysis of taxonomy articles and include refinements from interviews with experts who provided feedback on the ETDP’s and TDR’s understandability as well as expected usefulness (see Appendix 4 for details on the taxonomy experts and their feedback). Together, the ETDP and the TDR are intended to guide researchers through the entire taxonomy design by stimulating critical reflection and careful design decisions − while building, evaluating, and communicating taxonomies. We organize the ETDP and the TDR along the six activities of the DSR methodology proposed by Peffers et al. (2007). The starting point is the taxonomy development method by Nickerson et al. (2013).

Fig. 1
figure 1

Extended taxonomy design process (ETDP)

Full size image
Table 3 Taxonomy design recommendations (TDR)
Full size table

Like the original method and in line with the DSR paradigm, the ETDP is iterative. As a result, specific entry and exit points for design iterations are provided. This is important because it is not possible to determine a priori when the taxonomy will be completed, i.e. after how many iterations. More specifically, there are three steps after which taxonomy designers can return to previous steps (exit points after Step 12, 14, and 17, see beginning of dashed arrows in Fig. 1). From any of these three steps, researchers can return to one of the following four steps (entry points before Step 2, 4, 6, and 10, see end of dashed arrows in Fig. 1). In this way, the ETDP enables researchers to consciously distribute steps across research cycles (e.g., building and evaluating a taxonomy separately) allowing them to use the ETDP for the evaluation of taxonomies that have not been evaluated yet. This iterative nature of the ETDP enables taxonomy designers to update, extend, and refine existing taxonomies over time without the need to repeat the entire taxonomy design process.

Building upon and complementing the ETDP, 26 TDR provide researchers with operational support and examples of good practice. In other words, the TDR may guide taxonomy designers by encouraging and enabling them to apply the ETDP in a specific research context. This context is shaped by the phenomenon under consideration, the taxonomy’s target user group(s) and its purpose(s).

5.1 Identify Problem and Motivate

Steps 1 to 3 of the ETDP anchor the taxonomy in the DSR problem and solution space (vom Brocke et al. 2020). Thereby, they support explicating the specific problem that is to be solved using the taxonomy to be designed in a given context. This involves specifying the observed phenomenon (Step 1), the taxonomy’s target user group(s) (Step 2), and the intended purpose(s) (Step 3). Justifying the value of the proposed solution (e.g., Hevner et al. 2004; vom Brocke et al. 2020; vom Brocke and Maedche 2019) helps not only to understand why a taxonomy is important for its intended user group(s), but also allows to emphasize the importance of the phenomenon under consideration. Many researchers face the problem that the constructs underlying a phenomenon are either unknown or known but unstructured. This in turn leads to two typical purposes of taxonomies:

Purely structuring purpose: The purpose of the taxonomy is to structure known constructs and their relationships related to the phenomenon under consideration (e.g., established phenomena for which constructs must be synthesized from different contexts or disciplines).

Combined identification and structuring purpose: The purpose of the taxonomy is to identify and structure constructs and their relationships for the phenomenon under consideration (e.g., emerging phenomena for which constructs must be identified for the first time).

There is an (re-)entry point before Step 2, because later demonstration or evaluation of the taxonomy may reveal that it is only be suitable for parts of the originally defined target group(s) (i.e., specification) or even beyond the originally defined target group(s) suitable (i.e., generalization).

TDR 1 to 3 help clarifying how a taxonomy can be a solution to the problem of an unknown/unstructured phenomenon and how such a taxonomy relates to potentially already existing taxonomies.

5.2 Define Objectives of a Solution

Steps 4 and 5 of the ETDP support reflecting upon the taxonomy’s objectives. So far, researchers have followed Nickerson et al. (2013) and determined both the meta-characteristic and the ending conditions. Beyond that, we suggest determining evaluation goal(s) before initiating the taxonomy design to align the problem and the solution space of a taxonomy project at an early stage (e.g., McKay et al. 2012; vom Brocke et al. 2020; vom Brocke and Maedche 2019). This requires not only deciding which angle a taxonomy takes on the phenomenon under consideration (determining meta-characteristic, Step 4), but also when taxonomy building is completed (determining ending conditions) and how to evaluate taxonomies (determining evaluation goal(s), Step 5). This is because the definition of a future solution includes setting the goals not only of the artefact’s building but also of its evaluation (Sonnenberg and vom Brocke 2012). We derived potential evaluation goals from our sample of taxonomy articles following analytical and abductive reasoning principles (e.g., Gregory and Muntermann 2011; Van de Ven and Johnson 2006) (see Sect. 3.2). The evaluation goals comprise, for example, better describing, identifying, classifying, analyzing, and clustering of objects that represent a certain phenomenon compared to doing so without a taxonomy or other classification schemes (see Table 4).

Table 4 Taxonomy evaluation goals
Full size table

In line with defining the problem space in DSR, the meta-characteristic defines what is relevant for the specific taxonomy design and what is not. Consequently, all the following characteristics and dimensions of the taxonomy must relate to this meta-characteristic. A problem-space chosen in this way must always be considered in its environment (Simon 1996). A taxonomy’s environment is shaped by the target user group(s), purpose(s) and evaluation goal(s) which in turn should be aligned to each other right from the beginning of the taxonomy design process. Therefore, we stress the importance of defining the meta-characteristic, bearing in mind that it can be further refined after the first iteration(s), as it is impossible to robustly determine it at this stage – even more so in the case of emerging phenomena. Further, there is no ‘right’ meta-characteristic since phenomena can be viewed from different angles. Rather it is important to explicate the specific angle chosen. To allow refining the meta-characteristic during taxonomy design, the ETDP provides a (re-)entry point before Step 4.

TDR 4 to 6 help determine and refine the meta-characteristic that defines the objective of the taxonomy design as the specific angle from which the phenomenon is perceived (see Fig. 2).

Fig. 2
figure 2

Meta-characteristic as the taxonomy’s angle on the phenomenon under consideration

Full size image

5.3 Design and Development

Steps 6 to 10 of the ETDP support building the taxonomy. We build on the corresponding steps as proposed by Nickerson et al. (2013) and carefully refine a selected part. In this way, we emphasize the idea of combining inductive and deductive approaches for designing taxonomies (Steps 6 to 9). As the creation/revision of taxonomies is often not made explicit, we constitute a new, separate step in which we offer researchers pre-defined taxonomy operations (Step 10). By defining a taxonomy operation as a concrete change made to a taxonomy following an iteration, we thus allow for a taxonomy being altered without the need to re-examine existing or examine new objects (e.g., through renaming or re-structuring characteristics and/or dimensions after the evaluation). Taxonomy operations can change either one or more characteristics and/or one or more (sub-) dimensions. We combine previous work on taxonomy operations (Mwilu et al. 2015) and ‘CRUD functions’ from computer science (Martin 1983) to conceptualize a set of taxonomy operations which includes adding, updating, and deleting characteristics and dimensions (see Table 5). These operations support creating, revising, and documenting taxonomies in a more consistent and transparent way. Since taxonomy design is iterative without knowing a priori the kind and number of iterations as well as whether objects need to be investigated or not, the ETDP provides (re-)entry points before Step 6 and 10.

Table 5 Taxonomy operations
Full size table

TDR 7 to 16 help taxonomy designers to decide on the number of iterations required for taxonomy building, which approach to adopt, and when (i.e., conceptual-to-empirical or empirical-to-conceptual) as well as how to implement the individual iterations for iteratively creating and revising the taxonomy.

5.4 Demonstration

Steps 11 and 12 of the ETDP support demonstrating that a taxonomy is formally valid and satisfies the definition of a taxonomy independently of its purpose(s) and target user group(s). The objective ending conditions proposed by Nickerson et al. (2013) help to demonstrate whether a taxonomy meets the essential criteria for a taxonomy. This involves objectively verifying whether a taxonomy is mutually exclusive and collectively exhaustive. TDR 17 to 19 help demonstrating the validity of the taxonomy.

5.5 Evaluation

Steps 13 and 14 of the ETDP support initiating the evaluation of a taxonomy while considering the taxonomy’s purpose(s) and targeted user group(s). Here, researchers have to evaluate whether the taxonomy is applicable and useful for providing structure to the phenomenon under consideration. The subjective ending conditions proposed by Nickerson et al. (2013) collectively constitute the necessary condition of an ex ante evaluation during the taxonomy building iterations (Nickerson et al. 2013; Szopinski et al. 2019b). This comes down to subjectively verifying that a taxonomy is concise, robust, comprehensive, extendible, and explanatory. Our status quo analysis shows that subjective ending conditions are often reviewed by reaching a consensus among the co-authors of an article as to whether the subjective ending conditions have been met (i.e., whether the taxonomy is perceived applicable). Apart from this being a necessary condition, the rigorous evaluation of a taxonomy requires that it also meets sufficient conditions after the building process has been terminated (i.e., ex post evaluation) (Nickerson et al. 2013; Prat et al. 2015; Sonnenberg and vom Brocke 2012).

Steps 15 and 16 of the ETDP support assessing the sufficient conditions of taxonomy evaluation upon completion of Nickerson et al.’s (2013) taxonomy development method (i.e., that ends/restarts after Step 14) that depends on the taxonomy’s evaluation goal(s), purpose(s) and targeted user group(s). This implies adequately configuring an evaluation (Step 15) and subsequently performing it (Step 16). To observe and measure how well the taxonomy supports target user group(s) in achieving the intended purpose(s) (Step 17), researchers need to configure and perform an ex post evaluation in light of the evaluation goal(s). In DSR, evaluation is critical and should challenge the usefulness of the artefact (Hevner et al. 2004; Prat et al. 2015; Venable et al. 2016), here the usefulness of taxonomies. In addition, the evaluation in DSR depends on the type of artefact which is reflected in the evaluation methods (Cleven et al. 2009; Hevner et al. 2004; Prat et al. 2015) and evaluation criteria (March and Smith 1995; Prat et al. 2015; Sonnenberg and vom Brocke 2012) for a given artefact type. As we consider taxonomies as models, from the DSR perspective, the evaluation of a taxonomy is the evaluation of a model. Furthermore, taxonomies are usually reported together with definitions or descriptions of the taxonomies’ dimensions and characteristics (i.e., constructs of the model). Consequently, the evaluation of taxonomies also involves the evaluation of the constructs. The configuration and performance of taxonomy evaluation require taxonomy-specific answers to the ‘why’, ‘how’, and ‘what’ of DSR evaluation (Prat et al. 2015; Venable et al. 2016). In Table 6 we provide an integrated overview of the steps from demonstration to evaluation along these three guiding questions. For more operational guidance, we describe in Appendix 3 commonly used, taxonomy-related evaluation methods and evaluation criteria. In this way, the ETDP provides an evaluation trajectory starting from ex ante evaluation by checking the necessary (Is it a taxonomy?) and sufficient condition (Is it an applicable taxonomy?) to an ex post evaluation (Is it a useful taxonomy?).

Table 6 Overview of the Extended Taxonomy Design Process' (ETDP) demonstration and evaluation steps along three guiding questions from design science research
Full size table

In summary, taxonomy evaluation involves configuring a triad of evaluation methods, evaluation criteria, and target values/thresholds in light of the taxonomy’s purpose(s). It further includes performing the evaluation with partners relevant to or recruited from the target user group(s) (e.g., Sun and Kantor 2006). This triad allows investigating whether the evaluation goal(s) have been met and whether the actual use of the taxonomy enables the structuring of a phenomenon of interest (i.e., the taxonomy is useful). Bearing the iterative nature of taxonomy design in mind, it is important to note that we have complemented the additional guidance for the ex post evaluation of taxonomies with another exit point (after Step 17) for (re-entering) and reiterating into the taxonomy building before Step 2, 4, 6 or 10. For example, when the ex post evaluation shows that the taxonomy is not useful and does not meet the evaluation goal(s).

TDR 20 to 23 help configuring and performing an evaluation that fits the taxonomy’s target user group(s), purpose(s) and evaluation goal(s).

5.6 Communication

Step 18 of the ETDP supports the reporting of both the process of designing a taxonomy and the resulting design product (i.e., the taxonomy). Furthermore, reporting a taxonomy involves providing visualizations that fit the purpose(s) and target user group(s) (Szopinski et al. 2020) as well as descriptions for each characteristic and dimension. Beside this taxonomy-specific communication, it is also important to consider communication that is specific to the phenomenon under consideration (e.g., Hevner et al. 2004). By doing so, researchers can ensure that their taxonomies serve as a structure-giving artefact for the relevant target user group(s) and towards understanding certain phenomena in domains of interest.

TDR 24 to 26 help communicating the taxonomy and making it accessible to the target user group(s), for example, for practitioners who wish to use the taxonomy for decision-making or for researchers who seek to build their research upon the taxonomy.

6 Discussion and Conclusion

Taxonomies are important structure-giving artefacts that support understanding, describing, and analyzing novel and existing phenomena. Given that taxonomies shape past, present, and future conceptualizations and given that they represent the most fundamental form of theories (Gregor 2006), they should be rigorously designed. However, eight years after the publication of Nickerson et al.’s (2013) widely applied taxonomy method and after analyzing 164 taxonomy articles in the IS context, we observed an inconsistent adoption of existing methods, an often non-transparent reporting of relevant design decisions, and a demand for ex post evaluations. Thus, we infer a need for advancing and extending existing methods and conclude that taxonomy designers would benefit from further guidance supporting a more rigorous design of taxonomies – a motivation that was shared by the taxonomy experts during the evaluative interviews. Against this backdrop, we provide prescriptive knowledge contributing to an update for taxonomy designers in the form of an ETDP and corresponding TDR. Building on these contributions, we highlight three major implications, namely (1) access of taxonomy design to relevant DSR knowledge and methods, (2) guidance for taxonomy evaluation, and (3) evolution of taxonomies across iterations.

First, the integration of the ETDP with Peffers et al. (2007) DSR methodology implies an enriched body of knowledge for taxonomy designers. While we are not the first to relate taxonomies to DSR (e.g., Snow and Reck 2016; Yang and Varshney 2017), our explicit integration of taxonomy design into DSR methodology strengthens the taxonomy designers’ access to and allows for better leveraging DSR knowledge for taxonomy design. This includes, among others, general guidance and good practices to anchor artefacts in the problem and solution space (e.g., vom Brocke et al. 2020), fundamental principles for design-oriented science (Hevner et al. 2004), and reasoning mechanisms to generalize knowledge based on a DSR project (Kuechler and Vaishnavi 2008). In addition to general, cross-phase guidance, there is also phase-specific guidance. For example, to define the problem (e.g., Maedche et al. 2019) as well as design (e.g., Gregory and Muntermann 2014), evaluate (e.g., Peffers et al. 2007; Prat et al. 2015), and communicate the artefact (e.g., Gregor and Hevner 2013).

Second, beyond accessible DSR guidance on evaluating artefacts in general, we provide artefact-specific guidance on evaluating taxonomies responding to Nickerson et al.'s (2013, p. 353) call to examine taxonomies “for [their] usefulness for the intended users and the intended purpose”. Thus, we not only advance, but also extend methodological guidance providing complementary support for the evaluation of taxonomies in the form of detailed demonstration and evaluation steps along the guiding questions ‘why’, ‘how’, and ‘what’ (Prat et al. 2015; Venable et al. 2016). Thereby, formal validity is demonstrated through the use of objective ending conditions. Applicability and, most importantly, usefulness are validated through the application of subjective ending conditions and evaluation goals. Hence, researchers are asked to evaluate a taxonomy’s usefulness as a design artefact (Hevner et al. 2004).

Rigorously building and evaluating artefacts opens avenues for deriving more general design knowledge (Baskerville and Pries Heje 2019). Likewise, such knowledge can be derived based on rigorously designed taxonomies. Taxonomies in the form of models can be classified as nascent design theories (level 2 contribution, Gregor and Hevner 2013). Such taxonomies describe a phenomenon of interest and provide preliminary knowledge on what can be done. In this way, they provide prescriptions for design and action (Gregor et al. 2020). For example, identifying and structuring software features using a taxonomy to prescribe how existing and new software should be (re)designed (see Schöbel et al. 2020 for an example). Taxonomies can also inform well-developed design theories (level 3 contribution, Gregor and Hevner 2013). A prerequisite for such theories is a sound foundation of the constructs and the relationships between these constructs. Taxonomies can assist with this conceptual grounding because the development of design theories is typically preceded by an identification, structuration, and definition of characteristics and dimensions (Goldkuhl 2004).

Third, our ETDP is characterized by an iterative nature and various exit and entry points for revising a taxonomy. Thus, the ETDP supports the advancement of taxonomies, including after an initial publication. This implies a more effective evolution of taxonomies building upon existing and creating new structure-giving knowledge to pass on to future taxonomy design projects (vom Brocke et al. 2020). This could involve researchers designing and publishing an initial version of a taxonomy (e.g., at a conference), with other researchers building upon their work to evaluate, refine, and eventually update that taxonomy. Such cumulative taxonomy design should always start by scanning existing taxonomies and by justifying why another taxonomy is needed. Taxonomies from other disciplines, that often build upon and consolidate one another, could serve as role models in terms of knowledge accumulation and evolution. Examples include biology and astronomy that focus on one major taxonomy of organisms and planets (McKelvey 1982, 1987). Looking ahead, the IS community might also want to further reflect on potential mechanisms to manage the growing landscape of taxonomies, for example, by consolidating standalone taxonomies covering the same phenomenon. In addition, our ETDP allows for early involvement of practitioners, for example, to identify real-world objects or to evaluate the taxonomy. Thereby, our work paves the way for ensuring practical relevance, as demanded by many IS scholars (e.g., Baskerville et al. 2018; Hevner et al. 2004; Moeini et al. 2019).

As with any research, our work is beset with limitations that stimulate future research. First, the ETDP allows for research-specific method configurations, for example, following the ideas of Fazal-Baqaie and Engels (2016). As we do not provide any guidance on when and how to configure the ETDP, future research should examine potential avenues for method configurations. Second, we derived and evaluated our TDR based on the information described in the articles we analyzed as well as from author knowledge and expert interviews. Future research should strive for evaluating the ETDP and the TDR in naturalistic settings, for example, following-up whether and to what extent our work is applied and perceived useful by taxonomy designer. Thus, a study similar to our status quo analysis should be conducted in a couple of years after publication. Third, in the systematic analysis of taxonomy articles, we may have missed relevant articles since we did not cover all potential keywords and sources, but focused only on the most relevant IS journals and conferences as well as on articles citing the original taxonomy development method. However, for validating the coverage of our results and for ensuring alignment with the existing body of knowledge, we performed a cross check with related methods and recommendations on taxonomy design from IS and other disciplines. This check indicates that the ETDP and TDR address additional aspects and needs that have been brought up in related guidance.

As understanding the multitude of new and fast-evolving phenomena is crucial for society and business alike, taxonomies will continue to gain importance. By advancing taxonomy design in IS, we explicitly value taxonomies as a relevant foundation for understanding and further theoretical developments towards theories for explaining or design and action (Bapna et al. 2004; Gregor 2006; Iivari 2007; McKelvey 1987; Posey et al. 2013). We are confident that our methodological update will assist fellow researchers in more rigorously designing taxonomies.