Keywords

1 Introduction

Conceptual modeling is an established means to perform systems analysis in information systems research [8]. Conceptual models are employed to fulfill a variety of purposes, most importantly to understand the information system under investigation, and facilitate the communication between different types of stakeholders [34]. A process model is a specific type of conceptual model listed among the most frequently used conceptual modeling types [8]. Process models represent business activities, events, and control flow relations that constitute a business process [21]. Process models may also incorporate aspects such as the data that is being processed, the organizational resources that are involved in their execution, and the information systems that are supporting the processes under consideration [26].

A precondition to use process models is that they are properly understood by the people who use them [21]. Streams of research have focused on the identification of factors that influence how users look into and understand process models [3, 11, 21, 25, 27]. However, those studies generally do not take into account that the information that is being sought in a process model may differ depending on the user or the occasion. To illustrate this, let us consider the study on the use of abstract labels in process models (i.e. “A”, “B”) [21]. The authors show that the use of abstract labels may improve the understanding of a model by its users. However, this improvement manifests itself if a user looks at a process model to understand its behavioral dimension, i.e. the ways activities are being sequenced. Our point is that this specific information-seeking goal of the user is only one of the many different aims that people may have when consulting a process model. Clearly, abstract labels will not help to understand any aspects of the activities themselves, i.e. grasp its functional dimension.

We observe that existing research on process model comprehension does not explicitly consider the information-seeking objectives of the model user. This means that it is unclear how universally valid the factors are that have been identified as influential on process model comprehension so far. In addition, we may be missing what matters for other types of information-seeking than what has been investigated to date. Against this background, this paper aims to establish a basis for better understanding what individuals look for when using process models. The various information-seeking objectives are shaped in the form of so-called Business Process Model Comprehension (BPMC) use cases. The use cases are focused on the individual task of making sense of process models, in contrast to, for example, the purposes organizations have for them. The use cases are meant to be generic in the sense that they are independent of the specific notation used, the purpose for which the process model is developed, the business domain that the captured process is situated in, and so on.

We evaluate the relevance and completeness of the proposed use cases through a series of interviews and a focus group session. Overall, these involved 24 participants from 8 organizations. We establish how the different use cases are perceived with respect to their relevance and prevalence, also taking into account the various process roles a user may have. We further examine how easy or difficult it is to carry out the use cases when relying on state-of-the-art technology. Building on these findings, we identify implications for researchers into process model comprehension, for process modelers, and for developers of process modeling tools.

The remainder of this paper is organized as follows. In Sect. 2 we elaborate on the related work, which lead us to defining the use cases. In Sect. 3, we describe how we constructed the BPMC use case list, and introduce the design of our validation approach and the protocol used. In Sect. 4, we describe the implementation of the validation approach, and present the results in three parts: an overall evaluation, an evaluation based on process roles, and an evaluation considering modeling tools. In Sect. 5 we discuss the implications of our work for research and practice before concluding the paper in Sect. 6.

2 Related Work

The primary purpose of process models is to improve communication between stakeholders [28]. To ensure that this communication unfolds smoothly, it is necessary that a process model is correctly understood by its users [25]. Recker et al. define process model comprehension as “the ability of a user to retain domain information from the elements in a process model” [25]. Domain information to be retained from process models fall into four categories: functional (the activities performed), behavioral (sequencing and conditions between the activities), organizational (roles and systems that perform the activities), and informational (data and artifacts produced or manipulated) perspectives [7].

In the BPM field, factors influencing model comprehension are along three axes: content, content representation, and user characteristics [25]. The factors in the content and content presentation categories are, for example, modeling notations [24], symbol visualizations [11, 13], labeling style [21], lay-outing [20], modeling direction [14], and complexity [10]. In the user characteristics category the factors mostly studied are: domain experience, process modeling knowledge, and process modeling experience [11, 21, 24, 27]. In addition, deeper personal factors have been examined, such as cognitive style (spatial vs. verbal) [12], cognitive abstraction ability, and learning style [25]. Some personal factors depend on the context in which the user uses the process model. Among such factors are the intrinsic and extrinsic learning motivation, expectations, and strategy [25, 29].

In contrast to the focus of process model comprehension on the retrieval of domain information, the studies mentioned do not explicitly define the type of information to be retained as a factor of comprehension. In most cases, the information-seeking behavior assumed by the user is not even mentioned. The BPM community mostly measures the comprehension of process models for cases where the user looks into the model to obtain information on the behavioral perspective [18]. Some studies explicate this fact by indicating that the comprehension questions asked are on control-flow aspects, e.g. [13]. Others take it for granted by not explicitly specifying the type of information to be retained, e.g. [21]. However, other perspectives are as important to fulfill the various purposes of using process models [1, 16]. For example, project managers view roles to plan and monitor the project, while auditors investigate the activities and related deliverables to evaluate compliance [6].

In addition to the type of information sought, the number of process models used to retrieve such information is another dimension of process model use. The distinction between the examination of a single process vs. a set of processes is recognized by various studies on process model modularity [28, 33]. Once again, the type of information to be retained from those process models are not explicitly reflected upon in these studies. At this stage, contradictory results seems to have emerged on multi-model comprehension (e.g. [28] vs. [33]). When considered more closely, one of the studies actually covers comprehension questions on only the behavioral perspective of the process models [28], while the other includes questions on the organizational and informational perspectives as well [33]. A reason for the incompatible results obtained in these experiments may be the difference in information-seeking behavior of the users, which is a further motivation for the presented work.

In summary, the type of task that a user performs has an impact on how the model is comprehended. In the BPM field, the effect of task differences are emphasized in terms of purpose, such as improvement and execution [12]. The term “process use” came into use that relates process model use to information-seeking [22]. However, current comprehension studies do not explicitly define how the users look into process models in order to get information on different process perspectives. Our motivation is to support the BPM community to explicitly characterize the various ways process models are read and consider the type of information sought as a factor in their own research endeavors. The set of use cases that may be used for this will be presented next.

3 Research Method

In this section, we first explain how we construct the use cases and present the list of BPMC use cases. We then introduce the design of the interviews and the focus group study we performed to validate the use case list, and lastly present the protocol devised for these studies.

3.1 Construction of the Use Cases

We start out with this section by defining the concept of a BPMC use case as “a case where a users displays a certain type of information-seeking behavior to comprehend a process model or a set of process models”. Note that with this definition, we focus on the level of understanding a process model, in contrast to the superficial reading task that it pre-supposes. Specifically this means for our work, in accordance to the views of Von Foerster [15], that understanding a process model encompasses the notions of intent behind and context of the elements in a process model beyond the mere identification of their presence. In addition, we adopt the concept of information-seeking behavior as defined by Wilson: “the purposive seeking of information as a consequence of a need to satisfy some goal”[35]. The definition of a BPMC use case resembles the concept of repeated process model use as introduced by Nolte et al. [22], which explains how users employ process models to obtain information from these for the objective they wish to accomplish.

There are different perspectives that can be taken when organizing use cases in the context of understanding process models. Notably, use cases can be tied to organizational goals, such as process improvement, redesign, automation, and compliance checking [5, 16]. Depending on the exact organizational goal, different insights should be retrievable from a process model. This underlines how relevant it is for both practice and industry to be aware of the purpose of a particular process model. However, regardless of the specific goal to inspect a process model, a person must make sense of its separate elements and their relations to build up a mental model of that process. This individual perspective, which is tied to the micro-structures within a process model, is where the focus of our use cases is. Clearly, there is a relation between the organizational use of a process model and this individual, sense-making task. Process models need to be understood by individuals on a micro-level to carry out problem-solving tasks on a higher, organizational level. Our motivation to focus on this individual, micro-level of the process model to generate use cases is that these will be relevant for a broad range of organizational purposes, exactly because they are so fundamental in nature.

Based on our view on BPMC use cases, we identified the set of BPMC use cases listed in Table 1. The use cases are organized along two dimensions, which align with our analysis of the literature (Sect. 2): (1) the perspective they are focused on (i.e. behavioral, functional, informational, organizational, and all), and (2) the number of processes examined during the information-seeking action. We identified three aspects for a use case: (1) the “importance” to assess the value of a use case [2, 31]; (2) the “prevalence” to understand how often the use case is encountered; and (3) the “difficulty” of carrying out the use case based on the modeling tool used.

Table 1. List of BPMC use cases
Full size table

3.2 Design of the Interviews and Focus Group Study

We used interviews and focus group study to validate the relevance and completeness of the proposed BPMC use case list. Interviewing is the most prominent qualitative data collection technique [23]. It enables researchers to generate rich data from the individuals’ experiential lives, and “reach beyond the superficial layers of their experience” [30]. We decided to use interviews to collect in-depth data on how professionals perceive the list of use cases in the context of their work. Focus group research, also known as group interview, is seen as a one-to-many version of interviews as a data collection technique [23]. It has an additional benefit of bringing varied opinions together in an interactive environment [32]. We planned to perform a focus group session to complement the data collected through interviews with opinions generated in an interactive setting.

We wanted to examine if a use case is relevant for people using process models in their business settings. Furthermore, we wanted to examine if the BPMC use case list is complete. Process model users with different roles may be displaying diverse information-seeking behaviors on process models. We aimed to capture this diversity by revealing opinions of professionals having different process roles. We used the following four most common process roles in the BPM field: Process Analyst, Process Architect, Process Consultant, and Process Owner [19]. Our main consideration for selecting participants for the interviews and the focus group session was to have all these process roles represented in the study. To take into account cultural variety, we aimed to select participating organizations from diverse sectors and geographical locations.

3.3 Interview and Focus Group Protocol

The protocol we planned to conduct an interview session was as follows. We started an interview by introducing the study and asking the participant to fill out a background survey. Then, for each use case, we presented the use case and followed the folllowing steps. We first asked the participant to score the importance and prevalence of the use case, together with the difficulty of the use case based on the process modeling tool used. For this purpose, the participant used a survey on which these three aspects were scored on a 5-point likert scale. We then asked the participant to elaborate on her scores, and provide examples from her work. Upon the completion of these steps for all use cases, we asked the participant to evaluate the completeness of the list, and provide recommendations for changes. Lastly, we asked the participant to provide improvement ideas for tool features to better support the process model use. The focus group protocol only differed from the interview protocol by allowing discussions among the participants.

We planned to validate the relevance of each use case by the importance and prevalence scores of the participants. If a participant would score the lowest (1 on the 5-point likert scale) for both the importance and the prevalence, it would mean that the use case is irrelevant for that participant. If a use case would be irrelevant for most of the participants, then we would conclude that it is an irrelevant use case. We also planned to evaluate the completeness of the use case list based on the comments of the participants. If the participants would not find the list complete and provide examples of additional use cases, we would conclude that the list is incomplete and update it accordingly. To verify our protocol, we performed two test interviews. The feedback from the test interviews supported that the protocol was clear and led to relevant discussions.

4 Data Analysis and Results

In this section, we first explain how we performed the interviews and the focus group study. Then, we present the analysis results in three sections: (1) an overall evaluation of the perceived importance and prevalence of the use cases, (2) evaluation based on process roles, and (3) evaluation of the perceived difficulty of the use cases considering modeling tools.

4.1 Implementation of Interviews and Focus Group Study

In conformance with our selection criterion for the interviews and the focus group study participants, we selected 17 professionals from seven organizations for the interviews, and seven professionals from one organization for the focus group study. An overview of the participating organizations and the performed interviews/focus group sessions can be seen in Table 2. Seven organizations selected for the interviews operated in different sectors. Five of them are located in the Netherlands (NL), while two are located in Australia (AU). The participants of the interviews from each organization performed different process roles. The participants of the focus group session were the employees of the same consultancy company working with different clients from telecommunication and banking sectors. Overall, we reached a diverse set of participants who work with process models regularly, the median of the number of models read in the last year being 70. Each interview took around one hour, and the focus group session took 1.5 hours.

Table 2. Overview of the participant organizations for the interviews (Organizations No. 1–7) and the focus group study (Organization No. 8)
Full size table

After the interviews and the focus group session, we transcribed and coded the recordings. First, we analyzed the first two interviews, and identified an initial set of codes and related categories. These two interviews were chosen due to their rich content, and the potential to include a high number of codes. All remaining interview and focus group transcripts were traversed and coded based on the initial category list. Whenever a new code was identified, previous transcripts were re-checked to see if this code was applicable.

We present the result of the survey data analysis in box plots in Figs. 1 and 3. The median scores are indicated by the horizontal lines in the boxes, and the boxes represent the lower to upper quartile of all data. Figure 1 displays the perceived importance and prevalence of the use cases, and Fig. 3 depicts the perceived difficulty of the use cases based on the tools used. The values for Cronbach’s alpha confirm the reliability of our instrumentation: it ranges from 0.8 for importance and prevalence, to 0.81 for difficulty. In the sections below, we present the findings from our interviews and focus group analysis together with the survey results.

Fig. 1.
figure 1

Perceived importance (top) and prevalence (bottom) of the use cases (5: very important, and very frequent)

Full size image

4.2 Overall Evaluation

As can be seen in Fig. 1, all of the use cases were perceived to have a high importance (scores over 4), while also being considered as moderately prevalent (scores over 3) by at least one participant. Only use case 15, discovering the relations between multiple processes, was perceived to have both a low importance and low prevalence (median scores of 2). Two use cases on seeking information on a single process model were perceived to be of high importance by most of the participants: understanding an individual process as a whole with a high-level perspective (use case 1), and with its complete details including the tasks, flow, roles, data, and IT systems (use case 2). Thus, use cases integrating multiple perspectives for a single process were important for all process roles.

Two bar charts comparing the perceived importance and prevalence of the use cases grouped by process perspectives can be seen in Fig. 2. The figure indicates that most process model users perceive the behavioral perspective to be highly important and prevalent, which is in line with the literature. However, while existing research focuses mostly on the behavioral perspective (Sect. 2), our results show that there are other process perspectives which are similarly important and prevalent for process model comprehension: the functional and organizational perspectives, as well as multiple perspectives combined (see Figs. 1 and 2).

Multiple processes were used by most of the participants to discover the behavioral relations between them, either by examining how processes follow each other horizontally (use case 16) or checking their hierarchical relations (use case 17). This finding is in line with the comprehension studies on process model modularity (Sect. 2). Moreover, use cases 12 and 13 on organizational perspective (roles and IT systems) were perceived to have a high importance and prevalence. Thus, multiple processes were used to obtain information on a single process perspective rather than multiple perspectives together, most importantly for behavioral and organizational perspectives.

Overall, the participants found the set of use cases complete. Specifically, they did not propose any significant addition. Four participants suggested the consideration of risk and performance as additional process perspectives. Although some process modeling languages and tools provide constructs to represent risk and performance indicator elements, they are not yet as widely used as other process perspectives [4, 9]. Hence, we did not include use cases to investigate how users look into process models to examine risk and performance perspectives. However, they are suitable candidates for an extension of the use case set if risk and performance aspects become popular over time.

Fig. 2.
figure 2

Perceived importance (left) and prevalence (right) median scores for process perspectives

Full size image

4.3 Process Role Based Evaluation

There is a high variability in the scores of the participants as can be observed in the box plots in Fig. 1. This variability, however, decreases if the scores are grouped per process role. This highlights the importance of the role of the model user for understanding the information she is seeking. Table 3 shows the median scores for the perceived importance and prevalence per process role, with a color-coded scale collapsed to three levels to better see the patterns in the Likert scale data [17]. Median scores lower than 3 are colored with light gray indicating low importance and prevalence, while the darkest gray is used for scores higher than 3 showing both high importance and prevalence.

Table 3 and Fig. 2 highlight the differences in process model use per process role. The process analysts mostly used an individual process, and looked at it from a general perspective (use cases 1 and 3). The specific information they wanted to get from a single process was about activities and roles (use cases 6 and 7). The process owners also favored the use cases on a single process, and mostly investigated a single process with a general perspective (use cases 1, 2, and 3). For them, the most important process perspective was the organizational perspective (use cases 7, 8, and 13). They analyzed the roles to “complete things with the lowest amount of hand-overs possible” (Process Owner 3).

The process architects were significantly more interested in the use cases on understanding multiple process perspectives and discovering relations among elements of a single or multiple processes than the other process roles (use cases 1, 2, 10, and 15). While many participants struggled to make sense of use case 15, discovering relations between multiple processes, a process architect stated that: “this use case explains the value I generate in my job”. The process consultant group perceived the highest number of use cases to be of high importance. For some use cases, they emphasized that they are important even though not used very much. For example, process consultants 1, 3, and 8 stated that use cases 2 and 3 come into the scene only “for impact analysis in process improvement initiatives”, and “to solve specific technical issues”.

Two use cases were perceived by most participants to have a low prevalence: use cases 9 and 14. Both are about the informational perspective. The process consultants stated that the informational perspective is highly important, but because of the lack of sufficient data elements in the process models created by their organizations they would not be able to apply these use cases properly. However, this was seen as a “missed opportunity” (Process Consultants 1, 5, and 7), and a cause of integration problems between systems (Process Consultants 7, 8, and 10).

Table 3. Median use case scores for importance (Imp.) and prevalence (Prev.) per process roles, color-coded to three levels
Full size table

4.4 Evaluation Considering Modeling Tools

The scores for the perceived difficulty of carrying out the use cases with tools are depicted in Fig. 3. We categorized the tools used by the participants as modeling tools that natively implement process modeling functionalities (e.g. Aris, Adonis), and generic drawing tools that can be used for any type of modeling (e.g. Visio, Powerpoint).

For each use case involving multiple processes, apart from the last two, the specific information being sought was difficult to establish with a drawing tool (scores lower than 3). This can be expected, since drawing tools have hardly any features for process model analysis. What comes as a surprise is that the participants were content with their drawing tools for almost all use cases on a single process, and two use cases on multiple processes (use case 16 and 17). The participants preferred such tools for their simplicity and flexibility. Practical features such as email support motivated some organizations to move from Aris to Visio, and onwards to Powerpoint (Process Owner 2). The perceived difficulty scores for modeling tools display a high variation. Although such tools were found to possess the functionality to perform most of the use cases, they were considered to be too complex. The participants viewed modeling tools as “not so user friendly” and “odious pieces of software”.

Although, in general, participants were content with their tools, their statements pointed out the need for better features. For example, for use cases 4 and 5, many participants (i.e. Process Architects 1, 2, Analyst 1, and Consultants 3, 4) mentioned the need to distinguish the happy path from alternative paths. Process Analyst 1 specified the need to use “colored activities or small icons” to better see the roles. The findings together with the ideas for new features point out to the need for developing tools that are simple to use and do not have a steep learning curve for different user types.

Fig. 3.
figure 3

Perceived difficulty of the use cases for tool types (5: very easy)

Full size image

Based on our findings, in the next section we discuss the implications of our work for researchers, process model tool developers, and process modelers.

5 Discussion

The evaluation of the BPMC use case list provides numerous implications for researchers, tool developers, and modelers in the BPM field. Our results support the importance of the behavioral perspective of process models in the literature, while pointing out that there are other process perspectives which are similarly important and prevalent for process model comprehension (Fig. 2). In addition, our findings highlight the salient differences between the information-seeking behaviors of process roles (Table 3). In light of these implications, we list the following guidelines for process model comprehension researchers to follow while designing and performing process model comprehension research:

  1. (1)

    Make an explicit choice of the use cases to work on,

  2. (2)

    Consider process perspectives other than the behavioral perspective, most importantly functional, organizational, and an integration of multiple perspectives, and

  3. (3)

    Consider that process roles have different information needs, and accordingly, that they employ diverse use cases.

Our evaluation of the difficulty perception for carrying out the use cases provides implications for process modeling tool developers. The high variation of scores with respect to perceived difficulty, specifically when tools specialized on process modeling are used, indicates that tool features cannot be utilized by the users properly (Fig. 3). The prevalent use of the drawing tools support this implication. Based on these, we suggest the following guidelines for process modeling tool developers to follow while developing new process modeling tool features:

  1. (1)

    Focus on developing tools that are easy to use for different user types, and

  2. (2)

    Check the use cases to see if the planned features match with the needs of the target users.

Lastly, the diversity of the use cases favored by process roles implies for process modelers that it is essential to understand the needs of users and plan modeling efforts accordingly. Based on this implication, we list the following guidelines for process modelers to follow while developing process models:

  1. (1)

    Consider the purposes of process models not only at the organizational level, but also at the personal level to discover which of the use cases are important, and

  2. (2)

    Spend the modeling efforts to cover process information essential for the users.

Some comments of the participants pointed to a number of opportunities to facilitate the information-seeking behavior of process model users by providing visualization techniques. These findings may provide directions to researchers for evaluating the impact of visualization techniques on process model comprehension, and to process modeling tool developers for adding new features. We derived the following suggestions on visualization techniques from the transcripts:

  • control-flow based animation,

  • visualizations to examine alternative paths,

  • navigation among multiple processes,

  • abstraction of information in combination with visualizations,

  • automated creation of different process views, and

  • visualizations for specific process perspectives (e.g. roles or systems).

A limitation of our work is formed by the number of the participants and the participating organizations. By using interviews and focus group study as the validation methodology, we collected in-depth qualitative data from a smaller group (for example, in comparison to surveys). The in-depth data enabled us to reach a deep understanding of the experiences of the professionals, and support the survey results with qualitative results. We designed our research to include participants of different process roles, and participating organizations from different sectors and locations. In this way we aimed to reveal different views on process model use.

6 Conclusions

Our motivation for this study was to capture the different types of information-seeking behavior that can be pursued by users of process models. The analysis of the related literature reveals that the type of information to be retained is not explicitly taken into consideration in process model comprehension research. However, the comprehension of the users may change based on how they use process models. Based on this observation, we constructed a list of BPMC use cases which categorizes the information-seeking behavior of the process model users in terms of process perspectives, and the number of process models under consideration. We performed interviews and focus group study to validate the relevancy and completeness of the use cases. We worked with 24 participants from 8 different organizations. The participants were process analysts, process architects, process consultants, and process owners. Results from the study indicate that users seek information on the organizational and functional perspectives as much as on the behavioral perspective. The combined use of these perspectives is similarly important, which is a completely new insight with respect to current literature. The perception of importance and prevalence of the use cases varies among process roles. Based on these indications, we provide directions to include process perspectives in further process model comprehension research. Additionally, we present a list of guidelines for process model tool developers and process modelers. In future work, the use of performance and risk information in process models is also worth to be investigated further.