PartiSim: A multi-methodology framework to support facilitated simulation modelling in healthcare

Discrete event simulation (DES) studies in healthcare are thought to benefit from stakeholder participation during the study lifecycle. This paper reports on a multi-methodology framework, called PartiSim that is intended to support participative simulation studies. PartiSim combines DES, a traditionally hard OR approach, with soft systems methodology (SSM) in order to incorporate stakeholder involvement in the study lifecycle. The framework consists of a number of prescribed activities and outputs as part of the stages involved in the simulation lifecycle, which include study initiation, finding out about the problem, defining a conceptual model, model coding, experimentation and implementation. In PartiSim four of these stages involve facilitated workshops with a group of stakeholders. We explain the organisation of workshops, the key roles assigned to analysts and stakeholders, and how facilitation is embedded in the framework. We discuss our experience of using the framework, provide guidance on when to use it and conclude with future research directions.


Introduction
This paper puts forward a framework to support stakeholder involvement in discrete-event simulation (DES) studies in healthcare. Healthcare simulation often requires the modelling of systems with complex behaviour, involving many stakeholders with plurality of opinions and objectives (Harper and Pitt 2004;Eldabi 2009;Brailsford et al 2009). Stakeholder involvement is considered beneficial for the success of simulation studies (Wilson 1981;Lowery 1994;Jun et al 1999;Fone et al 2003;Eldabi et al 2007;Gunal and Pidd 2005).
Organisations with several decision makers, distributed knowledge and power such as those in healthcare may require the involvement of a client group than a single client. Facilitated discrete-event simulation modelling in healthcare has received attention in two recent studies. Kotiadis et al (2014) propose the use of facilitated modelling to support stakeholder involvement in the initial stages of a simulation study, often referred to as conceptual modelling. Another key study provides empirical evidence and discussion around the extent to which client involvement is possible in the simulation lifecycle (Robinson et al 2014). Robinson et al (2014) conclude that the next step to facilitated simulation is the development of a set of steps for the facilitator to follow.
This study complements existing research efforts by providing a six stage framework that guide the modelling team to include stakeholder participation and facilitation throughout the whole simulation study lifecycle in healthcare. The framework, called PartiSim prescribes a number of activities and corresponding stakeholder-oriented deliverables (outputs) for each stage of the simulation study lifecycle, which enable participative and facilitated DES modelling to take place. However, some technical aspects of the lifecycle such as model coding remain a backroom activity undertaken by the modelling team because of the time involved and technical expertise required for coding. Facilitation in DES is not thought to be diminished by not involving stakeholders in model coding (Robinson et al 2014).
In addition to describing the framework stages we also explain how to think about each stage by declaring the paradigms used. The framework takes a multi-paradigm multi-methodology approach, by interweaving the DES modelling practice with soft systems methodology (SSM). SSM supports the existence of multiple opinions and the development of models that are meaningful to a group of stakeholders. Practitioners, especially novices, interested in undertaking a facilitated modelling approach in DES may find useful the explicit definition of generic activities and outputs as well as declaring the thought process in each stage of PartiSim. Guidance in setting up the team to support the PartiSim process is also provided.
From an academic perspective the framework makes a contribution to the field of discreteevent simulation and facilitated modelling. The prescribed process and the underlying approach provide practical guidance to support a multiparadigm approach to be undertaken, which can be replicated in other simulation studies.
The paper is structured as follows. The following section explores the existing literature relevant to stakeholder involvement in healthcare and then refers to methodological aspects relevant to mixing methodologies that combine softer approaches into the discrete-event simulation lifecycle. The third section explains the framework, including the stages and relevant outputs. A discussion follows with reflections on the usefulness of the framework based on the authors' observations. This section also puts forward practical and methodological considerations and concludes with future research directions. Journal of Operational Research, Volume 244, Issue 2, 16 July 2015, Pages 555-564 3

Approaches to mixing simulation modelling with participative methods in healthcare
Healthcare organisations are generally characterised as organisations with distributed decision making structures. Furthermore, healthcare stakeholders, such as clinicians and managers, tend to hold tacit knowledge of different parts of the system. These individuals also tend to have conflicting interests and perspectives that need to be taken into consideration (Brailsford and Vissers 2010). Failure to involve all these stakeholders in the simulation study lifecycle can often lead to the findings not being implemented (Brailsford and Vissers 2010;Fone et al 2003;Young et al 2009). Some reasons for this are the lack of a common understanding about the problem, conflicting views among stakeholders and/or loss of interest in the study (Eldabi 2009). It is believed that the active participation of stakeholders during the study can alleviate such problems, producing strong ownership of the problem formulation and acceptance of responsibility for actions to be taken (Rosenhead and Mingers 2001).
In an effort to integrate stakeholder participation more rigorously into the DES modelling lifecycle, we turn our attention to methods that inherently involve stakeholders during the modelling process, in what is called facilitated modelling. Facilitated modelling has been described as the process of developing models jointly with a client group, face-to-face, with or without the assistance of computer support (Eden and Radford 1990, Franco and Montibeller 2010). In stakeholder-oriented workshops OR methods, such as strategic options development and analysis (SODA), soft systems methodology (SSM), group model building (GMB), multi-criteria decision analysis (MCDA), etc., are used to build models. These permit a subjective analysis of the problem (e.g. many views incorporated) and the operational researcher engages jointly with the client(s) in the modelling process towards desirable and feasible solutions (Franco and Montibeller 2010). Furthermore, it is possible to deploy a facilitated approach by integrating different OR modelling methods (ibid). Out of these methods, that Franco and Montibeller (2010) describe, Soft Systems Methodology has been used in simulation studies.
Existing studies that combine SSM elements in a DES study are limited and nearly always found in the healthcare context. For example Lehaney and Paul (1994;1996) use SSM to build a simulation model of an out-patient services at Watford General Hospital, to draw system boundaries and to identify system activities. Lehaney et al. (1999) provide an account of a study where a SSM approach combined with simulation was used in an outpatient NHS Published in European Journal of Operational Research, Volume 244, Issue 2, 16 July 2015, Pages 555-564 4 dermatology clinic to address the gaps between the customers' and providers' expectations. Kotiadis and Mingers (2006) consider the benefits and barriers of combining SSM with DES in a community-based intermediate care context. Kotiadis (2007) uses an SSM approach to determine the simulation objectives, which can then be explored in the simulation study.
Similarly, Holm and Dahl (2011), use SSM to inform the development of a simulation model of an emergency department at Akershus University Hospital. SSM is mainly used to structure and understand the problems faced and to identify the factors to be included in the simulation model.
In all the cases referred above, SSM is mainly used in the initial stages of or before starting the simulation modelling process. The only study found so far where the simulation model is developed as part of the SSM process is that by Holm et al (2013). However, the process followed is focussed on the case study and does not provide generic guidelines on how the two methods could be combined for others to follow. A later study by Kotiadis et al (2014) use SSM tools to aid facilitation and provides generic steps for others to follow. However, this study is only focused on the conceptual modelling part of a simulation lifecycle. This paper extends the work to cover the whole lifecycle. Robinson et al (2014) suggest that the next step in facilitated DES is to develop a methodology. In light of the fact that SSM can be combined with DES to aid facilitation , we put forward a multimethodology embedded in a framework, rather than a methodology as suggested by Robinson et al (2014). The framework aims to support the multi-methodology by detailing the combination of methods involved. In other words, the framework provides the practical structure to support the multi-methodology. The aim of the framework is to be informative and useful by striking the right balance between the breadth of the process (number of stages) and the level of detail involved. Before making methodological and philosophical considerations about mixing the two approaches in what we call a framework, we first provide a brief introduction to SSM and DES and their respective stages.

What is SSM?
SSM is an organised learning system that deals with problematic situations where there may not be an obvious problem or solution (Checkland, 1999a). The ultimate aim of SSM is to enable the stakeholder(s) involved in the intervention to learn about the problematic situation and based on that to decide on action that will bring about improvement. Journal of Operational Research, Volume 244, Issue 2, 16 July 2015, Pages 555-564 5 SSM was conceptualised in the seventies and over the years was refined to form the four main activities version in the 1990s consisting of the following stages (Checkland, 1999b):

Published in European
1. Finding out about a problem situation, including culturally/politically; 2. Formulate some relevant purposeful activity models; 3. Debating the situation, using the models, seeking from that debate both: Changes which could improve the situation and are regarded as both desirable and (culturally) feasible, and The accommodations between conflicting interests which will enable action-toimprove to be taken; 4. Taking action in the situation to bring about improvement.
The stages described above form the methodology that can be thought of as the principles of the method. The stages are supported by a set of SSM tools that, as the approach, are generic in nature and can be applied to any context (1999b). In this study we are primarily interested in the methodology and therefore will not explain the tools.

Discrete-event simulation (DES) lifecycle models
DES studies are typically guided by simulation lifecycle models rather than a methodology.
A number of simulation life cycle models have been developed that describe the steps undertaken during a DES simulation study. Most life cycle models in DES take a hard OR approach, where the computer model is normally the key outcome of the study, developed by modeller(s), experts in model coding. Some differences between life cycle models may appear at a first glance but in reality they cover the same stages at a different level of detail: conceptualisation, model coding, obtaining solutions and implementation. For example, Kreutzer (1986) describes the 'life cycle of a simulation project' as a sequence of nine steps. Robinson (2004) puts forward four stages (conceptual modelling, model coding, experimentation and implementation), whilst Pidd (2004) describes simulation modelling as a three stage process starting from problem structuring, and then modelling leading to implementation. For a more detailed view of the different life cycle models, interested readers are referred to a number of sources (Hoover and Perry 1990;Nance 1994;Banks et al. 2001;Sargent 2001;Pidd 2004;Robinson 2004;Law 2007;Balci 2012).

Methodological and philosophical differences and commonalities
In order to explain the multimethodology of SSM and DES behind the PartiSim framework we next briefly look at their differences and commonalities at a methodological and philosophical level. shown to have a variable number of stages, for the purposes of the comparison we take a four stage division in order for it to better map on to SSM's four stages. On reflection from table 1, the stages appear to be similar in terms of the outputs derived, albeit the approach taken differs. This mapping of their respected stages can help us identify how these two methods could complement each other. For example the first stage in SSM could be used to inform the first stage of DES as it provides a clearer instruction as to what should be done to clarify the problem situation. In the second stage, both SSM and DES use formal models to represent the problematic situation, but different approaches are used. The actual coding of the model on the computer does not provide much room for involving the stakeholders as already argued by Robinson et al (2014). However there are still aspects, such as the input of client data, were an SSM approach could inform the simulation model coding stage. Similarly, SSM could inform the next two stages of the DES modelling process, that of obtaining solutions and implementation. Implementation Implement solution, findings or learning in real life setting Take action Take action to improve situation Similar outcome, but different approach taken: − In DES the optimal solution is assumed to be acceptable to the client, whereas SSM accepts uncertainty and the possibility that a second cycle of the study may be required. − In SSM, the client tends to take ownership of action to be taken and the implementation plan due to their involvement throughout the process. ) However a combination of SSM and DES involves linking together parts of methodologies, from different paradigms (Mingers and Brocklesby, 1997). A paradigm is 'a very general set of philosophical assumptions that define the nature of possible research and intervention' (p. 490). Whereas a methodology is 'a structured set of guidelines or activities to assist people in undertaking research or intervention' (Mingers and Brocklesby 1997, p. 490). The problem here is that SSM and DES do not come from the same paradigm which means that they also have to be combined at the paradigm level.
DES modelling is considered a hard OR method, belonging to the hard paradigm, whereas SSM is considered a soft OR methodology, belonging to the soft paradigm. The differences in the two paradigms are best explained using Table 2. The six key characteristics of the hard and soft paradigm (Rosenhead 1999) clearly show how different the thought process is for a methodology allied to the soft paradigm from one allied to the hard paradigm. For example when taking the fourth characteristic into account, people in SSM are considered active subjects involved in decisions, whereas in DES people are conceptualised as passive objects.
All SSM steps involve stakeholders, who take active part in finding a solution and discussion.
In some DES steps, the client may be involved, but mainly as the recipient of study outcomes or provider of information required by the modeller. There are overwhelming data demands, with consequent problems of distortion, data availability and data credibility.
Scientization and depoliticization, there is an assumption of consensus between stakeholders.
People are treated as passive objects.
There is a single decision maker with abstract objectives from which concrete actions can be deduced from implementation through a hierarchical chain of command.
There is an attempt to abolish future uncertainty, pretaking future decisions.
Non-optimizing; seeks alternative solutions which are acceptable on separate dimensions, without tradeoffs.
Reduced data demands, achieved by greater integration of hard and soft data with social judgements.
Simplicity and transparency aimed at clarifying the terms of conflict.
Conceptualises people as active subjects.
Facilitates planning from the bottom up.
Accepts uncertainty, and aims to keep options open for later resolution.
In fact a recent ten year review of published papers up to 2008 on mixing OR methods in practice (Howick and Ackermann, 2011) concluded that only a handful of the papers found involve a multimethodology of DES and SSM. A possible reason for this, among many, is the difficulty in combining the methods at both the philosophical and methodological level. Kotiadis and Mingers (2006) explored the combination of SSM and DES at the philosophical and practical level and noted feelings of unease during the method deployment in their case study. They argued that such barriers to multiparadigm multimethodology can be overcome if authors declare their paradigm position and strategy for deploying the paradigms. Hence, a framework supporting a multimparadigm multimethodology should guide the process and the paradigm selection at each stage.

Overview of the PartiSim framework for DES modelling
The proposed PartiSim framework aims to support the execution of the modelling stages adapted or derived from either method (DES or SSM). Each stage can be carried out by following the hard and/or soft paradigm. The PartiSim framework is outlined in Table 3 and it consists of the following parts: stages (column 1), activities (column 2), deliverables or outputs (column 3) and the paradigm deployed (column 4). The framework's parts include information beyond the stages and paradigm deployment in order to offer sufficient guidance to novice modellers. We next explain the rationale for each part of the framework. PartiSim includes six main stages (1, 2, 3, 4, 5, 6) and five sub-stages (1.a, 2.a, 3.a, 4.a and 5.a) (column 1, Table 3). These stages are based on the DES and SSM stages in Table 1 but adapted and expanded to meet the aims of facilitated DES. Each stage (and sub-stage) has a specific purpose, which is achieved by undertaking the prescribed dedicated activities. Some activities such as those undertaken in stage 1 and mostly in the sub-stages are generic in nature and related mainly to organising the project or liaising with the stakeholder team. They could be used in any type of OR intervention carried out in a facilitated mode. Other activities are adapted or borrowed from one of the two methods (DES or SSM). For example, the activity "Define system & boundaries" (stage 2) is adapted from SSM which involves decomposing the system into the activities that take place in that system. Whereas the activity "Debate desirable and feasible solution space" (stage 5) is an activity derived from DES, where the results of relevant scenarios are presented and debated. The difference from a traditional DES activity is that it is adapted to be carried out in a facilitated environment, where stakeholders can express their preferences and discuss alternatives.
Four of the PartiSim stages (2, 3, 5 and 6) support the modelling team's interaction with the stakeholders in a workshop. During the facilitated workshops the facilitator guides the group of stakeholders through workshop activities. The sub-stages that take place before or after the facilitated workshops support the process carried out during the facilitated workshops.
The activities prescribed as part of each stage (Table 3,    The framework is next explained focusing on the organisation of the participants, the workshops and the main stages involved.

Study organisation: roles and workshop facilitation
A key feature of the PartSim framework is the organisation of facilitated workshops. Prior to the organisation of a workshop the project team who will run the workshop needs to be setup and the relevant roles assigned to those taking part in the intervention and ultimately attending the workshops. We distinguish two teams: the modelling and stakeholder team. The team roles are briefly described in table 4. Assigning different roles can have a positive effect on the group work (Belbin, 2010). Ideas about team roles have been put forward in the literature on group processes (Friend and Hickling 1987;Schein 2006;Vennix, 1996;Roberts 1977). We have adapted the views expressed in the literature to fit the requirements for undertaking a simulation study and to fit with the PartiSim approach. For example, a key role is that of the final decision maker whose involvement can help increase the likelihood of implementation (Mason and Mitroff 1981). PartiSim does not assign a specific role but it is recommended that this person is identified within the key stakeholders' role and invited to the workshops.

Roles of project team: Description of each role: Modelling team
The simulation modeller (model coder) Someone experienced in DES modelling, particularly in coding the model. Is responsible in communicating the viability of transforming the conceptual model into a computer model within the agreed timeframe. The recorder Take notes and generally observes the situation and is on hand to provide the facilitator with assistance in organising the workshop particularly in terms of pre-workshop (e.g. sorting agendas preliminary outputs etc.) and post-workshop activities (e.g. disseminating the output of workshops or chasing up data or information). Recording equipment cannot replace this role if confidential information is discussed. Also if recording equipment is used then this role can safeguard in the event of an unexpected electronic failure. The facilitator A person that leads activities within a workshop with good facilitation skills such as active listening, chart writing, managing group dynamics and power shifts and reaching closure (Franco and Montibeller 2010). A workshop can be led by one or more facilitators whose role is to enable the group to meet their workshop objectives within the available timeframe by guiding the participants in any activities undertaken, keeping the interaction among the participants relevant and at the centre of the room.

Stakeholder team
The project champion This person could be either someone enthusiastic about the study or the initiator of the study (Brailsford et al 2009). He/she serves as a link between the modelling team and the stakeholder team. The project champion will motivate other stakeholders and will liaise with modelling team to organize workshops.

Key stakeholders
People with tacit knowledge of the organisation involved and usually with decision making power in the stakeholder organisation.

Other stakeholders
People with tacit knowledge of the organisation.
The modelling team comprises of the simulation modeller(s), the facilitator and the recorder.
As a group, they manage the process and stakeholders' expectations, but also encourage participation. Although the facilitator role and the computer model coder roles can coincide (the same person); at least one member of the modelling team is needed to record the information during the workshop (the recorder). Hence the modelling team could consist of as few as two individuals. A key role within the modelling team is that of the facilitator.
His/her role is multifaceted and within a workshop can take on the role of: an information seeker, a guiding force, a clarifier, a consolidator of opinions, a peacekeeper, a motivator and a technical advisor with respect to the technical aspects of the simulation model/study. The descriptions of each role (Table 4) provides further guidance regarding the required skills and person specifications for each role.
The stakeholder team will typically include subject matter experts, who have an involvement in the organisation or institution of interest. After an initial stakeholder analysis the different stakeholders and their roles, representing different parts or tiers of the organisation are considered in order to decide who should be invited in the workshops. The main roles considered important to include in the workshops and to involve in the study are: the project champion, key stakeholders and other stakeholders. Involving key stakeholders in the study enables a broader level of ownership of the simulation study and its results within the organization (Robinson 2008). Particularly positive for the study is the identification of the project champion (Brailsford et al. 2009), called gatekeeper in system dynamics group model building literature (Richardson and Andersen 1995). From our experience, the project champion has had a huge impact in the successful completion of the study, not only for providing useful information to the modelling team, but also for promoting the study within the organisation.
When designing the workshops, group size and composition are also considered important (Papamichail et al 2007). Phillips and Phillips (1993) describe that the number of participants taking part can have a major influence on the balance between individuality and group. In fact, the suggestions of group size are different depending on different types of group. For a group that the participants had worked together before, no more than around 6-8 members is recommended (Miller 1956;Belbin 1981). When the participants only have little experience of working together previously, the most effective group size is considered around 12-14 (Phillips and Phillips 1993). Other research suggests that brainstorming in a big group of around 20-25 members is more effective (Nunamaker et al. 1998). This is because each  (Mason and Mitroff 1981).
However, it is generally thought that the effectiveness of a group meeting declines when the group size increases (Grinyer 2000;Ackermann 1996). This is because more and more problems will occur in the decision making task such as difficulties in reaching a common action plan (Shaw et al. 2004).

The PartiSim framework stages
In the PartiSim framework, the study starts with Study Initiation (stage 1), where the modelling team collates preliminary information about the system to be studied and undertakes role analysis to identify a suitable stakeholder team. Obviously, these two activities, the collection of preliminary information and identification of suitable stakeholders can be undertaken simultaneously and can inform each other. Mason and Mitroff (1981) claim that identifying stakeholders is an easy way of generating the prevalent assumptions about a problem situation. Furthermore, Landry et al. (1983) ascertain that identifying stakeholders is important for developing models that have acceptable levels of conceptual and operational validity, which may subsequently lead to a successful model implementation. As explained earlier stakeholders and modellers are allocated a role and if necessary further stakeholders are identified (table 4).
Stages 2 and 3 take the form of facilitated workshops (Workshop 1 and 2), where the modelling team and the stakeholder team convene in a participative environment. More specifically, the aim of Workshop 1 (stage 2) is to define the system to be studied and to gain a shared understanding about the problematic situation in that system. Workshop 2 (stage 3) aims to determine the modelling objectives, model inputs and outputs and to abstract a communicative model focusing on a specific part of the system, which will become the basis of the simulation study. The information collected and the outputs developed are elements of the conceptual model. In these two workshops, principles from Soft OR, such as SSM are used to initiate debate and enable understanding (Checkland 1999b Next, the focus turns to experimentation, which is the topic of the third facilitated workshop (stage 5). In this workshop stakeholders initially explore the computer model to determine if it is valid for its use. . A live demonstration of the model structure is a means of generating confidence in the model and its results (Barber 1977). The focus of this stage lies in identifying relevant scenarios and changes to experimental factors (inputs) and not necessarily choosing the most preferred scenario.
The process followed during this workshop is closer to the interactive experimentation approach in DES (Robinson 2004), where the aim is to develop a general understanding of the model and its key problem areas. After workshop 3 (stage 5.a), the modelling team undertakes further experimentation in order to obtain statistically significant results (Robinson 2004). A report outlining the model results and findings is subsequently prepared and sent to the stakeholders for reflection (stage 5.a).
Stage 6 (Workshop 4) aims to establish an implementation plan consisting of changes to be introduced in the real system as a result of the study. Developing change plans, planning for training and exploring the impact of system implementation can develop favourable attitudes toward the model and the study (Hoover and Perry 1990). This stage builds upon the scenarios identified in Stage 5, moving beyond the simulation model towards identifying an action trail for change. This is achieved in a facilitated workshop environment, where stakeholders are invited to express their views and debate the plans for the future. An important aspect of the implementation workshop is to create awareness of the learning generated throughout the study as this is one of the main benefits of DES studies (Robinson 2004).. The modellers/ facilitator may need to intervene in creating awareness of the learning achieved (Nisbett and Wilson 1977;Robinson 2004;Rouwette et al. 2009), which can in turn help develop actions to address their problematic situation.
Next, the risks and feasibility of each change in a potential scenario are discussed in order to agree on a preferred scenario(s) to be pursued. Roberts-Gray (1985) suggests that supporting the stakeholder team in overcoming barriers to change is a key factor to enabling the success  (Schoemaker 1995), which include also psychological perceptions (Ajzen 1991) that may hinder the stakeholders from taking action. Having agreed on a preferred scenario, an action trail is next determined, with clear actions, including what, by when and who is responsible. The end of the workshop marks the end of the intervention process, however the implementation of change(s) continues beyond Workshop 4.
Some additional ideas incorporated into the framework include: • A gap of 1 to 3 months between workshops is believed to give stakeholders the time to process (both consciously and subconsciously) the results. This is based on the results of a study undertaken by (Dijksterhuis et al. 2006), which found that clients who were left to "sleep on" their decisions made better and more consistent decisions. • Follow-up reports are developed after workshops (especially 1, 2 and 3), because not all those who will be involved in implementing change attend the workshops for reasons such as stakeholder availability, time, group size effectiveness, space. Communication and coordination at all levels of an organization is considered crucial for implementation study success, so that employees understand the reason for change and the implementation strategy (Barber 1977;Größler 2007;Snabe 2007).

Discussion: the PartiSim framework
In this section we provide our observations of using the PartiSim framework, consider when it should be used and put forward issues around the deployment of a multi-paradigm multimethodology. Finally we discuss the future research needed to support the development of PartiSim.

Observations of using PartiSim
We have used the PartiSim framework in two real life studies. We refer to these as the Obesity  and the Colorectal study. Both studies took place in a similar time frame and were conducted by the same modelling team. The same stages were used in both studies. A summary of the key aspects of both studies can be found in table 5. In both studies workshop participants engaged well with the process, interacting with the facilitators and each other in the workshops. Both studies led to consensus about action to be taken as a result of discussions taking place within the workshops. We will put forward some of the stakeholders closing remarks on the modelling approach and process that were made at the end of an implement findings workshop. An extract from the conversation follows: From this extract we can deduce that these stakeholders felt positive about their involvement in the process and the findings. However it is harder to make broader claims about the Published in European Journal of Operational Research, Volume 244, Issue 2, 16 July 2015, Pages 555-564 19 benefits of PartiSim from the stakeholders' point of view with only a few case studies to hand. Indeed for most of our stakeholders this was their first encounter with simulation and indeed OR and therefore no comparisons could be made. Nevertheless Stakeholder B's words of quantification and clarification offer real insight about what PartiSim offers above the use of a single methodology (DES or SSM) and single paradigm (Hard or Soft). It offers the stakeholders the benefits of a quantitative and a qualitative approach. Specifically, it offers the outputs that a traditional DES approach can offer but enables a level of engagement and involvement that is more often found in Soft Systems Methodology. Hence the reflection made by the project champion about good involvement.
We now turn our attention to the modelling team's view of PartiSim interventions when comparing these to past experiences of DES studies in healthcare. The benefits of PartiSim can be summarized as: • Awareness from the outset that we needed to consider the membership of the group. In previous experience the focus was around an individual and their interpretation of the problem and other stakeholders were mostly treated as sources of information. The process gave the stakeholders the opportunity to really consider from the start who to involve knowing that there would be a final workshop that would focus on implementation.
• Validation and verification activities in previous non PartiSim studies were aimed at meeting typically one main person's approval rather than a collective. In PartiSim, the modelling team felt greater pressure in gaining the groups confidence in the model.
However satisfying a group of individuals, which arguably is harder, gave the modelling team more confidence in the model. • Better communication through the medium of workshops rather than meeting up with stakeholders individually. Engaging concurrently with all the stakeholders led to the stakeholders feeling less fearful of the simulation model and its results as everyone met the modelling team at the same time (transparency). We also believe that the multiple workshops enabled the more junior stakeholders to gain confidence with the modelling team as well as with the more senior colleagues within the group and to actively contribute. Conflicts and heated discussions took place in workshops but these were resolved within workshops and led to the creation of common views and consensus by the whole group. Journal of Operational Research, Volume 244, Issue 2, 16 July 2015, Pages 555-564 20 • Workshop participants were involved in the process without being challenged with technical aspects of simulation modelling. • The modelling team had not previously experienced involvement in the decision making discussions over implementation. Our collective past experience was explaining results and making recommendations rather than engaging in detailed discussions. Setting out a final workshop on implementation signalled to the group of stakeholders that there would be a process where both the stakeholder group and modelling team would be involved.

Published in European
Reflecting from our perspective as modellers, we believe that the PartiSim framework, provided structure to the overall process and guidance to the facilitation process as part of the individual workshops. Having individual aims for each stage (table 3)  On the other hand, there are potential drawbacks to using PartiSim over expert mode DES.
The demanding elements to the interventions mentioned were to gain the facilitation skills and to produce reports and tidy up outputs following workshops. In fact, the preparation for a workshop often involved running trial workshops with those in the modelling team acting as stakeholders to trial different forms of communication. Our facilitation skills have

When to use the PartiSim framework?
PartiSim is a DES framework that has been designed to support the modellers' interaction with a group of stakeholders in healthcare interventions. However that is not to say that PartiSim could not be used in other contexts outside of healthcare. The decision to use the PartiSim framework should be initially based on whether a group of stakeholders need/want to be involved in the DES study and whether this group can influence implementation.
Commitment of a group of stakeholders to participate in the workshops at the start of the project must mean that they are willing to participate in four workshops and also respond to any other communication before and after workshops. Similarly, the analysts comprising the modelling team must be willing and able to take up the required roles as described in table 4.
In PartiSim the modelling teams and the stakeholders' team are partners in the process with roles and responsibilities. This is not necessarily easy to achieve without negotiation and flexibility by all parties involved. In healthcare simulation modelling it is often easier to involve nurses and managers than to involve physicians and surgeons in the process (Robinson et al 2014) because of work scheduling constraints. In the interventions described in this paper, each workshop has not exceeded two hours in order to encourage a wide representation of stakeholders. The decision to keep workshops to a minimum duration by having activities and outputs to fit the time frame was the result of several discussions with surgeons and physicians prior to the design of the PartiSim framework. In both interventions referred to in section 4.1 above, the modelling team met with the groups (table 5) at unsociable hours (e.g. 7am) in light of the stakeholders' clinical commitments. Adopting PartiSim in healthcare or any other context requires a conscious effort by the facilitator to keep workshops to an acceptable to the stakeholders' duration and flexibility from the modelling team on the start time and location (e.g. hospital meeting room). Robinson et al (2014) describe facilitation in the simulation lifecycle over two consecutive full days which is a shorter overall timespan but requires the commitment of two full days by the participants.
In addition they propose building a 'throw away' model to fit with the shorter time span. The PartiSim framework is best suited to a longer time span so that the pre-and post-workshop activities can be undertaken. Although PartiSim can support the exploration of 'throw away models' it can also be used to develop models of greater fidelity to the actual system of interest.
A key factor in using PartiSim is being able and prepared to move from one paradigm to another and more specifically from the hard to soft paradigm and vice versa. The interplay  (Schultz and Hatch 1996) best explains the paradigm deployment because the modelling team can move back and forth between paradigms within a stage or from stage to stage allowing cross-fertilisation between paradigms whilst maintaining diversity. The learning gained from applying each activity with a particular paradigm informs the next activity whether a soft or hard paradigm is deployed. This is not necessarily straight forward as most simulation modellers are taught simulation from the expert's mode perspective rather than the facilitative mode (Robinson et al 2014). Typically most management scientists identify with hard OR and find it easier to work within that paradigm. For novice DES modellers this can mean being consumed by the model rather than being focussed on the client interaction and the process (a framework, its stages and outputs). In this paper we have attempted to explain how each paradigm is deployed at each stage to help the user adopt the multi-paradigm framework. However for those individuals or groups identifying with the hard OR paradigm, it may be beneficial to be taught SSM and generally gain an appreciation and understanding of the problem structuring field. Even a simple understanding of the differences as put forward in tables 1 and 2 can aid the understanding of SSM and its adoption. Although there is an amassing of papers combing SSM with DES that help towards this, university teaching of DES still lags behind by not proposing the partnership of the two methodologies. This means that non-academic practitioners, who are arguably less likely to stay abreast with journal papers due to subscription costs but more likely to adopt such practice, remain unaware of it.

The future of PartiSim (future work)
In developing PartiSim we used an action research approach, where the framework is first developed and then specific modelling activities are followed by a phase of testing and reflection. The product described here is a result of the amendments made, which can be further developed and improved by future work.
In the first instance PartiSim needs to be trialled by a modelling team that does not include the original PartiSim founders. Ideally further real case studies should be undertaken and reported. The case studies should not be necessarily limited to healthcare but other contexts, where it is practical and beneficial to involve a group of stakeholders, should be explored.
However, case studies are said to provide anecdotal evidence to support outcomes. Therefore alongside these cases studies we believe that a more systematic analysis of outcomes is needed to explore the effectiveness and usefulness of the facilitation process and approach followed. For example, one outcome, learning is said to occur gradually during the intervention and the subjects themselves may not be aware of it happening as it changes the system of beliefs and attitudes, used to make judgements (Nisbett and Wilson 1977;Ajzen 1991;Rouwette 2011). Further research is currently being undertaken analysing our recorded workshop materials (from the current and subsequent interventions). We are keen to identify whether divergent (e.g. facilitator helps participants think about their objectives and develop creative and feasible solutions) and convergent (e.g. facilitator helps participants' converge best ideas into options and then to action plan) (Kaner 2007;Franco and Montibeller 2010) thinking processes take place and identifying the presence of human emotion and distress (Taket 2002) in the workshops. Nutt (1986), who studied a number of case studies that involved creating change in management strategy, found that case studies that involved stakeholder participation were more successful in terms of implementation compared to the ones that did not involve stakeholders. Involvement and the ability of key stakeholders to influence decision making not only in modelling but in the decision for change, can be a way of motivating and supporting changes through nonmonetary means (Snabe 2007). This is particularly the case for simulation modelling in healthcare (Fone et al 2003;Eldabi et al 2007). This paper puts forward PartiSim a participative and facilitated DES framework with its activities and outputs and our experience of undertaking it in practice to encourage the OR community to consider using or investing time in researching this new mode of practice. PartiSim is the result of a multi-paradigm multi-methodology that requires the modelling team to adopt a paradigm crossing strategy. We identify best with the interplay strategy (Schultz and Hatch 1996) and also relate this to each stage in our framework in order to fully convey the thought process undertaken. This is to encourage simulation modellers that approach simulation in a hard OR (expert) mode to understand what alterations are required for facilitated simulation and specifically in applying the PartiSim framework.