Multi-method simulation modelling of circular manufacturing systems for enhanced decision-making

Circular manufacturing systems (CMS) constitute complex value networks comprising a large and diverse set of stakeholders that collaborate to close the loop of products through multiple lifecycles. Complex systems modelling and simulation play a crucial role in providing quantitative and qualitative insights into the behaviour of such systems. In particular, multi-method simulation modelling that combines agent-based, discrete-event, and system dynamics simulation methods is considered more suitable to model and simulate CMS as it allows to capture their complex and dynamic nature. This paper provides a step-by-step approach on how to build a CMS multi-method simulation model in order to assess their economic, environmental, and technical performance for enhanced decision-making. To model and simulate CMS three main elements need to be considered:• A multi-method model architecture where the CMS stakeholders with heterogeneous characteristics are modelled individually as autonomous agents using agent-based, discrete-event, and system dynamics.• An agent environment defined by a Geographic Information System (GIS) to establish connections based on agents’ geographic location.• The product journey resulting from the product's interaction with various CMS stakeholders in the circular value network is traced throughout its multiple lifecycles.


a r t i c l e i n f o
Multi-method simulation modelling of circular manufacturing systems Name and reference of original method; Agent-based; Discrete-event; System dynamics Resource availability; A simulation development platform that combines simulation features of agent-based, discrete-event, and system dynamics modelling approaches (e.g., AnyLogic) is recommended for reproducing the method.

Background
Circular manufacturing systems (CMS) refer to systems that are designed intentionally to close the loop of materials and products through multiple lifecycles [7] . These systems can be viewed as complex adaptive systems composed of an interconnected network of a large and diverse set of stakeholders that collaborate to close the loop of products through multiple lifecycles [8] . Complex systems modelling and simulation is becoming an increasingly popular approach to analyse the behaviour of complex systems as this approach allows to capture non-linear behaviour as well as time and casual dependencies [6] . The major simulation modelling paradigms employed to simulate complex systems are system dynamics (SD), discrete-event (DE), and agent-based (AB) [2] . SD is mostly employed for strategic modelling as this modelling approach operates at a high abstraction level; DE takes a process-centric view of the system and supports medium and medium-low abstraction; whereas AB takes a decentralized and individual-centric approach to the system by describing the system as interacting objects with their own behaviours [2] . Moreover, SD deals with continuous processes, whereas DE and AB work mostly in discrete time, i.e., the system has state changes in discrete points of time.
Multi-method simulation modelling, i.e., a combination of the aforementioned simulation modelling methods, provides a higher level of flexibility for modelling and simulating complex systems as it allows to take into account the complementary capabilities and recognize the limitations of each modelling method [ 4 , 12 ]. Thus, multi-method simulation modelling is considered more suitable to model and simulate CMS as it allows to capture their complex and dynamic nature [ 1 , 8 ]. As different multi-method model architectures can be built by combining the different modelling approaches, the authors presented in Roci et al. [8] a multi-method model architecture to model and simulate CMS. By considering CMS as a complex value network composed of many stakeholders engaged in achieving circularity as a collective outcome, the authors proposed a multi-method model architecture where the different CMS stakeholders are modelled individually as autonomous agents using AB, DE, and SD modelling methods. Thus, this approach allows for characterising and analysing CMS by modelling the system as a collection of autonomous decision-making entities with their own behaviours and relationships. This paper provides a step-by-step approach on how to build a CMS multi-method simulation model using the multi-method model architecture proposed by the authors in Roci et al. [8] . In addition, as a simulation development platform that supports AB, DE, and SD simulation methods is needed to implement and reproduce the CMS multi-method simulation model, this paper provides the detailed procedure to construct the computer program using the case study of a white goods manufacturer.

Method details
An overview of the CMS multi-method simulation model is shown in Fig. 1 and each step required to build this model is explained in detail below. Notice that the CMS multi-method simulation model is not a simple sequential process. As one proceeds with building the CMS simulation model, it is often desirable to combine the steps or go back to a previous step. For instance, while defining the behaviour of a CMS stakeholder-agent (i.e., step 1) it is preferable to set up the agent's interface (i.e., step 2) needed to establish the communication with other stakeholder-agents.
Step 1 -Define the set of CMS stakeholder-agents and their behaviours The first step in building the CMS multi-method simulation model consists in defining the set of CMS stakeholder-agents and their behaviours. The set of CMS stakeholder-agents consists of all relevant value chain stakeholders involved in achieving circularity as an overall outcome (e.g., suppliers, manufacturers, service providers, customers, and value recovery entities). Given the heterogeneous nature of the CMS stakeholders, an AB statechart, a DE process flowchart, an SD stock and flow diagram, or a combination of these modelling constructs are employed to simulate the behaviour of each CMS stakeholder as shown in Fig. 1 . Statecharts are an advanced construct in AB modelling used to simulate event-driven and time-driven agent behaviour [2] . Thus, for stakeholderagents where event-and time-ordering of operations is very pertinent (e.g., customers), one can best characterize their behaviour by employing AB statecharts. Process flowcharts are an advanced construct in DE modelling used to adopt a process-centric view of the system. Thus, for stakeholderagents where manufacturing, logistics, and reprocessing activities are relevant (e.g., manufacturers and service providers), one can best characterize their behaviour by employing DE process flowcharts [3] . Whereas, stock and flow diagrams represent an advanced construct in SD modelling used to simulate continuous processes [10] where stocks represent accumulations (e.g., stock of material, products, money) and flows define how these stocks change over time. Thus, this modelling approach can be employed to capture continuous system behaviour.
Step 2-Define the set of relationships among CMS stakeholder-agents The second step in building the CMS multi-method simulation model consists in defining the relationships and interactions among the CMS stakeholder-agents. Unidirectional or bidirectional connections are employed to define how and with whom stakeholder-agents interact. These interactions among CMS stakeholders comprise exchange processes in terms of physical (e.g., products), information (e.g., order requests), and financial flows (e.g., revenue streams and cost streams) through time along the circular value chain. These connections between agents can be defined in simulation modelling by building the agent's interface [2] , i.e., the set of attributes, methods, ports, or messages that other stakeholder-agents can employ to communicate and interact.
Step 3 -Define the environment the CMS stakeholder-agents live and interact The third step in building the CMS multi-method simulation model consists in defining the simulation environment in which the stakeholder-agents will be placed. Among the different simulation environments available (e.g., discrete space, continuous space, or Geographic Information System (GIS) space), GIS space is more suitable for placing the CMS stakeholder-agents as it allows to consider real-life locations of the CMS stakeholders, thus, retrieve real distances travelled between CMS stakeholders to better estimate transport cost and greenhouse gas emissions. Thus, the different CMS stakeholder-agents are placed on a GIS map depending on their geographical location.
Step 4 -Simulating the product journey through its multiple lifecycles As effective implementation of CMS requires full control of the product throughout all the stages of the value chain, the fourth step in building the CMS multi-method simulation model consists in simulating the product journey through its multiple lifecycles resulting from the product's interaction with the various CMS stakeholder-agent in the circular value network. For this purpose, each product is modelled as an individual agent in continuous communication with the CMS stakeholders throughout its lifetime.
Step 5 -Measure system performance Finally, the last step in building the CMS multi-method simulation model consists in defining the set of variables needed for collecting statistics both at the agent level and at the system level. These statistics are used to assess the CMS performance in terms of economic (e.g., lifecycle costs, lifecycle revenues, and lifecycle profits), environmental (e.g., lifecycle environmental impact), and technical (e.g., quality, quantity and timing of product return) performance.

Method implementation
A simulation development platform that supports AB, DE, and SD simulation methods is needed to implement the CMS multi-method simulation model. In this paper, the multi-method simulation model developed by the authors in Roci et al. [8] to model and simulate the CMS adopted by a white goods manufacturer implementing CMS in practice is used to illustrate the technical steps employed to construct the simulation model. The AnyLogic modelling software, a Java-based simulation development platform that supports AB, DE, and SD modelling methods, is used to implement the multi-method simulation model developed for the white goods manufacturer.
The white goods manufacturer is deploying white-goods-as-a-service for washing machines within the EU funded ReCiPSS project [5] . The washing machines are offered through a subscription-based scheme where customers can choose between Gold, Silver, and Bronze service packages depending on their preferences (e.g., service level, washing machine type, and subscription flexibility). The CMS stakeholders involved in this process of closing the loop of the washing machines through multiple lifecycles include the manufacturer, the service providers, the customers, and the value recovery entities. The manufacturer is responsible for manufacturing long-lasting washing machines. After manufacturing, the washing machines are delivered via forward transport to the service providers responsible for deploying the subscription-based offerings in the market. During the deployment phase, the service providers are responsible for repair and maintenance activities in case of washing machine failure, and perform the recovery operations (e.g., refurbishing) after each end-of-use. The service providers have a pool of technicians to perform the operational activities, e.g., installation, repair and maintenance, and deinstallation of the washing machines. When the washing machines reach their end-of-life, they are delivered to the local recycler for end-of-life treatment. To build the multi-method simulation model of the white goods manufacturer, the first step consists in creating the agents responsible for the system behaviour. Three different agent categories are considered in this model: stakeholder-agents, product-agents, and order-agents. The stakeholderagents represent the set of stakeholders responsible for closing the loop of washing machines through multiple lifecycles. These stakeholder-agents include the Manu f acturer agent, the Ser v iceP rov ider agent, the Customer agent, and the Recycler agent. The product-agents represent the multiple lifecycles washing machines deployed through a subscription-based scheme denoted as P roduct agents, thus simulating the physical flow. Whereas the order-agents represent the agents responsible for establishing the communication between stakeholder-agents, thus simulating the information flow. These order-agents include the P roduct Request agent (i.e., Ser v iceP rov ider -to-Manu f acturer agent communication), the Serv iceRequest agent (i.e., Customer-to-Serv iceP rov ider agent communication), and T reat ment Request agent (i.e., Ser v iceP rov ider -to-Recycler agent communication).
In AnyLogic, one can create a single agent, a population of agents (i.e., a given number of agents of the same type), or an agent type only. In this case, the stakeholder-agents and the product-agents are created as a population of agents. While the stakeholder-agents populations contain a given number of agents, the P roduct agent population is initially empty and it is populated dynamically by the Manu f acturer agent at model runtime when a new washing machine is manufactured. On the other hand, the order-agents (i.e., the P roduct Request agent, the Serv iceRequest agent, and the T reat ment Request agent) are created as agent types only because there are no agents of this type initially. Order-agents are created dynamically at model runtime when a stakeholder-agent places an order on another stakeholder-agent.
An overview of the agents developed to simulate the CMS adopted by the white goods manufacturer and their interrelations are shown in Fig. 2 . DE process flowcharts are used to model the behaviour of the Manu f acturer agent, the Serv iceP rov ider agent, and the Recycler agent as these stakeholders deal with manufacturing, logistics, and reprocessing activities. Whereas AB statecharts are used to model the behaviour of the Customer agent, and the T echnician agent, as for these stakeholders the event-and time-ordering of operations is very pertinent. In addition, SD stock and flows diagrams used to simulate continuous processes, are employed within the Customer agent to simulate the continuous revenue flows and greenhouse gas (GHG) emissions generated by each customer during the subscription period. The technical steps employed to simulate each CMS stakeholder and their interactions are explained in detail in the following sub-sections.

Customer agent
To simulate the behaviour of the Customer agent in a subscription-based scheme, first a set of attributes relevant to characterize the Customer agent is defined. In Java, an attribute is a variable that has its own type (e.g., int, double, String, Boolean) that is used to represent agent characteristics. A detailed list of the attributes employed to simulate the Customer agent is reported in Table 1 . These attributes include a unique identifier number (ID), location, contract duration, monthly subscription fee, and the number of washing cycles per month. Second, the AB statechart shown in Fig. 3 is developed to define the behaviour of a customer in a subscription-based model as a sequence of states. When a new Customer agent is created, it enters the W antT oSubscribe state. Depending on the service package preferred, the Customer agent enters the RequestGold, RequestSilv er, or RequestBronze state. At this stage, an order-agent of type Serv iceRequest is created to model the requests the Customer agent sends to the Ser v iceP rov ider agent. This Ser v iceRequest agent contains two main parameters: the reference to the customer who sends the request, and the request type (e.g., Gold, Silver, or Bronze). These service requests are created dynamically by customers at model runtime by calling the method requestSubscription ( Request T ype t ype ) . A method is a block of code in java to perform a specific task  Fig. 3 ), thus, estimating the subscription duration. Notice that the model time unit is set to be months. Repair . When the Customer agent intends to terminate the subscription, it enters the terminate subscription state (i.e., T erminateGold, T erminateSilv er, or T erminateBronze ) where the method requestServ ice ( Request T ype t ype ) is called again, with the request type being T ermination . Notice that when the method requestServ ice ( Request T ype t ype ) is called in the AB statechart, an order-agent of type Serv iceRequest is created to model the requests the Customer agent sends to the Ser v iceP rov ider agent by indicating the reference to the customer who sends the request, and the request type (e.g., Repair or T ermination ).
In an AB statechart, the model time stays constant during an event execution and is only advanced between events [2] . Thus, the exact duration of the use phase state can be estimated only when the Customer agent exits the state. However, given the continuous nature of subscription-based business models, it is important to access the use phase state on a continuous-time unit basis (i.e., from the moment the Customer agent enters the subscription period until it terminates it). In this way, it is possible to assign user behaviour data (e.g., washing cycles) and extract statistics (e.g., revenues and GHG emissions) on a time unit basis instead of aggregated values based on the total subscription period. To capture this time continuity, an SD modelling approach is employed due to its continuous nature. Thus, the SD stock and flow diagrams shown in Fig. 4 is developed to simulate the continuous revenue flows and GHG emissions generated by a Customer agent during the subscription period. The stock usingGold, usingSilv er, and usingBronze are used to represent the subscription duration in time units (i.e., months) of a Gold, Silver, and Bronze customer respectively. These stocks are linked to the AB statechart shown in Fig. 3 by calling the conditional operators inState ( UseGold ) ? 1 : 0 in the SD f l owGol d element, inState ( UseSil v er ) ? 1 : 0 in the SD f l owSil v er element, and inState ( UseBronze ) ? 1 : 0 in the SD f lowBronze element. This conditional operator inState ( StatechartStat e stat e ) ? 1 : 0 returns one if the Customer agent is in the specified statechart state and zero otherwise. Thus, while the customer is in the use phase state, the value of the stock is incremented by one every time unit. As a result, the revenues generated from each customer-agent every month are estimated given the monthly subscription fee defined by the attributes subscriptionF ee _ Gold, subscriptionF ee _ Silv er, and subscriptionF ee _ Bronze . In addition, the GHG emissions during the use phase state are estimated by taking into account the number of washing cycles a customer uses during its subscription period defined by the attributes washingCycl es _ Gol d, washingCycl es _ Sil v er, and washingCycl es _ Bronze for a Gold, Silver, and Bronze customer respectively and the GHG emissions per washing cycle defined by the attribute unit _ Emissions .
The main attributes, methods, and agent links used to develop the model logic of the Customer agent are reported in Table 1 .

Service provider agent
A discrete-event modelling method is employed to simulate the behaviour of the Ser v iceP rov ider agent as this stakeholder deals mainly with service management activities. The DE process flowchart shown in Fig. 5 is developed to simulate the deployment management of subscription-based offerings from the Ser v iceP rov ider agent by using a sequence of operations performed over entities. The two  As mentioned earlier, both customer requests and products are modelled as agents, (i.e., the Serv iceRequest agent and the P roduct agent). In multi-method simulation modelling, agents become entities that enter the DE process flowchart to be processed [2] . Thus, when a request is received by a customer, it enters the Incoming _ CustomerRequests block of the DE process flowchart shown in Fig. 5 . Depending on the request type, the incoming customer requests are routed to different queues awaiting to be handled in the DE process flowchart.
The Ser v iceP rov ider agent is in communication with the Manu f acturer agent for ordering new washing machines. This process is modelled by creating a new order-agent of type P roduct Request that contains two main parameters: the reference to the service provider that sends the request, and the treatment type (e.g., Manu f acture ). These product requests are created dynamically by the service provider at model runtime by calling the method request Manu f act urer ( T reat ment T ype t ype ) . Note that T reat ment T ype is an attribute of type option list defined for agent attributes with a limited choice of alternative options. The available options for the Ser v iceP rov ider agent are Manu f acture , Repair, Re f urbish , and Recycle . In this model, it is assumed that the service provider performs the repair and refurbish activities itself. However, the model has the capability of requesting these treatment types from another agent. Regarding the recycling of the products, the Ser v iceP rov ider agent is in communication with the Recycler agent. This process is modelled by creating a new order-agent of type T reat ment Request that contains two main parameters: the reference to the service provider that sends the request, and the treatment type (e.g., Recycle ). These treatment requests are created dynamically by the Ser v iceP rov ider agent at model runtime by calling the method requestRecycler ( T reat ment T ype t ype ) .
In addition, the Ser v iceP rov ider agent has a pool of technicians that are reasonable for the installation, repair, and deinstallation of the washing machines. These technicians are modelled as agents with their own behaviour as shown in the next sub-section. The main attributes, methods, and agent links used to develop the model logic of the Ser v iceP rov ider agent are reported in Table 2 .

Technician agent
In the model, all technicians reside at the service provider's location, where they wait for requests to be processed. To define the behaviour of the T echnician agent, the statechart shown in Fig. 6 is developed as a sequence of states. The technician initially waits for requests at the service provider's location. Once the request is received, the technician moves to the customer that placed the request. On reaching the customer, the technician begins the service deployment process, which, depending on the request type, could be installation, repair and maintenance, or deinstallation. After operation completion, the technician moves back to the service provider to fulfil the next request.
The main attributes, methods, and agent links used to develop the model logic of the T echnician agent are reported in Table 3 .

Manufacturer agent
A discrete-event modelling methodology is employed to simulate the behaviour of the Manu f acturer agent as this stakeholder deals mainly with manufacturing and logistics processes. The discrete-event process flowchart shown in Fig. 7 is developed to simulate the manufacturing and delivering process of new washing machines. When a product request is sent from the service provider, it enters the Incoming _ P roduct Request s block awaiting to be handled in the DE process flowchart.
The main attributes, methods, and agent links used to develop the model logic of the Manu f acturer agent are reported in Table 4 .

Recycler agent
A discrete-event modelling methodology is employed to simulate the behaviour of the Recycler agent as this stakeholder deals mainly with reprocessing and logistics processes. The discrete-event process flowchart shown in Fig. 8 is developed to simulate the behaviour of the Recycler agent. When a treatment request and an end-of-life product are sent from the service provider, they enter the Incoming _ T reat ment Request s and Incoming _ P roducts blocks respectively awaiting to be handled in the DE process flowchart.
The main attributes, methods, and agent links used to develop the model logic of the Recycler agent are reported in Table 5 .

Product agent
When the Manu f acturer agent calls the method ad d _ washingMachines () in the manu f actureP roduct block of the DE process flowchart shown in Fig. 7 , a new agent of type P roduct is created and added to the population of washing machines. The agent-based statechart shown in Fig. 9 is developed to define the journey of the washing machine throughout its multiple lifecycles as a result of the product's Built-in AnyLogic function used to send a message to a given agent by indicating the parameters msg, i.e., the message, and dest, i.e., the destination agent.This method is called to send a message to the Cust omer agent after operation completion: • send("Installed", request.customer) when leaving the Inst alla tion state. • send("Repaired", request.customer) when leaving the Repa irAt Cust omer state • send("Deinstalled", request.customer) when leaving the Dein stal lati on state.
In addition, this method is called when the Tech nici an agent is in the Repa irAt Serv iceP rovi der state to request the Serv iceP rovi der agent to send another washing machine while this one is being repaired.if(request.customer.promptService == true)agentLink.send(request,sp);Note that an agent link is added to establish bi-directional communication between the Tech nici an agent and the Serv iceP rovi der agent. sendToWashingMachine (TreatmentRequest treatment, Product wm): void Customized method developed to communicate with the statechart of the Prod uct agent to define its journey. This method is called within the method requ estT reat ment ( Trea tmen tType trea tmen tType , Tech nici an tech nici an ) .
The method is defined as follows: void sendToWashingMachine(TreatmentRequest treatment, Product wm){ wm.treatment = treatment; wm.treatmentType = treatment.type; send(treatment, wm); } requestTreatment(TreatmentType treatmentType, Technician technician): void Customized method developed to communicate with the statechart of the Prod uct agent to define its journey.The method is defined as follows:void requestTreatment(TreatmentType treatmentType, Technician technician) {TreatmentRequest treatment = new TreatmentRequest (this, treatmentType);sendToWashingMachine(treatment, wm);wm.technician = technician;}This method is called on the following occasions: • When the Tech nici an agent is the Repa irAt Serv iceP rovi der state: requestTreatment(Repair, this); • When the Tech nici an agent is in the Dein stal lIat ion state: requestTreatment(Refurbish, this); Bi-directional links with the Serv iceP rovi der agent, Cust omer agent, and Prod uct agent. These links allow agent communications via message passing and inter-agent method calls.    Table 4 Main attributes, methods, and agent links of the Manufacturer agent.

Type
Name Description Attributes country: String Geographical location of the manufacturer indicating the country region: String Geographical location of the manufacturer indicating the region manufacturingCost: double Manufacturing cost of the washing machine manufacturingCO2: double Kg CO2-eq of due to manufacturing of the washing machine transportCost: double Transport cost expressed in €/km/truck used to transport the washing machines to service providers' location. It is assumed that a truck is used for the transportation of the washing machines. The truck transportation policy is full truckload. transportCO2: double Transport kg CO2-eq expressed in kg CO2/tkm due to the transport of the washing machines to service providers' location. It is assumed that a truck is used for the transportation of the washing machines. The truck transportation policy is full truckload.   interaction with the various CMS stakeholders from manufacturing, to distribution, product use phase, value recovery and re-distribution, to end-of-life treatment.
The main attributes, methods, and agent links used to develop the model logic of the P roduct agent are reported in Table 6 .
After defining the behaviour of the CMS stakeholders and their interrelations, the simulation environment in which the stakeholder-agents will be placed is selected. The different stakeholderagents are placed into a geospatial environment defined with a GIS map as shown in Fig. 10 depending on the stakeholders' geographical location. Thus, by considering real-life locations of the stakeholders, it is possible to retrieve real distances travelled between CMS stakeholders to better estimate transport cost and greenhouse gas emissions.
Once the model logic is built, one can run the simulation model to obtain statistics both at the agent level and at the system level using the values of the variables defined in Tables 1-6 . As the CMS multi-method simulation model allows to adopt a lifecycle perspective, the lifecycle costs, lifecycle revenues, and lifecycle environmental impact of deploying multiple-lifecycles washing machines in a subscription-based scheme are estimated. In addition, the quality, quantity and timing of product return flows are monitored over time. Thus, the CMS multi-method simulation model can be used as a decision support tool to design CMS that are feasible in both economic and environmental terms. In Roci et al. [8] the results of the multi-method simulation model developed for the white goods manufacturer are presented.

Method verification and validation
According to Sterman [11] "no model can be verified or validated because all models are wrong". This approach implies that all models are limited, simplified representations of the real world. ( continued on next page )  However, some level of verification and validation is necessary to ensure that the model is an accurate representation of the real-world system and it generates reasonable results. The approaches used for verification and validation of simulation models, in general, can be used to verify and validate the CMS multi-method simulation model. In this paper, the CMS multi-method simulation model has been analysed and validated using the validation techniques presented in Sargent [9] Thus, animation, comparison to other models (e.g., analytical models), degenerate tests and extreme condition tests, and face validity were employed to verify and validate the model. To ensure correct implementation of computer code, tests have been performed during the implementation of the sub-models (e.g., using specific parameter settings expected behavioural outcomes have been verified). Moreover, the AnyLogic platform used to implement the CMS multi-method simulation model allows embedding model animations, hence, checking for correct implementation of the model logic. In addition, the model for the case study has been developed in close collaboration with the white goods manufacturer and the model results have been discussed closely with representatives from the white goods manufacturer and their service providers.

Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.