Systematic Mapping Study of Security in Multi-Embedded-Agent Systems

Context: In this paper, we study distributed and decentralized systems in which each part is modeled as an agent in a multi-agent system. Those systems provide more scalable and easier ways to control complex, distributed and interconnected systems of embedded components. We are particularly interested in methods to secure these systems. Objectives: This study aims to identify the main security properties studied, the parts of a multi-agent architecture that are considered most often in security studies and the technical solutions used to secure those systems. Methods: We conducted a systematic mapping study on research works addressing the security of multi-agent systems with embedded agents. We identified which security features were addressed, and their roles in global security architecture. Results: We identified 70 papers published in journals and conferences. We classified the extracted data reporting a tendency to focus on securing the availability of systems under attack by means of trust schemes, sometimes supported by cryptographic primitives. Conclusion: The use of cryptography appears to be limited in decentralized systems. However, solutions should be provided to overcome those limits as other solutions such as trust schemes do not protect the system from the same type of attacks.


I. INTRODUCTION
Thanks to their high scalability, multi-agent systems are increasingly used to coordinate and organize the ever-increasing networks and systems of connected devices. Whether they are wireless sensors or autonomous vehicles, the need for security to make the users confident when using such systems with their personal data and safety is increasing.
In this context, we focus on systems that can be modeled as Multi-Embedded-Agent Systems. Such systems act as multi-agent systems with each agent embedded in a connected device. For example, they can be Wireless Sensor Networks (WSNs), Mobile Area Networks (MANETs) or Vehicular Area Networks (VANETs). We focus on these kinds of systems as they are an interesting solution to The associate editor coordinating the review of this manuscript and approving it for publication was Vivek Kumar Sehgal . decentralized control of connected devices but have specific security needs (detailed in Section II-B). However, we are not interested in multi-agent systems hosted in a single computer that control remote devices, as in the 4.0 Industry, or in systems of mobile agents that can move from one host platform to another.
The attack surface of multi-embedded-agent systems spans from hardware to software vulnerabilities and adds new attack vectors related to their particularity: attacks can also come from corrupted or infiltrated agents taking advantage of the absence of a central authority and coordination to harm the system, hijacking the cooperation process to their own benefit. To use the multi-embedded-agent system model in critical systems such as networks of autonomous vehicles, the academic and industrial communities need to find solutions that cover the whole attack surface. As the system under study is decentralized, so should be the security solution. Otherwise, it would impose constraints (such as having a connection to a distant server) that are impossible to satisfy in the studied system.
The main motivation for our work is to understand the current state-of-the-art in security solutions in multi-agent systems and all similar systems. A quantitative analysis of the current work in this domain will help identify possibly missing parts of a security architecture we aim to propose in future work; the results of our study will also help fellow researchers focus and contribute to less studied aspects of this domain.
Following the guidelines of [1] and [2] on how to conduct systematic mapping studies, we structured the remainder of the paper as follows: section II presents the background of our studies and related works. The research used methodology to lead this search is explained in section III. The results are detailed in section IV. Finally, we conclude and present future research directions in section V.
The data and details of each step of the systematic mapping study process and the complete list of selected papers can be found online [3].

II. BACKGROUND AND RELATED WORK A. MULTI-EMBEDDED-AGENT SYSTEMS
We define multi-embedded-agent systems as a specific subclass of multi-agent systems.
There are many definitions of multi-agent systems because they are used in many application fields [4]- [6]. From the software engineering perspective [6]- [9], a multi-agent system represents a complex system with more than two agents, which collaborate to achieve a global behavior and reach a global result. Each agent has a level of autonomy and achieves its own goal (local result).
Generally, an agent is an intelligent entity such as ''a computer system, located in some environment, which is capable of flexible and autonomous actions in order to meet its design objectives'' [10]. In this context, autonomy relates to several concepts [11]. First, an agent is proactive, so it does not necessarily require intervention from its users or designers to adapt or change its flow of actions regarding its goals. It can deny working with other agents if their goals are not in line with its own. However, as it may also need the cooperation of other agents, it is capable of negotiating [12], convincing or being convinced [13]. Last, it is reactive and can adapt its behavior according to its environment or past experience.
In multi-agent systems and multi-embedded-agent systems, there is generally no central entity coordinating the agents. Consequently, system-level decisions are distributed among agents, thus requiring high levels of autonomy in the decision-making process, from the individual agents.
The main difference between multi-agent systems and multi-embedded-agent systems is that in the latter, the agents are embedded systems. The embedded feature adds constraints such as energy management, safety management, or other issues related to mobility, communications and integrity of the agents in a physical environment [14]. We focus in particular on multi-embedded-agent systems and distinguish them from systems of mobile agents [15] and multi-agent systems as software architectures [16].

B. SECURITY IN MULTI-EMBEDDED-AGENT SYSTEMS
Securing a multi-embedded-agent system means securing an information system by providing confidentiality, integrity and availability [17] to minimize the vulnerability of assets and resources [18] but also securing a heavily networked system that needs authentication, authorization and accounting [18] for each agent relies on communications with other agents to achieve its goals.
However, it also means addressing specific threats to multiagent systems: agents rely on each other to achieve their goals and malicious agents can infiltrate the system to thwart it. As there is no central authority to rely on, the agents need to autonomously distinguish between malicious and trustworthy peers.
Last, since agents are embedded software, they also suffer from hardware vulnerabilities ranging from side-channel attacks to any spoofing, eavesdropping or modification of their communication that are usually performed through wireless media.
In the following, we distinguish between preventive security and security by detection. The first includes cryptography, language-level security, security policies or methodological system development to produce sound and secure systems. The second refers to intrusion detection systems, monitoring or trust models to discover and manage threats at run-time.

C. RELATED WORKS
The survey by [19] provides insight into security and challenges in multi-agent systems but focuses mainly on mobile software agents, which have different challenges from the multi-agent systems of interest in our study, where the ''host'' is not a separated entity.
Reference [20] presents extensive work on attack modeling taxonomy. Their paper focuses on open multi-agent systems of mobile agents but not on the specificities of mobility. However, the reviewed solutions make hypotheses that cannot always be satisfied in multi-embedded-agent systems; e.g., they rely on a security framework such as in JADE [21] that is not designed to include embedded constraints on agents.
Reference [22] provides thorough descriptions of general computer security, multi-agent systems and the application of security principles for multi-agent systems. However, only software multi-agent systems are considered.
In addition, though somewhat interesting, all three studies were published in 2012 and consequently do not cover most of the work conducted in the last decade.
More recent studies, [23]- [25], are also related to our work but each covers only a part of the systems we study.
Reference [23] covers more devices (agents in our case) centric systems but with a focus on hardware and physical sensor/actuator limitations.
Some applications of the works reviewed in [24] are also of interest because the studied systems rely on wireless communication before being connected to the Internet. Furthermore, Internet of Things devices are perfect candidates to create multi-embedded-agent systems as they share some embedded-agent features such as self-configuration or a strong link to their physical environment.
Last, the WSN studied in [25] is also an excellent example of possible application of multi-embedded-agent systems. Except for the base, the sensors fit most of the feature of embedded agents: resource limitations, large-scale deployment, wireless communication, strong link to their environment and even the need to aggregate information can be modeled as cooperation.
Using a more formal approach to literature study in the form of a systematic mapping study [26], our paper intends to review the work done on all systems that could be modeled as multi-embedded-agent systems. Our work focuses on the security properties, the technological solutions and the studied security architecture parts. Other related works such as [22], [25] are either too specific in their applications or too old to satisfy our needs.

III. RESEARCH METHODOLOGY
Reference [1] defines systematic mapping studies as studies that ''are designed to provide a wide overview of a research area, to establish if research evidence exists on a topic and provides an indication of the quantity of the evidence.'' A systematic mapping study is broader than a systematic literature study [26] in its search and data extraction stage and aims to summarize the results. However, the methodology used remains the same as a systematic literature study, so we followed the guidelines provided by [1], [2], [26] to perform our study. An overview of the search process flow we followed is given in Fig. 1.

A. RESEARCH QUESTIONS
The goal of this mapping study is to determine security practices in multi-embedded-agent systems to propose a generic security architecture. We aim to cover as many security needs as necessary with a focus on the least covered needs. This leads to the following research questions (RQs): • RQ1 What are the main security properties studied in multi-embedded-agent systems?
• RQ2 What are the specific technical solutions for securing multi-embedded-agent systems?
• RQ3 Which parts of a global security architecture for multi-embedded-agent systems are studied?

B. SEARCH STRATEGY
As suggested in [27], we describe our search strategy by answering the following questions: • Which? We followed a two-step search strategy: an automated search followed by a backward snowballing once the relevant papers from the automated search were identified. We first gathered all the results of the main venues for our field of research and applied two filtering processes (from inclusion/exclusion criteria and then full-text reading). Then, we applied one iteration of snowball sampling to the references of the included papers.
• Where? We used electronic databases from the four main venues in our research field: [28]- [31].
• What? We aimed to provide an overview of the efforts made to secure multi-embedded-agent systems from a security architecture point of view. We hence derived our search string from those two main topics: ''multiagent system'' and ''embedded agent'' from ''embedded multi-agent system'' and ''authentication,'' ''authorization,'' ''confidentiality'' and ''integrity'' from ''security architecture.'' This resulted in the following search string: ("multi-agent system" OR "embedded multi-agent system" OR "embedded agent") AND ("security architecture" OR "authentication" OR "authorization" OR "confidentiality" OR "integrity") We limited ourselves to eight Boolean operators as it was the limit for one of the search engines we used and remained purposely broad on the terms not to bias the results on a specific part of a security architecture.
• When? The study included works from 2010-01-01 to the date of the search, 2020-08-27. As cybersecurity has evolved substantially in recent decades, we kept only the most recent works.

C. STUDY SELECTION
We applied the following criteria: Inclusion: • Papers that propose a security solution for multiembedded-agent systems; • Papers that propose a security solution for a multi-agent system with no hypothesis on the type of agents (that may as well be embedded); • Papers that propose a security solution for a system that can be modeled as a multi-embedded-agent system (see Section II for examples of such systems). Moreover, we included papers referring to systems not characterized as multi-agent systems by the authors but that we could model as multi-embedded-agent systems. Examples of such systems are as follows: • Robot communities; • Wireless Sensor Networks; • Mobile Ad Hoc Networks; • Vehicular Area Networks; • Some Internet of Things setups; • Some Cyber-physical systems setups. The exclusion criteria were as follows: • Secondary or tertiary studies; • Papers not available in English; • Papers not available in full text; • Papers not subjected to peer reviews. We also excluded papers referring to multi-agent systems as software architectures with all their components running on a single machine with a process per agent. Examples of such systems are as follows: • Cloud-enabled computing (centralized, has no constraints on energy, computation power. . .); • Mobile agents (as they are purely software agents); • Multi-agent systems using the Web (communications are done through web technologies with very few limitations); • Multi-agent systems studied from an automation point of view. We also found a considerable number of papers presenting trust schemes or enhancements of trust schemes for multiagent systems. We only included papers proposing trust schemes (and not an enhancement of one) for the specific case of multi-embedded-agent systems or related cases.
Last, we kept ''borderline'' papers, papers that satisfied almost but not all our inclusion criteria. Our goal was first, to keep them to the full-text reading stage to be sure not to dismiss them too early and second, to add them to the included papers as starting points for the snowball sampling process.

D. DATA EXTRACTION
We extracted the relevant data to our search from the papers using the form presented in Table 1.
For the security property field, we listed which elements of the CIA and AAA models (see below) were taken into account in the studies. To this end, we did not try to deduce more than what the authors were presenting but only checked if keywords or related wording were present in the papers.
AAA model, from network security: • Authentication • Authorization • Accounting/Non-repudiation CIA model, from information security: • Confidentiality • Integrity • Availability Furthermore, we differentiated the confidentiality and integrity of data in transit between agents, referred to as ''communication confidentiality'' and ''communication integrity'') and data at rest, data stores and accessible by specific agents, referred to as ''data confidentiality'' and ''data integrity.''

E. ANALYSIS AND CLASSIFICATION
Except for the application field, the classification criteria are not exclusive. A paper can propose a solution to secure two or more security properties using two or more technical solutions and contributing to two or more parts of the security architecture. This means that the exact numbers shown on the different graphs should be used with care.
As we will explain in Section IV, we determined during the backward snowball sampling that there were many papers that we could qualify as multi-agent due to their characteristics (decentralized systems, autonomous subsystems. . .) but that were not characterized as such by their authors. Therefore, we decided to quantify the impact of those papers in our research. This is why we introduced the field application field in our extraction form.
We also added a threat field to give more context on the analysis of our results on security properties.

F. VALIDITY EVALUATION
Concerning the work in [32], the main threat to validity and especially to reproducibility is the subjectivity of the interpretations of the extracted data. Only the classification concerning the security properties was considered in the preparation stage as we had no hypothetical values range for the other data extraction fields.
This leads us to a second threat, the misclassification of primary studies, that can arise when all the classes cannot be considered at the beginning of the study. To mitigate it, we did several iterations of the classification process to refine our classification scheme each time.
Last, a threat to validity specific to our study is a broader than expected application field of multi-agent system solutions. The results of the snowball sampling will be discussed in more detail in Section IV but we found during the snowball sampling stage numerous papers in several fields of research, such as WSNs or MANETs, using multi-agent solutions or at least with the same features as multi-agent systems without naming them multi-agents. Therefore, for more detailed results on multi-agent systems, those research fields should also be included by using the keywords MANET or WSN in the initial search. The present study was not sized to include them; doing so would have added more than five thousand papers to the initial search results, but some of the works are represented as a result of the snowball sampling search.

IV. RESULTS
As illustrated in Fig. 2, from the 2492 papers obtained in initial search on the four main editor search tools, we selected 31 using inclusion and exclusion criteria and then added 39 from a backward snowball sampling on the included and borderline papers for a total of 70 resulting papers. The detailed dataset including the list of the 70 papers with their corresponding ID can be found online [3] and a list of the selected papers is given in Table 3.
The unexpectedly high number of added papers during snowball sampling for such a study must be put in perspective. First, snowball sampling was performed from the references of the included papers and the borderline papers. Sixteen of the added papers were found from references in borderline papers. Moreover, most of the added documents would not have been found during the initial search because they did not include the multi-agent system keyword but  Table 4).
proposed a system that we could model as a multi-agent system, e.g., a MANET with autonomous nodes.

A. STUDIED SECURITY PROPERTIES (RQ1)
For each selected paper, we identified the security properties targeted by the proposed solution and represented obtained results in Fig. 3. As the solutions did not always target one unique security property, the sum of the numbers on the lines does not correspond to the number of papers. Nevertheless, we can see that most, 55 70 ≈ 79%, of the proposed solutions in the selected papers had the objective of preserving the availability of the system under attack. The second and one-third most studied properties were the integrity of the communications and the authentication, but less than a third of the solutions considered them.
Such a prominent interest in system availability can be explained by one of the specificities of multi-agent systems, namely the need for inter-agent cooperation. Even if every information system requires confidentiality on a certain level, and every distributed system requires preserving integrity of the intra-system communications, multi-agent systems can be particularly vulnerable to malicious systems acting as agents and trying to disrupt their operation. Following this reasoning, we were surprised that authentication was not more studied but we were able to determine an explanation in several papers, including [33]- [35]. Indeed, authentication relies heavily on cryptography and, as we will present in Section IV-B, the use of cryptography in multiembedded-agent systems can be challenging and has limited results.
To better understand our results, we compared the security properties with the threats presented in the papers. The resulting graph is shown in Fig. 4. This graph shows that, even if availability is the most encountered as the main priority, attacks related to communication between agents are studied in half of the cases so communication integrity is more relevant than the results in Fig. 3 show. Details about the attacks are shown in Fig. 5 and Table 6. We classified the attacks according to the used means and the achieved goals. Most of the works studied internal attacks, so attacks from one or more malicious agents. Very few attack models were described but we presumed that the attackers had total control over the network since the communications were done over   Table 6). wireless technologies. Thus, the communications were the first part of the system to be attacked. Even though they could be considered as attacks on the communications, we differentiated denial of service attacks as they do not target specific security properties except availability, whereas attacks on the message content relate to availability and communication integrity. Last, we differentiate malicious and misbehaving agents as the latter implies that the attacker will only abuse the organization, by unnecessarily requiring help or refusing to help other agents for example, while not tampering or intercepting message content. This distinction allows us to understand that even if the attacks are coming from inside, they rarely happen at the organization level. Only 17% of the papers studied this threat.

B. TECHNICAL SOLUTIONS FOR SECURING MULTI-EMBEDDED-AGENT SYSTEMS (RQ2)
As shown in Fig. 6, 37% of the papers use cryptographic schemes to secure their system under attack and 64% of the papers propose the use of trust schemes. These trust schemes allow agents to detect malicious and misbehaving  Table 2). agents and to exclude them from the system. They can be seen as a decentralized intrusion detection system and should not be confused with works aiming at increasing the trust from the user to the computer system: the trust is computed by each agent regarding the other agents. The scarce use of cryptography in securing multi-embedded-agent systems, which is paramount to secure almost any computer systems, may be explained by the fact that, according to [33]- [35], cryptography suffers from two drawbacks when used in this context: (i) it does not protect the system from internal attacks (from malicious agents for example) and (ii) it requires a central third-party entity to manage the cryptographic keys. Last, the features arising from the use of cryptography (e.g., confidentiality, integrity, or authentication) are not specifically needed in multi-embedded-agent systems, so authors may assume that they were addressed earlier in the design of these systems.
As trust schemes are a large domain, we only focused on trust schemes specifically targeting multi-agent systems, but many other works also applies in this context. As shown in Fig. 4 they are essentially used to protect, at least the availability of the system: they aim to exclude any agent not behaving as expected by their peers. This also means that less effective or faulty agents can also be excluded even if they are not malevolent.
In Table 2, we can see that papers 2, 10,17,27,30,33,34,37,38,45,50,51,52,56, and 65 rely on cryptographic primitives to enhance their trust schemes (e.g., for authorization or identification). In a context where an attacker has total control over the communication media, it seems unrealistic to rely on exchanged messages to compute the trust of other agents as any message could have been tampered with. Moreover, non-authenticated agents could also deny their implication in malevolent acts or change their identity to clean their slate.
Last, we can see that a quarter of the solutions rely on new agents deployed specifically, the domain-specific agents, rather than adapting the applicative agents, the agents fulfilling the system tasks, to carry the security solutions. Examples of such domain-specific agents include agents storing a Blockchain to decrease the cost in energy or computation to run a Blockchain for the application, or agents logging the communications to detect intruders, being responsible for a specific task in a new security scheme such as storing keys or  access rules. While the use of specific agents helps decrease the weight of the security solutions on the applicative system, they may be problematic to use in certain conditions as new security agents should be deployed to replace failing ones as long as the system is running. Consequently, specific agents could prove to be more costly than simply deploying more capable applicative agents.

C. STUDIED ARCHITECTURE PARTS (RQ3)
To avoid redundancy with RQ2, we focused on classifying the studied security architecture parts on the multi-agent specificities. We collected the part of the multi-embedded-agent system architecture that was secured in each paper. Similarly as for RQ1, the need for cooperation between agents seems to be the main motivation. In particular, how to choose the right agent for cooperation or to route messages. In this specific case, peer selection and ad hoc routing are mutually exclusive, even if the second one can be seen as a subcategory of the first one. We distinguished them first because of the number of their occurrences and second because, as we can see in Fig. 9, the study of ad hoc routing is mainly done in wireless and mobile ad hoc networks. Those two application fields are the most predominant, but they are not the only FIGURE 8. Number of papers studying each part of a multi-agent architecture (details can be found in Table 5). fields with multi-agent solutions. See Figure 10 and Table 7 for the distribution of application fields in our study.
None of the papers investigated hardware security. This was no surprise as the field of hardware security is comprehensive and not specific to multi-agent systems, wireless networks, sensor networks or mobile area networks. Nonetheless, it should not be forgotten that any software security solution relies on the underlying hardware security, so, to ultimately secure a multi-embedded-agent system, suitable solutions from hardware security works should also be studied.
Overall, we can see that the papers focus on choosing the suitable agents to cooperate with rather than on how they would do so. Similarly as before, this can be explained as giving a choice to the agents to find the most suitable peers to work with is a specificity of multi-agent systems while wireless communication, coordination and access control also exist in other fields.

V. CONCLUSION
This systematic mapping study covered 70 papers selected from 2500 over 4 different editor databases and aimed at identifying and classifying the needs of security in multiembedded-agent systems and the provided solutions to meet those needs.
We discussed the benefits and limitations of the most commonly used solutions, applying trust schemes to distinguish between malevolent and trustworthy agents to cooperate with. That type of solutions protect the system against malicious     Table 7). agents trying to attack its availability, which seems to be the most studied security property in multi-embedded-agent systems. Nonetheless, protecting the confidentiality and integrity of the transmitted information in the system requires the use of cryptographic primitives in a context in which no central authority can distribute certificates to new agents connecting to the system during runtime for example.
Our paper showed that studies on this topic are very limited in the context of multi-embedded-agent systems. Therefore, this could be a challenging and relevant topic for future work on multi-embedded-agent systems security.

APPENDIX A LIST OF INCLUDED PAPERS
See Table 3.

APPENDIX B CHOICES OF APPLICATION FIELDS FOR MULTI-AGENT SOLUTIONS
See Figure 10.

APPENDIX C DETAILS ON THE GRAPHS OF THE FIGURES 3, 5, 8, AND 10
See Tables 4-7.