A framework for evaluating security risk in system design

Abstract

Design and development of ubiquitous computer network systems has become increasingly difficult as technology continues to grow. From the introduction of new technologies to the discovery of existing threats, weaknesses, and vulnerabilities there is a constantly changing landscape of potential risks and rewards. The cyber security community, and industry at large, is learning to account for these increasing threats by including protections and mitigations from the beginning of the design V process. However, issues still come from limitations in time for thoroughly exploring a potential design space and the knowledge base required to easily account for potential vulnerabilities in each. To address this problem we propose the G-T-S framework, which is an automated tool that allows a user to provide a set of inputs relating to the desired design space and returns a monetary security risk evaluation of each. This methodology first generates a series of potential designs, then dissects their contents to associate possible vulnerabilities to device elements, and finally evaluates the security risk poised to a central asset of importance. We exemplify the tools, provide methodologies for required background research, and discuss the results in evaluating a series of IoT Home models using the GTS framework. Through implementation of our framework we simplify the information an individual will require to begin the design process, lower the bar for entry to perform evaluating security risk, and present the risk as an easily understood monetary metric.

1 Introduction

Cybersecurity weaknesses and vulnerabilities are constantly being found in the wild due to issues ranging from lack of awareness of potential problems to inadequate security evaluations. This has led to significant efforts to address them in most computing sectors. However, security in large scale systems and the Internet of Things has been a long-neglected area of concern. The larger the system the more difficult to secure because of the multitude of attack vectors, especially at the physical and hardware device level. Risk-related work being done in the field of cyber security is quite extensive. These broadly range from efforts to better understand the business representation and decision models of risk to the formalization of human influence on decision making and scoping of risk. Recent efforts focus on the lens through which one begins to contemplate cyber security [25], while others examine the effect that algorithms have on human decision-making (especially in high-stake contexts) [10]. Further expanding these concerns, researchers have examined how applicable the established concepts of safety, reliability, and resilience are to the quick growing cyber security realm [17]. Other investigations have been performed to clearly outline the need for “decision-making actors” to be security experts due to the complexity of risk decision making [24], and some focus on how the mathematics and politics of risk clash in the realm of cyber security [2]. Work has been done by [38] to establish a meaningful monetary metric behind a risk-based approach for network architecture security modeling and design. While useful for the exploration of contributing parameters for the risk-based approach, this work was nascent and still required formalization of its resources.

A major failure in preventing the exploitation of device weaknesses is that security is still seen as an afterthought, though there is greater effort to tackle this by assessing security at earlier stages within the design process. While this does help alleviate some of the concerns, a great deal of the effort currently has to be done by hand and with reliance on a Subject Matter Expert (a.k.a. “Security Guru”). Due to this requirement, it is far more difficult for a security layman to begin analysis of risk to their systems let alone effectively explore all their options when deciding on high level design choices. For this purpose our research presents a design-to-evaluation framework that can take user-defined attributes for a potential system, generate a series of models, and evaluate their representative risk based on known costs. These costs are associated to the initialization and deployment of the intended design as well as based on research and assessment of potential expenditures for individuals defending or attacking the same design.

1.1 Author’s Contributions

The purpose of the research by P.W and J.C. is to provide an automated solution that can allow anyone to explore the potential security risk of their system architecture as well as explore the larger design space encapsulated by the potential system arrangements. An advantage of our framework is that the user is not required to know detailed information about the modeling language, how to translate that information into an evaluable structure, or how to perform the evaluation of the system. The user can instead leverage the databases and knowledge of the framework to perform repeatable assessment of designs and calculation of security risk. However, users still need to do some research to estimate the associated costs from both the defender and attacker perspectives. To address this problem P.W. and J.C. supply not only the framework but detailed examples of how one goes about estimating these values. This is of particular interest because our research takes an in-depth examination of vulnerabilities with special interest in hardware-based ones. The contributions of our work are as follows:

• A framework for the exploration of a potential deign space through automated generation of models;

• A toolset for evaluating Security Risk posed by each model design using a clear and easy to understand monetary metric; and

• Methodologies for performing the necessary expense assessments from attacker and defender standpoints.

Despite the broad scope of security threat modeling and risk-based decision making, our work is intended to address a more narrow scope of security risk in system design. The landscape of cyber security, and its evaluation of security risk, is incredibly complex due to the variety of contributing elements that a subject matter expert has to account for. To further exacerbate this issue, risk is measured via qualitative or quantitative metrics that are arbitrarily chosen by the originator of the risk framework. These risk metrics also require primer information, or a legend, that allows the users to properly decipher to tool output. Our research has focused on overcoming this limitation by providing an immediately understandable metric (monetary value) as well as an automated framework for producing these assessments. The larger goal being a greater adoption of baking security into designs from the start, rather than attempting to staple on solutions to these concerns after the system design has been created. All authors read and approved the final manuscript.  

1.2 Paper structure

The paper first introduces related work by authors exploring the challenges faced when examining vulnerabilities and their impact, with special attention towards hardware vulnerabilities. Next, we present an overview of the larger framework along with description of the more recent efforts in developing the associated tools. This is followed by a discussion on vulnerability modeling and the challenges faced. After this we present these issues in analyzing vulnerabilities and demonstrate methodologies to estimate the involved costs. We then apply these techniques and asses a collection of IoT Home models originated from the user-defined attributes of the system. Lastly, we discuss our results and the effective successes and limitations of the framework before providing a conclusion of the research effort.

2 Related works

Dealing with security risk can have quite a large scope depending on the interests of the individual performing the assessment. These can range from concerns about how analysis is performed to the structures used to represent these models. Work by [16] shows the importance of formal documentation and semantics when making use of these constructions. While a useful guideline on the importance of clear establishment of rules for any used attack tree model, we chose to adopt the attack tree model accepted by [38, 37], and [35]. Research performed by [36] and [31] both investigated the introduction of additional circuitry elements to embedded devices for the purpose of improving security of electronic devices. The work in [36] focused on the combination of PUF and PRNG models for enhancing security, while the exploration done in [31] investigated the union of PUFs and Biometrics to improve supply chain security as well as implementing low-cost authentication. Efforts by [40] were to incorporate security aspects into earlier design stages, with specific focus on the medical domain since this area is known to have extensive security issues [7]. While this endeavor was useful in determining necessary parameters and aspects for improving security design, it was still too complicated for a layperson to effectively pick-up or use the tool without pertinent security knowledge.

Work is constantly being done trying to create, improve, or provide methods for evaluating and estimating risk of cyber security environments. Tsaregorodtsev et al. [32] produce a method for estimating risk for cloud infrastructure systems. This work does leverage the MITRE CVSS metric and combines this with a Markov process representation, however the work provided is not mature, does not provide an automatic tool, and does not specify a clearly understandable metric for risk. Alfakeeh et al. [1] implemented hesitant fuzzy-sets based decision making to model security risk assessment of a healthcare web application. An advantage of this work is that they leverage the MITRE CWE, however the operation of their model is complicated, produces metrics that cannot be easily understood, and lacks automation for users of the risk assessment model. Kalinin et al.  [14] proposes a neural network based model for cybersecurity risk assessment. This technique allows for effective working with big data and a changing environment. Although this neural network approach is focused on labeling data, it does not produce an easily intelligible risk metric and lacks an automated toolset for users. Gusmão et al. [6] present an examination and comparison of a fault tree analysis and fuzzy decision model for cybersecurity risk analysis. Aside from the limitations presented within the manuscript, the major problem with this model is the implementation of arbitrary quantification for risk contributors and representation of risk.

An additional venue of research being performed is focused on the analysis of existing weakness and vulnerability records to automate their understanding. The purpose of this work being to remove the need for a Subject Matter Expert and allow less experienced individuals to understand and identify these issues. Bandi et al. proposed the use of a linguistic engine to read in the corpus of the MITRE CVE and CWE datasets into their Ontology-Driven Storytelling Framework (OSF) [3]. The goal is to better analyze the vulnerabilities and impacts provided by the National Vulnerability Database (NVD). The OSF can readily identify possible attack impacts and scope as well as related weaknesses for each vulnerability. In summary, this research is still nascent although the authors do show the basic information that the OST produces for “spoofing of log data” as user input, which shows the relevance of CVE and CWE IDs (i.e.,  54% relevancy to CVE-2017-6404 and CWE-276). Huff et al. [11] presents a system to pipeline natural language processing fuzzy matching and machine learning to significantly reduce the human effort for software product vulnerability matching. Their system uses Natural Language Processing (NLP), fuzzy matching, and machine learning to provide useful results based on the ecosystem of software vendors and CVE Naming Authorities. This research is intended to automate the process for matching hardware and software inventory to Common Software Product Enumerators (CPEs) used by standard vulnerability sources (e.g. CVE). Unfortunately, this research is still emerging and only gives some time savings and improved accuracy, however the authors’ intended future work is to use a larger, more realistic dataset.

Another emerging area of cyber security research is the examination of hardware vulnerabilities. While software vulnerabilities are well documented, hardware-based weaknesses and vulnerabilities are not, which has led to improved efforts to record and examine hardware security. Coverage of hardware vulnerabilities in community standards was explored by [39], which reviewed the categorization and documentation of the existing hardware security landscape. This work summarizes the nascent state of this field and pushes for continued efforts of the security community to establish formalized standard language, as has been done in the software realm. Prinetto and Roascio proposed a meaningful and comprehensive taxonomy for hardware vulnerabilities and the attacks that exploit them to compromise the system [23]. In addition to this, the authors also provide a definition of Hardware Security to help clarify a concept often confused within this domain and the corresponding literature. In summary, this research essentially works as reference material for defining terms as well as providing a broad overview taxonomy of Hardware Security. Other research in the realm of hardware vulnerabilities is quite varied due to the relative lack of mature standards and documentation. The authors Bellay et al. [4] examined and evaluated the apprehensiveness of sharing hardware vulnerabilities, further analyzed the current hardware vulnerability reporting efforts, and discussed how to quantify security for hardware. This work examines the existing hardware vulnerability reporting resources and databases, look at the need to have a method for estimating hardware security, and the efforts that are being made to expand this space. Overall, the authors illustrate the point for needing open communication about hardware vulnerabilities as well as the tendency to avoid sharing or disclosure due to the immutability associated with hardware weaknesses and vulnerabilities. While the owners (e.g., manufacturers) of the hardware may be hesitant to divulge vulnerabilities in their hardware, this creates greater strain on the security community by not allowing for wider knowledge of the problem as well as limiting efforts to mitigate it.

As stated in Sect. 1 our research addresses the gaps existing in the space of cyber security and risk evaluation. The framework we present in the following section aims to alleviate these problems by automating the process of exploring the potential design space, overcoming the lack of public databases by outlining repeatable methodologies for determining the necessary expenses for attackers and defenders, and by presenting an easy to understand risk metric in monetary form (i.e., U.S. dollars).

3 Overview of G-T-S framework

We have called our security risk analysis framework the GAM-TAMSAT-SMART (G-T-S) Framework, and it is made up of three separate tools that effectively allow a user to generate, translate, and evaluate Architecture Analysis & Design Language (AADL) models for their respective Security Risk. The three aspects of the G-T-S framework are as follows:

• Generate AADL Models (GAM) is focused on the generation of AADL models based on some preconceived user-defined attributes. The tool will create AADL models that contain the bus, data, device, and system definitions that represent the intended model (e.g., IoT Home).

• Translation of AADL Model to Security Attack Tree (TAMSAT) is focused on the interpretation and translation of AADL model files into a structure that captures the attack profile of the design. This structure, either in XML form or as a Python Graph Object, can then be passed along to the SMART tool for evaluation of the Security Risk.

• Security Model Adversarial Risk-based Tool (SMART) evaluates an AADL model and attack tree representation and produces a security risk metric. The tool requires a series of variables that represent costs and value assessments relating to defender and attacker. It should be noted that determining these values can be a complicated task. The end evaluation of the AADL model’s Security Risk is a monetary value in USD.

The connectivity of the above-mentioned tools is reflected in Fig. 1. The purpose of this framework is to allow a user to explore a design space by evaluating potential designs with respect to Security Risk. The framework does this by first taking some user input which sculpts the potential design space as well as a nascent concept of how the system is implemented. From this input, the GAM tool generates a collection of AADL files that represent example models within the design space. These models are then fed through the TAMSAT tool which identifies potential security vulnerabilities in the designs and generates attack trees that incorporate the design structure and known vulnerabilities. The final step, SMART, evaluates the attack tree structure output from TAMSAT in combination with known vulnerability information in order to produce a Security Risk value in monetary terms. In other words, this framework allows for an individual to explore the conceivable Security Risk presented by various designs using an easy-to-understand monetary value. By providing this capability not only does G-T-S provide a clear methodology for evaluating risk in a quantitative manner but also automates the process such that minimal effort is required by a designer or developer to scrutinize probable models.

Fig. 1

Illustration of flow graph for the G-T-S framework

3.1 GAM

The Generate AADL Model (GAM) tool creates a series of AADL models that represent a set of potential designs that implement a given system using a library of known devices, buses, and data types. The models are generated based on a “database” of known device and system definitions, along with a set of decision logic that determines how those elements are combined and connected. In addition to data types, bus mediums, device, and system definitions, GAM also generates implementation instances that realize the intended design. As shown in Fig. 3, GAM takes in user-defined attributes that provide constraints on the design space explored by the tool. The two aspects of the user-defined attributes and database of known AADL definitions are processed by GAM to generate the intended AADL model instances.

For GAM to operate properly a user must provide key details about the system attributes that are desired in the model. The information required in the user-defined attributes are:

• Devices: Listing of device class elements, type, I/O, and bus specifications for components to include within the design space

• System: Listing of system name, I/O, and bus attributes

• System Architecture: A Python graph representation of the various device class elements and their respective neighbors

• System Edge Devices: List of the device classes within the system architecture map that are expected to be system edge devices

• System Asset of Importance: Single device class item that is intended as the “root” of the later Security Risk evaluation by SMART

Note that this does not require the user to know what the end product should be, but simply be able to identify what the pool of potential devices is. This is assumed to be acceptable since this is commonly required for the design process; one cannot design something without knowing some sense of the end goal. The more complicated piece of information that the user must provide GAM is a “general sense” of how the devices will connect and which elements are planned to interact with the external world. Figure 2 provides this illustration of a system architecture and device classes. Through providing this initial information GAM can then extrapolate the existing design space, explore the potential component deployments, and generate the AADL model files that can be passed to the rest of the framework.

Fig. 2

Illustration of the basic IoT home model

For example in the IoT Home system we discuss in Sect. 5 we have to produce a similar set of information. The device classes we contemplate are a graphical user interface, an application, a server, a database, and at least one sensor. Our expectation is the database will connect to the server, which will connect to both the sensor(s) and application. The graphical user interface will be connected only to the application element. Furthermore, we expect that a customer would desire remote connectivity, and therefore, the application, graphical user interface, and sensor(s) should be able to communicate data over the internet. This information forms the basis of our design space and is the information fed into GAM.

Prior to diving into the GAM tool, we first give an overview of the AADL language and the meaning behind some key syntax.

• Data is used to describe data types and static data in with the AADL model’s text.

• Port represents a communication interface for the directional exchange of data between components. We interpret ports as a type of software-defined connection that can be established between devices. Note that we also assume these port connections will require a communication medium which would include hardware requirements with it.

• Bus is a component that provides access among execution platform components. The GTS framework interprets bus elements as a representation of the physical or hardware connections that exist between components. Regardless of whether the medium is physical (e.g., ethernet) or ethereal (e.g., wireless), there is a hardware-based aspect of the communication medium that GTS exemplifies through bus.

• Device is used to delineate components that interface with and represent the external environment. Our framework adapts this meaning to encompass components that can interface with other devices and the external environment. Additionally, we assume a level of granularity where the device description is treated as a "black box" representation where port and bus elements are used to represent the software and hardware aspects; respectively.

• System is intended as a composite of software, execution platform, or system components. We maintain this idea with the system being the highest level "black box" description of the overall design.

A core language concept in AADL is the description of components through type and implementation declarations. Traditionally AADL sees a type declaration as a definition of the component’s interface elements and externally observable attributes. These attributes include features that are interaction points with other components, flow specifications, and internal property values. The implementation declaration is used to define a component’s internal structure in terms of subcomponents, subcomponent connections, subprogram call sequences, mode, flow implementations, and properties [8]. Our framework leverages these same concepts and uses the system type declaration as a "black box" representation of the general design with the system implementation declaration describing the subcomponents, internal buses, and connection flows of the design. At the end of its operation, the GAM tool produces a set of AADL models that represent an exploration of a design space.

Fig. 3

Illustration of flow graph for GAM

A detailed example of the structures used by GAM to generate each AADL model is shown in Fig. 4. The main input into the GAM tool is the set of user-defined attributes which are the primary shaping force for the generated AADL models. From these attributes, GAM produces a number of design instances represented by a series of mapped structures: System I/O Map, Device I/O Map, and System Implementation Map. The System I/O Map contains a JSON mapping of any intended system models along with the high-level representation of the system’s inputs and outputs. The I/O list in this mapping is similar in purpose as the device I/O map, except that in this scope the intention is to describe the “black box” inputs and outputs for the system description. The Device I/O Map is a JSON object that encompasses the devices within the model along with each device’s I/O list and their bus connections. The I/O list contains the communication protocols for each input/output port on the device. The bus connections represent the communication mediums that each device uses to communicate the aforementioned protocols. Essentially, these communication mediums can be interpreted as depicting the hardware requirements of the device. Finally, the System Implementation Map embodies the detailed internals of the system model, which is portrayed by the system implementation’s subcomponent elements and the connections. The subcomponents array contains the list of intended components that will make up the larger system. The connections array contains the list of connection mediums that will be used to communicate between the various elements within the system implementation. In addition to these major structures there are lesser ones, such as the Bus and Data arrays, which contain a simple list of all bus and data types that are to be implemented within the AADL model. The purpose of these being to be define the information before the device and system model elements can be described.

The process GAM follows is to first read in, dissect, and extract the structures provided by the framework’s user, as well as import GAM’s database of known device descriptions. From these, the tool moves to generate a blended amalgamation of the two device sets the create a pool of potential devices that is categorized by device class. Leveraging the user-defined system attributes and architecture map, GAM generates and filters combinations of device class implementations to produce a valid set of design combinations. In addition, the process forms the bus and data arrays required to describe each model. Lastly, based on these design space specifications the system implementation map is constructed. These elements are then passed on to AADL model generation functions to describe all the spawned design space solutions.

Fig. 4

Illustration of detailed framework for GAM

Figure 5 provides a visualization of the algorithm implemented within GAM to produce the potential models. GAM extracts the potential set of devices based on the user-defined input and the set of known AADL definitions by leveraging any user-defined system attributes to help define the design space. Using these potential devices, GAM creates a Cartesian product of all potential devices combinations that could produce the various system models. From this pool, additional filtering removes any combinations that are "nonsensical" or excluded from design space via user-defined limitations of the explored models. At the end of this process GAM produces a set of candidate AADL files that model the desired system architecture.

Fig. 5

Illustration of flow graph for the GAM model generating algorithm

3.2 TAMSAT

There has been significant prior research in security modeling, particularly with respect to how models can be constructed, how to represent different threats within those models, and how to evaluate models and attack trees. However, there has been a lack of work in developing effective methods for translating system architecture models into an attack tree security model. To this purpose we have created the TAMSAT (Translation of AADL Model to Security Attack Tree) tool which aims to help fill this void by providing a method for converting user defined systems into easy to understand and assess security models [35]. In other words, we wish to enable the translation of an early life-cycle development model into an attack tree structure that can be used to determine the security risk of the model. This examination can occur before a specific architecture has been decided, after architectural decisions have been made, and as a method to explore and evaluate the design’s choices and constraints. The original publication work for TAMSAT [35] shows how TAMSAT can be utilized repeatedly to evaluate existing designs, determine the validity of the current security constraints and requirements, and highlights how TAMSAT could be used as part of an automatic toolset. In this section we will explain in detail the design of TAMSAT and how it takes an AADL system model and produces an attack tree model. Figure 6 illustrates the flow of TAMSAT.

Fig. 6

Illustration of TAMSAT flow graph [35]

TAMSAT is responsible for the translation of AADL model files into a security attack tree format that can later be evaluated by SMART, our security risk analysis tool described later. TAMSAT can produce an AADL XML file which follows the same format used by the OSATE IDE as implemented by the AADL community. In addition, TAMSAT can also produce Python Graph Object and Python JSON files that represent the security attack tree along with supporting model “helper files” in a form easier for SMART to digest. Figure 7 provides an illustration of the Python Graph Object and helper files that TAMSAT uses to interpret the AADL model into a format that can be passed to SMART.

The procedure by which TAMSAT goes about the conversion process is as follows. First, TAMSAT reads in the system AADL model file and produces three datasets of information. These JSON dictionaries are a device dictionary, a system dictionary, and an implementation dictionary. The Device Dictionary is a JSON structure that contains the information read-in by TAMSAT in relation to the various devices that were defined within the provided AADL file. The System Dictionary is a JSON construction that contains the information read-in by TAMSAT relating to the system that is defined within the presented AADL file. Lastly, the Implementation Dictionary is a JSON arrangement that contains the information read-in by TAMSAT that details the system implementation designated within the AADL model file. After having fully read the provided AADL model, TAMSAT takes the aforementioned datasets and reconstructs the original AADL model in a security attack tree format that can be understood and evaluated by SMART. By default, TAMSAT represents the attack tree model using a generated Python Graph Object and helper files to simplify the translation and later assessment by the SMART tool. Note: For the purpose of the G-T-S framework, the research focuses on the use of the Graph Object and helper information since this allow for the most enriched version of the AADL model translation within the security attack tree format. The AADL XML format remains as an artifact of earlier implementations of TAMSAT.

Fig. 7

Illustration of detailed framework for TAMSAT

The Python Graph Object simply provides a mapping of the attack tree connections between vertex points with the Asset of Importance (AoI) acting as the root of the tree. The AoI is user-provided and represents the target or goal of an attack on the modeled system. An AoI is any data, service, or other asset that has intrinsic value and must be protected. Additionally there is a vertex map, an edge map, an “entry/exit” map, and a vulnerability map that are all generated by the TAMSAT tool. The Python Graph Object is generated from a custom Python Class written to represent the connections of vertices and edges that exist in the attack tree. The created Graph Class contains the functionality to identify vertex points, edges between vertex nodes, identification of attack tree depth for each vertex from the root (or AoI) node, as well as determination of any existing leaves in the attack tree. The purpose for identifying these leaves is to locate entry/exit points within the AADL model representation. The vertex map is a JSON database that maps each existing vertex node (or device element) within the Graph Object representation of the AADL model. This mapping contains the name of each vertex along with supporting information including any “Ports” or “Bus” connections/requirements for the vertex node as well as a flag for identifying the “Root Node” (i.e., AoI). The edge map is a JSON structure that maps each existing connection between vertex nodes (i.e., device elements) which comes from the system implementation definition of the AADL model. These connection names (which are provided in the “connections” definition of the original AADL model file) are used as the identification tag for each edge. In addition to the connection name, the edge map contains information relating to the start and end nodes for the edge connection, the data type, bus information, if a port is used in the connection, and the direction of the edge connection.

The entry/exit map is a JSON database that contains a list of the entry/exit points within the attack tree Graph Object. These come from a mix of TAMSAT identifying entry/exit points within the AADL model (i.e., feature definitions of the system definition) as well as depending on any hardware vulnerabilities found to potentially exist within the model. There are some key points to make here relating to the assessment and addition of these entry/exit points to the JSON map. First is that the AADL model file is assumed to be a series of device definitions and a single system (and system implementation) definition that represents that highest level abstraction of the model. Therefore any features defined in the system definition are interpreted as the outer I/O of the model. The second point is that any association of hardware vulnerabilities to the AADL model is done via the bus connection requirements defined within the description. This allows for hardware vulnerabilities to be interpreted within the AADL model via their influence on the overall Security Risk (SR) evaluation. Note: In the scenario that no hardware vulnerabilities are evaluated by TAMSAT (i.e., added to the vulnerability map or the entry/exit map), then the tools will default to a “Brute Force” mapping of the generated attack tree object. This is done by leveraging a “leaf-finding” functionality of the customized Python Graph Class and combining this information with a Cartesian Product of all vulnerabilities mapped to each of the root-to-leaf paths found. The purpose for this behavior is to provide an “un-directed” security risk evaluation of the supplied AADL model description.

The vulnerability map is also a JSON database that maps each existing vertex node within the AADL model to a “Vulnerability List” which contains a set of vulnerabilities associated to the vertex based on CVEs (i.e., CVE-IDs). These CVE identifiers are related to both software and hardware vulnerabilities as defined within a user-made JSON structure of known vulnerabilities. The software vulnerabilities are associated via the given device definitions (e.g., MySQL, Intel Server, Generic Application) and the hardware vulnerabilities are defined via the related bus type and requirements delineated in the AADL model description. The reason for leveraging CVE-IDs is that the majority have an associated CVSS score which is later interpreted by SMART as a probability of success ($p_s$) for each vulnerability being exploited within a given attack path.

The existence of a hardware weakness is not enough since the means to exploit this weakness may not be present and therefore the hardware vulnerability cannot be counted as present. Despite the presence of the problem, nothing can be done to leverage this for malicious action. Not only does one examine the implementation of the model (e.g., the description of the connections over ports and bus mediums) but review the description of the individual devices. There may be additional interfaces (e.g., ports, buses) that are described in the device model that were not implemented in the system implementation description. In other words, this is the equivalent of a device having a UART or JTAG interface that is not connected to any other device in the system but still poses the risk of a physical attack since the interface exists and can be accessed. Lastly, this information indicates if there is a new Entry/Exit point for an attack path into the system. This is paramount for SMART so that all potential attack paths are examined and the associated risk is known. The presence of the hardware vulnerability potentially can shorten an existing attack path and alter the costs for the attacker.

The process that TAMSAT follows for mapping vulnerabilities is as follows. First, TAMSAT checks for each device in the vertex map of the system implementation model description and compares them to the items in the vulnerability database. Based on the match, TAMSAT then compares the vulnerability description to known information about the device (e.g., version number of a database implementation) and limits the vulnerabilities associated to the device based on this information. Note that in its current state TAMSAT uses simple text matching to associate this information. Second TAMSAT then examines each of the devices in the system implementation description to check for the bus mediums that are associated with known vulnerabilities (e.g., BreeZ, Zigbee). Based on the relationship of the device and the bus mediums, TAMSAT decides if additional vulnerabilities are present and need to be added to the vulnerability map of the AADL model. Last TAMSAT searches through the device dictionary that contains information on each device that is described within the model. Unlike the earlier checks, this is not limited to the system implementation details, but the descriptive elements of the devices themselves. For example a UART bus being present on an ASPEED server that is known to have a vulnerability. Due to the combination of the device and the specific bus TAMSAT’s logic determines that not only is the vulnerability present due to the chosen device, but it presents a hardware vulnerability that would include a new attack path for later examination. During the above procedure TAMSAT always checks that any vulnerabilities are not duplicated thus inflating the risk during later evaluation by SMART.

3.3 SMART

The Security Model Adversarial Risk-based Tool (SMART) is a systems security design evaluation tool that takes an attack tree model for a particular system design and performs a monetary evaluation that is representative of the security risk posed by the given design [37]. This value essentially shows the potential risk of having a malicious actor targeting a given asset of importance (AoI) and the conceivable impact of a successful attack. The flow of SMART is shown in Fig. 8. A more detailed examination of SMART, the parameters required by the tool, and its internal functions is left to the reader to explore in [37]. SMART starts with an attack tree that is representative of the entry points in a system design and its pathways to an asset of importance. SMART evaluates the attack tree using input from parameter records and data from the MITRE Common Vulnerability Enumeration (CVE) database. The final output is a security risk assessment calculated using the formulation described below:

Fig. 8

Illustration of flow graph for SMART

The Security Model Adversarial Risk-based Tool is purposed with the evaluation of a presented attack tree for a monetary Security Risk value reflected in USD. SMART leverages associated vulnerabilities for each vertex node in the provided attack tree mapping of an original AADL model file. SMART utilizes a Python Graph Object and helper data, which provides an enriched representation of the security attack tree. In this scenario SMART is capable of performing a “Brute Force” evaluation of the model’s Security Risk or can leverage a provided Entry/Exit map to have a “directed” evaluation of potential Security Risk (i.e., based on provided attack paths). In the “directed” case, one can assume that any hardware vulnerabilities associated to the AADL model provide a separate Entry/Exit point, which adds an additional attack path to the Security Risk evaluation. The tool leverages known associated vulnerabilities (both software and hardware) which are represented through CVE-IDs attached to each node in the vertex map. The reason that CVE-IDs are used is that the majority have an associated CVSS score that is interpreted as a probability of success ($p_s$) in the Security Risk evaluation.

In addition to the CVSS score, SMART also requires information relating to the following variables: Probability of Attack ($p_a$), Cost of Attack ($c_a$), Value to Attacker (A), Impact of Successful Attack (I), Cost of Maintenance ($c_m$), Cost if Initialization ($c_i$), Cost of Operation ($c_o$), and Scale Amount. The Probability of Attack is a quantitative value that represents that probability that an attacker will make the effort to attack a given system. Following the work of Ferrante et al. [9] this research leverages a set scale of values from 0.0 to 10.0 to represent the range of possibilities that a given system will be attacked. 0.0 being no possibility and 10.0 being an absolute possibility. In order to help standardize this scale the research relies on the Common Vulnerability Scoring System (CVSS) as the depiction of the probability of attack. The argument behind this mapping is that while the CVSS is meant as a severity score associated to the specific CVE vulnerability, this same CVSS value can be used as a portrayal of how likely it is to be targeted by an attacking entity. Cost of Attack is a monetary value that represents the costs that an attacker will need to expend in the process of attacking a given system in order to access a chosen Asset of Importance (AoI). Note that this value is particularly difficult to assess due to a variety of variables, including but not limited to research, practice efforts (successes and failures), equipment costs, and social skills. For example, there is no clear delineation on how to evaluate the cost of procuring an exploit off of the Dark Web, although there is literature exploring the subject. The Value to Attacker variable is also a monetary value that represents the assessment to the Asset of Importance (AoI) by the attacker. It is worth noting that this can be represented as a “Return on Investment” or other potential monetary gain relating to the attacker’s efforts and plans. This value could be an expected offset to costs associated with the action of performing the attack. Impact of Successful Attack is represented as a monetary value that indicates the impact to a defending entity should an attacker successfully access the target of the attack (i.e., Asset of Importance).

The last four variables are more related to direct costs of the defender for hosting the system being examined. Cost of Maintenance is a monetary value representing the USD associated with maintaining the system element. For the purpose of the framework this value is interpreted as a cost of having to hire an individual to care for the element. Cost of Initialization is another monetary value depicting the USD spent in initial setup and configuration of a given system element. This value can be viewed as a single point in time cost related to the element. Cost of Operation is a portrayal of the USD spent in operating a given system. For the purpose of the framework this value was considered over a year of time which causes an amortization of the Security Risk to an annual value, however this value could be assessed over any desired time scale. The Scale Amount is a scaling variable that is used to scale certain variables to the same range as other input values (e.g., $c_o$, $c_m$, $c_i$). Note: Certain cost variables (e.g., $c_o$, $c_m$, $c_i$) are amortized over a chosen time period. This means that the resulting Security Risk (SR) value is a monetary amount over a year, since this is how these values are estimated within the framework. For example: The $c_o$ value is estimated as $2450 which represents an approximate of electric costs over a year of time.

The aforementioned variables can be difficult, if not impossible, to nail down due to proprietary information, relative/arbitrary association of values, and the general secretive behavior when it comes to information relating to costs, impact, or other forms of intellectual property (IP). With the assumptions being made to these values, SMART is able to evaluate the Security Risk of the provided attack paths. Note: This is possible and effective because so long as the numbers used are accepted and constant, then the relationship between evaluations can be used, even if the information “gives the wrong answer” [27].

In the end SMART produces a set of information related to the Security Risk evaluation of the attack tree. This information includes: the most expensive/risky subpath tuple, the associated CVEs to the Max Security Risk Subpath, Maximum Security Risk seen from the evaluated subpath tuples, Maximum Security Risk subpath shown in total cost (e.g., scaled to the USD amount), the least expensive/risky subpath tuple, associated CVEs for the Min Security Risk subpath, the Minimum Security Risk seen from the evaluated subpath tuples, the Minimum Security Risk subpath in total cost (e.g., scaled to the USD amount), and lastly the full path list Security Risk evaluation generated by SMART. A summary description of key output is provided as follows:

• Aggregate Security Risk: This value is the Security Risk associated with the entire evaluated model and all potential attack paths based on existing vulnerabilities.

• Subpath Security Risk: This is the Security Risk associated with a specific attack path within the evaluated model. It is worth noting that this value is generally coupled with minimum or maximum adjective to illustrate the extremes of risk due to vulnerabilities.

Note that the aggregate and subpath security risks are produced in a maximum and minimum form. The purpose for this being that SMART shows the user the most risky and least risky subpaths existent within the model as well as proof of this claim.

4 Vulnerability modeling

Modeling vulnerabilities has been at the heart of the cyber security community efforts. However, the purpose of modeling vulnerabilities extends beyond the aspiration to remove them from a system but to also better understand the probability of an attacker successfully using a vulnerability to advance their objective of attacking a target or obtaining an end-goal. Before continuing, note that there is a difference between a “weakness” and a “vulnerability”. A vulnerability is the exploitation of a weakness to achieve some goal, which tends to be malicious and not benign. The second point is that by no means is the set of known vulnerabilities complete. New vulnerabilities are constantly being found and presented to the cyber security community for the purpose of expanding the community’s knowledge and awareness of these issues. However, for these same reasons, some vulnerabilities are unknown due to a variety of potential reasons. Overall modeling vulnerabilities can be difficult, but there is a method by which one can understand them and effectively analyze them.

4.1 Challenges

The secretive nature of exploit developers leads them to protect their Intellectual Property (IP) and efforts. This can be due to an expectation of selling said exploitation (e.g., rootkit), due to attempts to reach out to manufacturers or Original Equipment Manufacturers (OEMs) in-order to provide patches prior to an exploit’s public release, or even due to the suppression of CVEs due to a company’s policy of protecting their image (e.g., there are no Mainframe CVEs). Furthermore in the case of hardware vulnerabilities, due to the sense of immutability of hardware, a company or organization may attempt to keep issues secretive for as long as possible to protect their image or bottom-line [4].

Due to the research and effort that is required to discover, document, and present vulnerabilities, this type of work is generally left to the cyber security community through CVE submissions or conference presentations (e.g., DEFCON, BlackHat). To support this effort of vulnerability enumeration, vulnerabilities exist in essentially two spaces. First are those CVE ID vulnerabilities, which are reported publicly as known vulnerabilities and exposures. The second are the unreported or undiscovered vulnerabilities. These include ‘Zero Day’ vulnerabilities, which are previously unmitigated weaknesses that are exploited with little or no warning.

One can start by searching through the existing MITRE resources to find good hardware and software CVEs that might apply to the system in question. Whether one is dealing with embedded systems and IoT devices or approaching a more traditional environment of server and database elements, the analysis will require the background research and planning to properly account for potential Security Risk. These issues can become further complicated due to a need to determine how vulnerabilities associated with any Graphical User Interface (GUI) or application elements might be treated differently based on the component hardware they are running off of. In all cases one should follow a clear method of research and analysis to help normalize their findings and provide a more meaningful overall Security Risk.

The processes outlined below are related to the method by which an individual can determine potential costs associated with a given vulnerability and how to estimate probable expenditures by an attacker to exploit said vulnerability. In each case one has to examine not only the scenario description outlined in the CVE entry, but also account for costs of the equipment, additional protections put in place, and any other potential hurdles one has to overcome as a defender or attacker element.

4.2 Research analysis methodologies

In the process of developing the GTS framework, it was necessary to estimate costs and expenses related to the vulnerability, the attacker, and any mitigations. Without these values, SMART will be unable to produce an associated security risk value. The motivation behind producing these research analysis methodologies is that there are currently no publicly available databases of expense approximations, and therefore we are required to produce a clear and repeatable method for estimating these costs. By providing these methodologies we ensure users can repeatedly produce the same work for each vulnerability examined.

As a reminder, the end goal of these investigations is to establish values for the following SMART variables: cost of attack $c_a$, cost of initialization $c_i$, cost of maintenance $c_m$, and cost of operation $c_o$. Additionally, these efforts can provide clarifying information to improve our approximations for the impact I and value to attacker A in relation to the asset(s) of importance.

4.2.1 Vulnerability

The first approach we present is Algorithm 1 which can be used to determine the general costs associated with a given vulnerability. The methodology is a high-level overview of the efforts needed when determining the large scope expenditures of vulnerabilities examined by the GTS Framework.

Algorithm 1 represents the high-level process to generate the necessary SMART variables related to each vulnerability examined. The first step in the process is to search the existing CVE database for any applicable vulnerabilities. Note that currently this effort is done by using a search bar function on the MITRE CVE website which therefore requires a “guru” or Subject Matter Expert (SME) knowledge to improve this search process. To begin, one conducts a detailed examination of the vulnerability description looking for constraints and requirements outlined for an attacker or defender. This is to plan for and potentially deal with any of these future issues. In parallel the individual should also perform a dive into any proposed or provided mitigation(s) that exist for the known vulnerability. Locations to check for this mitigation information include:

• Any existing CVE entry (e.g., MITRE, NVD) either within the description or within some of the supporting information.

• Any announcements/bulletins put out by manufacturers, software developers, or companies that detail out the existence of the vulnerability along with any potential fixes, mitigations, or news of potential fixes being generated.

• Other “curated websites” may also include “readings” or digested information intended to provide a better layout of the vulnerability landscape. Two example websites include “vuldb.com” and “cxsecurity.com”

Furthermore, one should examine any additional technology or equipment needs and costs for successful deployment and exploitation of the known vulnerability. This value may be similar for the defender and attacker elements since either entity may have to purchase the same equipment in order to be able to practice, train, and better understand the target of the exploitation or attack. Additional costs are going to come from skillsets (e.g., hardware, software, social), general equipment needs (e.g., screwdriver set, cables and pins), specialized equipment needs (e.g., JTAGulator, electronic key bumper), research and learning time, as well as practice trial and error. Specifics of these actions may relate to identifying sources of cost or determining how to assign accurate cost metrics to attacker cost elements. At a minimum this will require updating the values in two of SMART’s databases. The first is the Cost Values Database which contains costs related to the device elements deployed with the AADL model (i.e., Cost of Initialization, Cost of Maintenance, Cost of Operation). The second is the Asset Value Database which contains Impact, Attacker Value, and Alpha values intended for use with any Assets of Importance being examined.

While Methodology 1 provides a higher-level overview of the deep dive process, the next methodology provides greater detail in determining the attacker-related costs with exploitation of vulnerabilities present in the design.

4.2.2 Attacker

A point to make prior to the exploration of Algorithm 2 is that when performing a deep dive investigation of mitigations to the known vulnerability, one may find further illumination to the hurdles an attacker may have to overcome. For example, the requirements for specialized or proprietary hardware or software in order to interact with a vulnerable element.

Algorithm 2 determines the expenditures associated with exploitation of a chosen CVE vulnerability. This process needs to be iteratively applied for each vulnerability that would be encountered by a malicious element. These efforts first begin with a determination of generalized costs for performing the intended exploitation. The generalized costs include hardware exploitation gear, software exploitation gear, and social exploitation gear. In addition to the general tools, an attacker will need to work on developing and honing the necessary skillset that might be required for a successful exploitation. The attacker may be required to perform a variety of hardware skills in order to circumvent and leverage any existing physical protections and interfaces. These skills can include but are not limited to picking locks, faking keys, removing access panels, soldering connections, and connecting interfaces. On the software side, the attacker will need to perform a mix of research, script preparation, and skill development to allow them the highest probability of success for the attack despite all foreseen and unforeseen obstacles. Note that, in the wild, there is the additional scope of time where a malicious actor may deploy an Advanced Persistent Threat (APT) for the purpose of maintaining access and having the time to further develop scripts and circumvention of defensive detection. Finally, attackers will need to develop their social engineering knowledge and techniques. The malicious actor may need to talk their way past or elicit the help of others in order to gain access to the target of the vulnerability exploit (e.g., obtaining physical interface access). These costs can be variable and depend on the degree of trust that one attempts to establish with other individuals ranging from commercial owners to larger enterprise or government defenders.

In addition to the process provided, there exist other resources that can be leveraged to determine potential attacker costs (e.g., Cost to Attacker $c_a$). One can always search for an “exploit broker” or other form of black market Subject Matter Expert (SME) that can help determine the skills, knowledge, or cost associated to performing a successful exploitation. These requirements can greatly vary from having to personally develop all necessary tools and capabilities to simply purchasing a “ready-made” toolkit focused on exploiting the intended vulnerability. For example the website vuldb.com [34] is, as of December 21st 2021, a MITRE CVE Numbering Authority (CNA) which means that VulDB is an organization that is authorized to assign CVE identifiers (CVE IDs) to vulnerabilities [18]. In addition to the CNA capability, the advantage of VulDB is that it is a community maintained resource that also provides details on the exploit, potential mitigations, and estimations on “exploit prices”. As provided in the description of the mathematical algorithm used to calculate prices, the “algorithm got developed by [their] specialists over the years by observing the exploit market structure and exchange behavior of involved actors.” The key points to take away are: there is a security community based effort that can be leveraged to provide meaningful estimations of cost; different potential ranges of cost are provided for each exploit; our own background research/estimations can be augmented in order to improve our approximations.

4.2.3 Mitigations

The next large hurdle in determining attacker costs is the procurement of specialized software or hardware which could require additional effort to obtain. In any scenario, there are embedded costs associated with each choice (e.g., time, communication, trust, authentication, monetary, effort). These efforts generally begin with the attempt to obtain privileged access from the manufacturer, traditionally by request, to determine price or cost of the specialized tools. When this effort bears no fruit or as a comparison one can move to determining costs associated with purchase via a “second-hand” or third-party seller or re-seller. As a final choice one can examine black market channels such as the dark web, which can introduce other additional costs (e.g., trust, effort, time) in order to obtain the desired information or tools. Note that in all cases the expenditures can be subject to market pressures and influence, as well as the whims of the potential actors involved.

Similar to the other two Algorithms 1 and 2, Algorithm 3 is focused on determining mitigations to a known vulnerability. The purpose is to have a methodical approach to finding existing solutions to lower risk presented by the issue, understanding the best practices for lowering risk, obtaining insight into the core of the problem, and how to associate expenses to each. Assigning cost to best practices can be difficult, but doable assuming that one is aware of these protective steps and the core of their effects. There is further complication when delineating between the type of mitigation (e.g., software, hardware), but this is why it is critical to document the information to ensure others can understand and repeat the work.

Mitigations can be tricky depending on whether the underlying weakness is in software, hardware, or a mix of both. In most cases, software-based vulnerabilities can be mitigated in a relatively easy manner while hardware-based vulnerabilities can be impossible to completely mitigate, although it is possible to produce protections that aid in reducing the risk presented. For example, watermarking designs is a common method for addressing threats, weaknesses, and vulnerabilities that can arise from alterations to the intended construction. An additional layer of protection that contributes to preventing malicious modification of any shared files. A key point to make is that hardware-based threats, weaknesses, and vulnerabilities tend to lead to either software patches to handle hardware concerns or to have a complete product recall to ultimately replace the affected hardware. This change in hardware is intended to remove the known vulnerability or otherwise alter the variables affecting Security Risk.

4.3 Review of hardware vulnerabilities

The potential landscape of the hardware vulnerabilities can be quite extensive if one takes a broader view of the security concerns. For example, counterfeiting could be a distribution threat that can occur in a variety of locations within the supply chain. Depending on where in the manufacturing, integration, or distributive process one examines these considerations can change their magnitude and influence. Another potential avenue of concern is the examination of side channel attacks. Taking AMD chips as an example, it was found that a generation of processors were vulnerable to known side channel attacks [30]. Examination of this vulnerability will lead an individual to research the concern, investigate the steps necessary to perform the exploitation, and scrutinize any solutions that claim a mitigation to the problem. In this case the decision may be straightforward to simple patch, better protect, or otherwise alleviate the risk with no second thought.

However, what about a scenario where a vulnerability in a Network Attached Storage (NAS) device that would leave sensitive or critical data open to malicious efforts. Now allow us to contemplate time as a factor in our considerations. For the sake of argument, let us assume that the manufacturer of our NAS solution immediately releases a mitigation that can eliminate the problem. However this same solution also causes a significant reduction in performance. If this NAS happens to be accessed by a large number of people, then this “side-effect” of the solution may be seen as unacceptable. If only one or two individuals are accessing this information, then perhaps one could live with this hopefully temporary problem. This relies on the expectation that an improved mitigation that is not as detrimental to the system will eventually be released as well. In this same regard, we are making assumptions in relation of the importance to the manufacturer that this specific NAS be patched, the size of the team focusing on addressing the problem, and the resources at hand for the aforementioned group. To this point, we can vary the previous decision by adding the possibility that a better, less adverse solution will come eventually and that we are willing to accept the potential risk of an un-addressed vulnerability for some unknown/estimated amount of time. It is important to present potential considerations, but for the sake of brevity we will simplify our view of the problem.

A core consideration one must make when modeling hardware vulnerabilities is the unique attack surface that can allow for low-privilege users to access system internals. Traditionally this is seen as being done through a Serial or UART interface that may, or may not, have some form of authentication or access control. More recently the JTAG interface has become known as a means of accessing and/or altering system internals such as memory. In all these cases the concerns range from debug information (e.g. system logs) to presenting a user interface, which at worst case is access to a root or admin level account. Essentially the hardware vulnerabilities can potentially provide a means to circumvent software-based protections intended to keep out remote attackers and give more direct access to the sensitive inner workings or operating system of the device.

5 IoT home model evaluation with G-T-S

In this section, we explore the GTS framework through a detailed example of GAM, TAMSAT, and SMART working in unison to generate and evaluate a collection of IoT Home models. We present Fig. 9 as a representation of a potential IoT Home system. To clarify, we see the “IoT Home User Interface” as being made of two parts. The application(s) that are run and the graphical user interface that any given user interacts with. All communication for the IoT home system is aggregated through a central server. This server also communicates with a local database that contains sensitive information, configuration data, and other protected material relating to the home and IoT devices.

Fig. 9

Pictorial representation of the basic IoT home model

The first step that needs to be taken in preparing the framework is to decide upon the specific examples of Hardware CVEs that will be included into the model device component hardware through scenarios that apply to the model. Next, we need to determine how to represent hardware versus software for a single element (e.g., server, database). In the AADL language this is represented by the difference between the ports and buses that are described within the models generated. Our framework leverages this to allow for the description of hardware that is running vulnerable software while retaining the necessary granularity to evaluate the effects from each. Thanks to the capabilities of the AADL language, it is possible to further increase the granularity of hardware to include component specifics (e.g., memory type). One can then leverage this to allow more accurate representation of DRAM that could be vulnerable to Cold Boot or Memory Remanence attacks. However, for the purpose of our framework we chose to maintain a more abstract level of hardware representation since this allows for great flexibility in exploring the design space. All contents of the AADL model file are interpreted by the TAMSAT and the information is maintained within TAMSAT’s internal structures. This allows for simple extension of the GTS logic to include additional expansion of the toolsets.

In order to simplify the presentation here, we have made a few assumptions when evaluating the IoT Home model’s design space. First is that only certain elements of the IoT Home model can have hardware vulnerabilities. These include the database, server, and sensor elements. Second is that due to the presence of hardware vulnerabilities, this means that the aforementioned design elements can be seen as entry points for individual hardware attack paths. In turn, this means that there is potential for shorter attack paths that can increase the risks posed by the full system. Last is that the other device elements of application and graphical user interface (GUI) can be seen as software specific and therefore can be ignored for hardware vulnerabilities. This does not mean that the application and GUI do not pose as their own entry points for attack, simply that any associated vulnerability can be expected to not be hardware-based. Examples of the intended hardware vulnerabilities to be implemented in the IoT Home model are shown in Table 1.

Table 1 Potential hardware vulnerabilities present in IoT home model

After determining the hardware vulnerabilities to be associated to the IoT Home model elements, one needs to also take a look at the potential mitigations that could prevent the inclusion of these vulnerabilities as risk contributors. First, we follow the process provided by Algorithm 3 for the sensor vulnerability of CVE-2014-2361. From this effort we find that the mitigation to this vulnerability is to use a different protocol (i.e., not BreeZ) or a different brand of sensors. One should note that while this would mitigate the risk posed by CVE-2014-2361 it may introduce new vulnerabilities based on the different hardware used. This could come due to brand change or due to the new communication medium (e.g. Bluetooth, Zigbee). Repeating this process for the database vulnerability of CVE-2019-2503, one finds that the mitigation is to use a different version (non-vulnerable) of the Oracle MySQL server or different database solution (e.g. MongoDB, CouchDB). In this scenario the designer or developer is making a risk decision based on the specific hardware vulnerability (i.e., CVE-2019-2503) associated with the chosen database solution. For the first server vulnerability of CVE-2018-3652, following the process of Algorithm 3 one finds that the mitigation is to use a server that does not implement an Intel Atom C/Xeon processor as the CPU. However, if the mitigation decided upon is to use instead an ASPEED ast2400 or ast2500 device then a different hardware vulnerability (i.e., CVE-2019-6260) becomes a potential problem. This acts as a good specific example of trading one hardware vulnerability for another. Researching the second vulnerability of the server (i.e., CVE-2019-6260) one finds that the solution for the mitigating the issue is to disable any component buses, update software, or otherwise change configurations to prevent malicious action. In the majority of scenarios this vulnerability can be viewed as being software or network based, but when an accessible UART interface is present this concern becomes part of a physical hardware-based attack path.

Table 2 Table of IoT Home Model Variations and Components

With these concepts in mind we move to discuss the design scenarios that GAM will generate for the IoT Home. For brevity, we only discuss eight of the possible designs as shown in Table 2 with different database, server, sensor, and bus type implementations models. Design 01 serves as the simplest model and produces a baseline evaluation of the Security Risk presented within the design space. Designs that use the BreeZ Sensor have the CVE-2014-2361 vulnerability that can be treated as a separate Entry/Exit point but will fluctuate with the cost of deployment since the BreeZ system will require specialized hardware and considerations. Use of the Oracle MySQL database introduces a vulnerability that creates a new, shorter attack path to the asset of importance. However, the costs related to performing the attack change due to the need for the requisite low privileged attacker to have access to the physical communication interface attached to the hardware. The CVE-2018-3652 vulnerability affects designs with the Intel Atom processor and can be treated as a new Entry/Exit point, but one that requires a degree of physical (limited) access for leveraging a debug interface. Designs with the ASPEED server are susceptible to the CVE-2019-6260 vulnerability as long as the system also has a console UART attached to a serial connector in order for an attacker to leverage the intended exploit.

Now that we have an idea of the type of information and research required to better understand and represent the evaluation of Security Risk for these models, the next step is to perform deep dives on the vulnerabilities and associated mitigations. Sect. 5.1 leverages the methodologies presented in Sect. 4.1 (i.e., Algorithms 1–3) and applies them to the vulnerabilities shown in Table 1.

5.1 Deep dive of vulnerabilities and mitigations

This subsection is focused on the “Deep Dive” required to examine a given vulnerability, determine any known mitigations to the issues, and establish the necessary SMART variables that will be required for the eventual evaluation of Security Risk (SR). For conciseness, we examine only the following two vulnerabilities:

• i. CVE-2018-3652 [ Intel Direct Connect Interface (DCI) Vulnerability ]

• ii. CVE-2014-2361 [ BreeZ Vulnerability ]

5.1.1 CVE-2018-3652 deep dive

We begin by following the process of Algorithm 1 and search through the CVE entries for CVE-2018-3652. Examining the description of the vulnerability we find: Existing UEFI setting restrictions for DCI can potentially allow a limited physical presence attacker to access platform secrets via debug interfaces. In other words, an attacking entity can use a debug interface to access platform secrets assuming they have limited physical presence and the system has specific UEFI settings. Keeping this information in mind, we follow Algorithm 3 to determine the potential mitigations to the risk presented by the vulnerability.

Security advisories from both Intel [13] and Lenovo [15], indicate that the recommended mitigation is to simply update configurations and software version to remove the vulnerability. In addition to these solutions, another option is to control the physical access to the deployed system(s). One should implement all of these mitigative efforts to provide a defense in depth to counteract the potential exploitation. In terms of cost, one can monetize the time required to determine correct configuration, update the necessary software, and troubleshoot for any residual risk or inadvertent problems caused by the changes. There is no entry cost or expense level that must be achieved beyond costs already involved in the deployment of the original device. Some expense is incurred in the erection of physical barriers or protections to limit direct accessibility of the vulnerable hardware. In total, depending on the intended granularity of cost during evaluation one can include all these amounts, but for the purpose of our investigation these expenses are negligible compared to the deployment costs. Next, we move to Algorithm 2 to determine attacker related expenditures for exploiting the CVE-2018-3652 vulnerability.

Understanding and improving accuracy of attacker expenses requires a more involved dive into the subject matter of the vulnerability description and its associated mitigations. Re-examination of the exploit description leads to a closer review of the “DCI” element. DCI stands for the Intel Direct Connect Interface which provides access to DFx-features (e.g., JTAG, Run-control). In addition, there is a hardware requirement of a USB3 port to exist as well as one owning an Intel SVT Close Chassis Adapter (Product code: ITPXDPSVT). Purchasing the ITPXDPSVT adapter requires either a request for privileged access from the manufacturer (i.e., Intel) or acquisition through a third-party vendor (estimated at approximately $350 on eBay). Increasing complexity is the requirement to use Intel ITP software or Intel System Bring-Up Toolkit NDA software which are sold separately from the adapter cable. As with the adapter, acquiring the software will require either establishment of trusted access with the manufacturer (i.e., Intel) or purchase off the black market. In addition to these rigid requirements, an attacker would need to spend the time learning, practicing, and streamlining their reconnaissance and attack processes before attempting attack on a real world target. This is to minimize the “noise” produced during the attack, thus minimizing the risk posed to the malicious actor. Now that we have a better idea of the equipment and knowledge requirements for the defender and attacker elements, we need to produce values that can be fed into SMART for design evaluation of Security Risk.

The next piece of following Algorithms 1–3 is to establish the cost values that will be associated by the framework to evaluate the Security Risk. To start, we price the necessary specialized hardware. We assume that it is not trivial to obtain the trusted status with Intel required to obtain the specialized hardware. Therefore, we rely on procurement on a “Second Hand” or “Third Party” sales website. From our research, we found the best estimate to be $350 USD, but these costs can fluctuate based on market pressures. Pricing the specialized software is far more difficult to estimate since one needs to either obtain privileged access from the manufacturer to even determine the price or find the software via black market channels (e.g., dark web) which can introduce additional costs. In either scenario there are embedded costs associated with time, communication, trust, authentication, monetary exchanges, and effort. An estimate can be based on similar diagnostic software and platforms used within the automotive industry. Based on an amortization of a handful of specialized industry diagnostic software as well as a full solution technology platform, we arrived at an estimate of $4000 for acquisition of the essential software.

Thus, we have found that an attacker may have to spend an additional $4350 ($350 + $4000) in order to exploit the weakness. In comparison to the attacker costs, the defensive expenses for implementing mitigations are seen as relatively negligible for the purpose of this discussion. Although, this granularity of cost can easily be calculated with examples ranging from the cost of a padlock to prevent physical access to the wattage expended in downloading software updates.

After these specialized expenses, we need to determine potential costs of developing the necessary skill sets that a malicious actor would need in order to successfully exploit the CVE-2018-3652 vulnerability. One should note that due to the need for an attacker to be either prepared and/or perform necessary reconnaissance, the inherent costs associated with the attack and probability of successful attack can be variable. To successfully exploit the vulnerability, a malicious actor will require some basic social, hardware, and software skills. The attacker may need to talk their way past or elicit the help of others in order to gain access to the target of the hardware vulnerability exploit (e.g., DCI access). The attacker may be required to perform a variety of hardware skills in order to circumvent or leverage existing physical protections and interfaces. Examples include picking locks, faking keys, removing access panels, soldering connections, and connecting interfaces. The malicious actor will need to perform a mix of research, script preparation, and skill development to allow for the highest probability of success for the attack, in spite of all foreseen and unforeseen obstacles. One can attempt to determine expenses related to the training, learning, insight, and practice required to learn these skills through finding courses on each skill set, tool sets focused on security professionals, and time estimates for hours spent studying the material.

The last set of expenses that need to be determined are the general equipment and knowledge exploitation costs. These costs include any disguise, tools, training, reconnaissance, bribes, etc. that may be required of the attacker to establish the necessary confidence for exploiting the known vulnerability. One should note that the above considerations need to be made for each and every vulnerability that is examined. For this reason the information is shown in detail, but for the other vulnerabilities it will be assumed that these similar concerns are also considered when estimating the cost values necessary for later Security Risk (SR) evaluation via SMART.

Summarizing, we estimate that the adapter cable would cost around $350 USD, the specialized software was approximated around $4000, and general hardware costs was appraised at around $600 based on ready-made kits that can be purchased online. The research, learning, and practice efforts were gauged at a rate of $35 USD an hour at eight hours of work time a day, with a week of preparation and a week of necessary background research. Another variant of this calculation judges a month of time spent performing background research and practice. This causes a range of around $1500 to $5900 based on the assumed time requirement. Further estimates on knowledge and skillset demands were done via comparison of courses one could obtain and measured against the material covered by the instruction. Overall the combined assumed knowledge required for physical, software, and social access was found to be around $10,000 USD. It is true that one could amortize the expenditure of developing skills across various attack scenarios, but our work assumes that this payment would need to be made each time as we presume an ignorant attacker or at least one that would want to perfect the attack prior to attempting successful exploitation. In addition to our researched estimates, we leverage the approximation of attacker cost given by the vuldb.com maintainers. All of this information is averaged to produce an estimate of $10,058 for successfully attacking CVE-2018-3652.

5.1.2 CVE-2014-2361 deep dive

As with the previous vulnerability, we begin with an in-depth examination of the vulnerability’s description. OleumTech WIO DH2 Wireless Gateway and Sensor Wireless I/O Modules, when BreeZ is used, do not require authentication for reading the site security key, which allows physically proximate attackers to spoof communication by obtaining this key after use of direct hardware access or manual-setup mode. The most glaring difference between CVE-2018-3652 and the CVE-2014-2361 vulnerability is the requirement that an individual has to spend a certain amount of time, effort, and money in order to implement the OleumTech BreeZ protocol.

The BreeZ Configuration Software is free, but including a bare minimum laptop, construction and case expenses, and making an assumption that internet service would further require an annual contract, the potential cost will range from $375 to $496 USD. In addition to the software platform, one may potentially need to purchase additional OleumTech products, including an OTC Gateway, OTC Transmitters, I/O Modules, and an all-in-one configuration cable [21]. Adding everything together the overall costs in hardware equipment can be estimated at $800 USD.

Examination of the CVE entries (e.g., MITRE, NVD) for mitigations does not lead to anything of value being found. Expanding the search to include the larger public domain for “vulnerability mitigation” suggestions related to CVE-2014-2361 finds recommendations by cisa.gov [5] to protect against this weakness. One option is to simply obtain the updates created by OleumTech for both BreeZ sensors and the gateway, all of which can be acquired free of charge from the OleumTech website. The other options are to perform additional defensive measures to protect against the vulnerability. These include minimizing network exposure, placing the system elements behind firewalls to further isolate them from business networks, and to implement VPNs or other secure methods of remote access. However, in the last option, one must recognize that any secure method solution may have its own challenges and vulnerabilities. The cost estimations for these best practices and education of users may vary depending on a plethora of potential expenses. These can include implementation of recommended procedures to protect deployed systems and educating users against social engineering attacks to better protect remote access credentials.

Lastly, we perform an attacker-centric evaluation of vulnerability CVE-2018-3652. To attack this type of OleumTech system deployment, a malicious actor may have to repeat the same hardware acquisition costs detailed above to learn how to attack the system. Along with these raw equipment costs, additional expenditures of time and effort could be required to improve the attacker’s skill level in a variety of areas. The main cost to both the attacker and defender in this scenario is the need for acquiring the specialized hardware and then learning the system deployment well enough to effectively exploit the vulnerability.

Following the same approximation process we find that the specialized equipment costs come to around $1600 USD and the expenditures of time, research, and learning come to the same range of $1500 to $5900. The estimations for skill training and general hardware exploitation costs were reckoned as $10,000 and $600 respectively. In total the cost of attack against this vulnerability was appraised at $10,414 for successful exploitation of CVE-2014-2361.

5.1.3 Deep dive summary

Table 3 gives a summary of the values generated for three server design elements using the deep dive examinations as detailed above.

Table 3 Table of SMART variables generated from vulnerability deep dives

Leveraging breach report research [12, 33] and blog posts [28, 29], we are able to estimate the impact (loss) to the defender and the value (gain) to the attacker. Note that these values can greatly vary between the two roles. For example, the potential gain for an attacker selling Social Security Numbers (SSNs) may only be a dollar [28], but the individual whose SSN has been sold may have to spend approximately $160 to correct the issue [12]. Another point to highlight is that the attacker can obtain value not only from the Personally Identifiable Information (PII) but from the production of the attack and potentially selling the attack itself. Assuming the IoT home has five individual’s records and combining this with the information collected from breach research, we arrive at the following values for Impact and Attack Value.

• Impact: $19,360.

• Attack Value: $11,600.

Note that we chose a more profitable scenario for the attacker where they can use the PII to create passports rather than simply selling SSNs.

5.2 Evaluating the IoT home

Having performed the necessary detailed background research for the hardware vulnerabilities and an exemplar software vulnerability, one can follow the process outlined in Algorithms 1–3 to produce the costs associated with required SMART variables. As was seen in the aforementioned examples, there can be a great degree of investigation required to understand the context of the vulnerability, related expenditures for equipment, and additional time or replication of the intended target. This means that a complication of this framework is having to either accept the values provided within the framework (e.g., Impact, Cost of Attack, Cost of Implementation) or spend the time adjusting the values to meet their circumstance.

Figure 2 provides an illustration of the basic IoT Home that visualizes the model design space being explored by GTS. Before we can implement the GTS framework to explore and evaluate the intended design space, one first has to determine the user-defined attributes that are fed into GAM. We begin by determining the specifics of the model that will be used to generate the substructures illustrated in Fig. 4. First, is to have a general sense of the larger system to be generated, along with a high-level concept (e.g., Fig. 2) of how the system should function. Additionally one needs to perform the background research outlined in Algorithms 1–3 in order to establish more accurate estimations of cost used by SMART. Once these initial pieces of information are known then the GTS framework can be used to explore the Security Risk represented. Table 4 shows the results from the IoT Home variants discussed at the beginning of Sect. 5. A key point to note about the results for design number eight is that there are two occurrences of CVE-2014-2361. The reason for this occurring is that both the sensor and server elements can be considered vulnerable to this CVE when implementing the BreeZ protocol. Meaning that when TAMSAT is associating vulnerabilities to design components the tool recognizes that both elements contain a BreeZ bus as part of their description and thus associates the vulnerability to both elements. This is acceptable since it adds additional potential attack paths to the evaluation and the purpose of GTS is to determine all associated security risk to the design.

Table 4 Running eight IoT home model generations and evaluations with GTS

For the purpose of further interpreting the results of Table 4, we present Fig. 10 to show two of the more complex designs side-by-side. Each of these provide an illustration of the model’s attack landscape and show the potential attack paths that SMART analyzes. As one can see in both figures there are additional Entry/Exit points caused by the hardware vulnerabilities in the server and database elements. The main contrast between the two being the difference in the server’s hardware vulnerability (i.e., CVE-2018-3652 or CVE-2019-6260). Note that the “baseline” vulnerabilities are still present in the design and that the additional vulnerabilities are due to the variations in GAM’s model generation.

Fig. 10

Illustration of the attack trees for IoT home design 06 & 07 side-by-side

Some key points to keep in mind about the IoT Home models examined are as follows:

• Certain vulnerabilities were chosen due to their hardware specific nature.

• Only specific components (e.g., server, sensors, database) are given to have these hardware vulnerabilities. That is not to say that the other elements could not have their own physical or hardware vulnerabilities, simply that in the case of our examination we limited the effect.

• Our cost estimations are based on the discovery of publicly available knowledge and does not depend on proprietary information.

The reason for needing to outline these topics is that they help color our viewpoint when examining the results from Table 4.

Examination of the GTS evaluation results tells us some key observations about the IoT Home design. First is that in the majority of scenarios, except for scenario number eight, the attack paths presented within each IoT Home do not present a favorable gain to the attacker. Hence, the Security Risk associated to these “subpaths” are all zero. The one scenario where this does not happen is in scenario number eight where a MySQL database of version 8.0.19 is used. One major difference for this scenario is the increased number of vulnerabilities associated with the MySQL database implementation. Unlike the other IoT Home models, this variant includes a much larger swath of potential vulnerabilities meaning a greater chance for an attack path that would be beneficial to the attacker. The GTS framework returns the set of CVEs related to that favorable attack path as well as the largest Security Risk contribution seen. One should notice, however, that the minimum risk path has the same zero value since this path would also be of no value to the attacker. This means that we can interpret the aggregate security risk as the monetary costs required of the defender to put the system in place.

$$\begin{aligned} p_a = {\left\{ \begin{array}{ll} 1 - e^{-\alpha (p_s*A - c_a)},&{} \quad \text {if } p_s*A \ge c_a \\ 0,&{} \quad \text {otherwise} \end{array}\right. } \end{aligned}$$

(1)

Now this raises a new set of interesting questions, such as what are the associated costs that would cause the design to become a greater risk, and can we leverage the framework to tell us which design variations would provide the smallest cost but within acceptable risk tolerances? While the GTS framework has not been specifically designed to perform iterative assessment of the cost values used, this is a relatively simple alteration to make. In its current form, SMART can output the final calculation variables used in Eq. 1, which represents the probability that an attacker will attempt to exploit a given system [37]. From this rough output, an individual could backtrack their way to the border of values that would increase the risk of attack, however this would become a non-trivial task when dealing with hundreds of vulnerabilities (e.g., Scenario Eight).

6 Discussion

Having completed a preliminary walk-through of the framework through implementation of a basic IoT home model, we move discuss the GTS framework as whole. Our framework is intended to help improve the incorporation of secure design as far left in the Design V process [22] as possible, ideally from the conceptual design stage forward. However, there is a need to review the user skills-set required for different levels of interaction with the tools within the framework, how one can go about upkeeping the databases and related values used by GTS, as well as the current limitations of our solution. These subsections provides an examination of these concerns as well as discussion of each, followed by review of the future development of this research.

6.1 User knowledge/skills

The framework aims to lower the “bar for entry” for performing design exploration and risk analysis, however there are still requirements for the user of GTS that greatly depend on their interaction with our tool suite. The most basic level of knowledge required for the individual using the framework is that they understand the components they wish to implement within the design, the AoI that serves as the critical aspect of the design, and the high-level “black-box” representation of the end-product design along with its internal architectural blueprint. While this basic information will allow GTS to perform the design exploration, this same user input allows for more granular control relating to the brand type of components, the I/O that each component is capable of, and any hardware constraints or restrictions that might apply to the desired design. The reader should note that at this level, there is a degree of acceptance that the user is making relative to the existing values within the internal databases of the GTS tools. Despite this, so long as users of the framework perform the risk analysis with the same initial database values their results can be meaningfully compared and contrasted [27].

To better improve the results of GTS requires a deeper knowledge by the individual using our tool suite. The depth of the required skill-set depends upon the degree of alteration or customization that the given individual requires. For example, if the user demands a more accurate risk analysis that is representative of their own scenario, then update of the related expenses (e.g., Algorithms 1, 2, and 3) is necessary. This means that the individual will have to be familiar enough with not only the costs, but any other contextual information related to the design or related constraints. In addition to these concerns, one would need a degree of understanding of the underlying framework logic and capabilities. This becomes especially true in the context where an individual is improving or actively building upon the software/hardware vulnerability analysis. This level of effort is seen as acceptable, since information required by GTS can be proprietary or otherwise closely guarded. Therefore it may not be possible to obtain this information openly, or the tool-suite may be used by an organization for internal purposes and thus not be shared with the public. To that end we provided documentation on methodologies that can be used for estimation of these protected values. Further discussion on the upkeep of GTS is given in the following sub-section.

6.2 Framework upkeep

One key aspect of using the GTS framework is the requirement for user upkeep of the framework, especially in the case of an individual customizing the internal databases for their specific scenario. The simplest, and most direct, method of upkeep for the GTS framework is through updating of the CVE data that is directly incorporated into the tool suite. This action requires an individual to obtain the pertinent NIST JSON Vulnerability feeds [20], unpack the Zip files, and incorporate this updated information into the respective SMART databases. Fortunately we have developed a simple script that performs these actions for the user that will be released along with the rest of the full code for GTS. The regularity with which this action must be taken solely depends on the individual making use of the tool, and will most likely be one of the first steps taken by an individual prior to using the framework for a batch of design explorations. The next level of in-depth maintenance labor required by GTS is alteration of the component descriptions used by the tools or updating of the expense information relating to the attacker and defender roles modeled within the framework. This aspect of updating and monitoring the tool is heavily manual since this work would require a greater degree of user knowledge and skill to represent properly; as mentioned in Sect. 6.1. Revising the device and system definitions (introduced in Fig. 3) does require the individual to understand the software and hardware constraints of each component, via representation of their I/O list and bus connections. However, the more involved improvement of the tool suite is updating of the expenses relating to defender and attacker costs against the produced designs. For this reason Sect. 4, and more specifically Sect. 4.2, are presented to the reader to illustrate the process required by an individual to estimate these values. This is central in the scenario where the user does not have access to proprietary or guarded organizational data to properly represent these costs. Nonetheless, these methodologies and related discussions are presented in a straightforward manner that allows the reader to feel comfortable projecting potential costs that more specifically portray their situation.

In addition to this, as discussed in Sect. 6.1, there is the ability of an individual to further improve the framework via code alterations of the tools’ functionality to augment the design exploration and vulnerability assessment logic. This is the most complicated level of upkeep present within the framework. As with any automated framework, especially within the realm of cyber security and risk analysis, while an individual can blindly make use of the tools present, their experience and results are vastly improved by understanding the underlying logic, assumptions, and limitations. Not only does this help with identification of false positives or negatives, it also allows for enriched use of the tool suite and tailoring towards the intended desire of the user. For GTS, this requires an understanding of the Python coding language along with familiarity of the concepts outlined within Sect. 3. Although, this aspect of framework modification and adjustment does require a SME, this is seen as acceptable since the same is required when using any automated tool for vulnerability or exploit enumeration/analysis.

6.3 Limitations

The current limitations present within the GTS framework ranges from the lack of access to standard cost/expense database resources to the rudimentary logic used for assessing component vulnerabilities and their effect upon the larger design risk. Instances of these limitations include how the attacker’s mindset is mathematically represented within SMART, the method by which by which expenses (both impact and gain) are estimated, and how mature the logic is for coupling together software and hardware vulnerabilities into a single attack path. However, these limitations are by no means exhaustive of the complications that can arise when analyzing risk. Additional complications that can arise when evaluating the defender and attacker standpoints. These vary from obtaining accurate potential gains for black market information exchange to constraints of cost and design that might be required by the designer of the system. Further complications arise when tackling the end-user that will be interacting with the system. Other research [19] has been performed to evaluate how a business minded examination of risk changes based on how risk-averse the user of given system is. However, GTS does not incorporate this variation since it assumes that the user of the end design would not actively seek risk within their implemented solution.

Another concern that an individual might about the limitation of our framework relates to the choice of our tool suite to represent risk as a monetary value. One may attempt to think of a greater diversity of risk representation such as “loss of service”, “loss of reputation”, “health and safety” concerns, or even the more direct “loss of customers”. However in all of these cases the base concern boils down to a negative impact for the individual performing the risk analysis. As exemplified by Adam Smith “It is not from the benevolence of the butcher, the brewer, or the baker that we expect our dinner, but from their regard to their own interest” [26]. Essentially, one is not performing the risk analysis for some act of public benevolence but rather as a form of assurance that the individual performing the analysis is sure of their product. For this reason we can ameliorate our representation of these risks as a monetary value, since this is the most simple representation of commodity that represents this value relative to any given individual(s). For instance, in the case of “health and safety” one is concerned with the health and safety of the individuals using the system, but this is only because any negative outcome can directly effect the interest(s) of the individual or organization that produced the product. In other words, the underlying concern is whether customers will continue to purchase the produced IoT design or more personally, if the individual responsible for producing the designs will find continued employment. This does not include the additional monetary costs that an individual or organization may need to expend in-order to recover any loss in faith or reputation with their consumer base.

6.4 Future development

Future work includes... Future development of the framework is focused on expanding the capabilities and adaptability of GTS as a design exploration and risk analysis tool. The first step is to expand the design space explored by providing a more open-ended presentation of the intended IoT Home design. Rather than presenting results on a limited set of eight potential designs, we will run the tool solely based on the more vague user-supplied input to the framework. Preliminary testing has shown that GAM produces around 140 potential IoT Home solutions. Following this, our research focus is the augmentation of the vulnerability assessment within the provided system design space. This would further incorporate decision logic into how both software and hardware vulnerabilities are identified and coupled within potential attack paths. Most importantly, towards the end of this research we aim to create a more comprehensive set of public databases that can be used for continued design risk analysis by the cyber security design and risk community. Our aspiration being the continued development of the tool from a practical use standpoint.

7 Conclusion

The GTS framework provides a suite of tools within an automated framework that can explore design spaces and evaluate their associated Security Risk. These risk assessments are provided in a clear and definite metric that can be easily understood by any user. There are still shortcomings of the framework that will need to be addressed. One such issue is the need for an effective language engine for interpretation of CVE vulnerability details. This would lessen the requirement of a Subject Matter Expert and thereby allow a greater range of adoption for the GTS framework. Additionally, there is a clear need for a standardized database of defender and attacker costs associated to the system components and vulnerabilities. A large amount of time and effort was necessary to perform the deep dive examinations of the vulnerabilities outlined in Sect. 5.1. A centralized knowledge base would overcome this massive time sink and allow for easier adaption of the framework to whichever design space desired. Limitations of the current framework are mainly related to the lack of standardized databases of expense information (i.e., impact and gain from successful attacks). The largest contributor being the fact that risk related information tends to be kept secret or private as a form of protection/obfuscation. The remaining limitations are related to the decision logic that determines the effect of user constraints as well as how different vulnerabilities affect the potential attack paths within the design.

The future work of this research moves from enhancements of the GTS toolset to expansion of other disciplines that would aid in its use. To further validate the results of the GTS framework, we plan to simplify the user input to a single file and then use this input to fully explore a user-defined design space. Through the larger design exploration not only can we verify the effectiveness of the tool suite, but we can expand the way in which GTS addresses constraints of the design as well as improve the examination of attack paths and their influence the overall risk. Improvements to the tool would allow for backtracking of specific risk costs that would make models vulnerable rather than just evaluating generated variants. This would provide verbose output from SMART to allow an individual to trace the risk calculations performed by the framework. Through examination of this information one could determine the precise constraints that transition each model from being too risky to within acceptable risk tolerances. Additionally, existing capabilities of searching for hardware vulnerabilities, accounting for implementation versions, and further constraining via mitigation incorporation would improve the effectiveness and adoption of the framework. Creation of a linguistic engine would aid not only in the deep dive research of each vulnerability but also aid in the conversion of more complex user requirements into meaningful input to the GAM tool (e.g., user-defined attributes). In that same vein, the establishment of additional automated tools for the estimation of attacker and defender costs would further simplify the GTS process. Altogether, these enhancements would not only refine GTS but also help the security community at large. Once this improved version of the total framework is completed, we plan to release the full code and example user input on Github.

The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.