Journal of Systems Architecture

Augmented Reality (AR) technologies are used as human–machine interface within various types of safety-critical systems. Several studies have shown that AR improves human performance. However, the introduction of AR might introduce risks due to new types of dependability threats. In order to avoid unreasonable risk, it is required to detect new types of dependability threats (faults, errors, failures). In our previous work, we have designed extensions for the SafeConcert metamodel (a metamodel for modeling socio-technical systems) to capture AR-related dependability threats (focusing on faults and failures). Despite the availability of various modeling techniques, there has been no detailed investigation of providing an integrated framework for risk assessment in AR-equipped socio-technical systems. Hence, in this paper, we provide an integrated framework based on our previously proposed extensions. In addition, in cooperation with our industrial partners, active in the automotive domain, we design and execute a case study. We aim at verifying the modeling and analysis capabilities of our framework and finding out if the proposed extensions are helpful in capturing system risks caused by new AR-related dependability threats. Our conducted qualitative analysis is based on the Concerto-FLA analysis technique, which is included in the CHESS toolset and targets socio-technical systems.


Introduction
Several studies have shown that Augmented Reality (AR) technology contributes to human performance [1]. The combination of the AR technology and humans constitute an AR-equipped socio-technical system. We focus on AR-equipped socio-technical systems because of the context of the ImmerSAFE project [2] and also due to the increased AR applications. AR technology superimposes virtual and computer generated information on the reality of the user [3]. The information can be visual, auditory, etc., for enhancing human capabilities [4]. An example of visual augmented reality is using navigational information superimposed on the windshield of a car for driver guidance.
In some cases the inclusion of the AR technology might undermine user reaction. For example, it can increase cognitive-processing load [4] and it would lead to new risks. Thus, exploiting AR in socio-technical systems demands risk assessment to make sure that it is not harmful for people and the environment. To assess risk of socio-technical systems equipped with augmented reality, it is required to identify new uncertainties, threats and their propagation.
According to ISO 26262 [5], the automotive standard for functional safety, risk assessment is a ''method to identify and categorize

Background
This section provides essential background information onto which our work is based. First, CHESS framework is introduced. Then, SafeConcert metamodel and AR-related modeling extensions are presented. FPTC syntax and Concerto-FLA analysis technique are also explained. Finally, ISO 26262, SOTIF, SEooC and SAE automation levels are presented.

CHESS framework
CHESS framework [13] provides a methodology, a language and a toolset for developing high-integrity systems.
The CHESS methodology, which is component-based and modeldriven, is based on an incremental and iterative process. Based on this methodology, components are defined incrementally with functional and also extra-functional properties, such as dependability information [17]. Then, developers can use a set of analysis techniques and back propagate the results iteratively.
CHESS-ML (CHESS Modeling Language) [18] is based on UML and provides the modeling elements required for modeling high-integrity systems.
CHESS toolset includes a set of plugins for code generation and provides various analysis capabilities. For example, Concerto-FLA (Failure Logic Analysis) [14] is a plugin related to analysis. In Concerto-FLA, component-based model of the system is provided and dependability information is used for decorating components. Then, analysis results can be back propagated to the system model. In this paper, we use Concerto-FLA as the analysis technique.

SafeConcert and its extension of AR
SafeConcert [12] is a metamodel for modeling socio and technical entities in socio-technical systems. In this metamodel, which is part of the CHESS-ML modeling language [18], technical (i.e., software, hardware) or socio entities can be modeled as components/composite components in component-based systems representing socio-technical systems. SERA taxonomy [19] is used for modeling human and organization, which are the socio entities of the system. In this metamodel human sub-components are modeled based on twelve categories of human failures including failures in perception, decision, response, etc.
In [9], we extended the human modeling elements based on AREX-Tax, which is an AR-extended human function taxonomy [7]. This taxonomy is obtained by harmonizing about six state-of-the-art human failure taxonomies (Norman [20], Reason [21], Rasmussen [22], HFACS (Human Factor Analysis and Classification System) [23], SERA (Systematic Error and Risk Analysis) [19], Driving [24]) and then extending the taxonomy based on various studies and experiments on augmented reality. These extended modeling elements are divided to four categories, shown in Fig. 1. Three of these categories are human functions including human process unit, human SA (situational awareness) unit, and human actuator unit. The one other category is human fault unit, which is related to human internal influencing factors affecting on human functions. We explain these modeling elements in the next two paragraphs. In the first paragraph we explain modeling elements related to human functions and in the second paragraph we explain modeling elements related to human fault unit and also other fault categories. Extended modeling elements are shown with white color and AR-stemmed modeling elements are shown with dotted line border.
The extended modeling elements in human process unit, human SA unit, and human actuator unit enable modeling of AR-extended human functions. For example, detection failure, which represents a failure in detecting human function, is a human failure introduced by several human failure taxonomies such as Reason [21] and Rasmussen [22] taxonomies. Based on experiments and studies on augmented reality including [25] and [26], detecting function would be extended to surround detecting while using AR (surrounding information would be augmented on real world view of the user by AR). Thus, surround detecting can be considered as an extended sub-component of human component; in other words surround detecting is an extended modeling element proposed for analysis of AR-equipped socio-technical systems.
In [10], we extended organization and human modeling elements based on AREFTax, which is a fault taxonomy including AR-caused faults [8]. This taxonomy is obtained by harmonizing about five stateof-the-art fault taxonomies (Rasmussen [22], HFACS [23], SERA [19], Driving [24] and SPAR-H (Standardized Plant Analysis Risk Human Reliability Analysis) [27]) and then extending the taxonomy based on various studies and experiments on augmented reality.
In [11], we proposed more specifications for organization and human modeling elements by considering digitalization, globalization and networked structure of organizations. More specifically, we extended the human and organization modeling elements based on post normal accident theory [28] and global distance metric [29]. The extended modeling elements are helpful to prevent post normal accidents and to include global distance metric while assessing risk in new AR-equipped socio-technical systems. These extended modeling elements are shown in Fig. 2   are shown with white color and AR-stemmed modeling elements are shown with dotted line border. These extended modeling elements enable modeling of various faults leading to human failures including AR-caused faults. Faults would be caused by human, environment, organization, etc. Human related faults are categorized as human fault unit of Fig. 1 and non-human faults are categorized as three categories of organizational factors including organization and regulation unit, environment unit and task unit. For example, failure in physical state of a human is a human internal fault leading to human failure. This is shown as human modeling element in human fault unit category shown in Fig. 1. Another example is condition, which is a non-human factor and it is categorized as extended modeling elements for organization components shown in Fig. 2. One example of the AR-extended modeling elements is social presence shown in Fig. 1. Based on studies on augmented reality [30], using AR would decrease social presence and failure in social presence can be considered as fault leading to human failure.

The FPTC syntax
FPTC syntax was proposed as part of FPTC analysis technique [15]. FPTC rules are set of logical expressions that relate output failure modes to combinations of input failure modes in each individual component [31].
Components' behavior can be classified as source (if component generates a failure), sink (if component is able to detect and correct input failure), propagational (if component propagates failures received in its input to its output) and transformational (if component transforms the type of failure received in its input to another type in its output).
Wildcard in an input port shows that the output behavior is the same regardless of the failure mode on this input port. NoFailure in an input port shows normal behavior.
Based on this syntax, ''IP1.noFailure → OP1.omission'' shows a source behavior and should be read as follows: if the component receives noFailure (normal behavior) on its input port IP1, it generates omission on its output port OP1.

Concerto-FLA analysis technique
Concerto-FLA [14], which extends FPTC [15], is a model-based analysis technique that provides the possibility for analyzing failure behavior of humans and organizations in addition to technical entities by using SERA [19] classification of socio-failures. As we recalled in Section 2.1, this technique is provided as a plugin within the CHESS toolset and allows users to define component-based architectural models composed of hardware, software, human and organization. This technique includes five main steps. of the failure propagations. This step is similar to FPTC technique that system architecture is considered as a token-passing network and set of possible failures that would be propagated along a connection is called tokenset (default value for each tokenset is noFailure, which means normal behavior). In order to obtain system behavior, maximal tokenset is calculated for each connection through a fixed-point calculation. 5. Interpreting the results at system level. Based on the interpretation it will be decided to do the re-design or not.

ISO 26262, SOTIF, SEooC and SAE
ISO 26262 [5] is the standard for functional safety. ISO 26262 provides the requirements and set of activities that should be performed during the lifecycle phases such as development, production, operation, service and decommissioning. ISO 26262 addresses functional safety and specifies risk assessment for risks due to malfunctioning behavior of the items. If the risk is because of intended functionality or performance limitation of a system, it is addressed in ISO/PAS 21448-SOTIF [6]. In ISO 26262, ASIL (Automotive Safety Integrity Level) is determined and used for applying the requirements to avoid unreasonable residual risk. ASIL specifies item's necessary safety requirements to achieve an acceptable residual risk. Residual risks are remaining risks after using safety measures. An ASIL value is one of four levels (A-D) and it is determined based on three factors: severity, exposure and controllability. The severity factor indicates class of severity in case of hazard occurrence and it is classified from 0 to 3 (shown by S0-S3). S3 shows the category with the highest severity and it is related to situations with life threatening injuries. The exposure factor indicates class of probability of exposure with respect to operational situations and it is classified from 0 to 4 (shown by E0-E4). E4 shows the category with the highest probability of exposure (with time in use more than 10%). The controllability factor indicates the class of driver controllability and it is classified from 0 to 3 (shown by C0-C3). C3 shows the category with the highest controllability (more than 99% of drivers can control). ASIL classification based on these three factors is shown in Fig. 3. QM (quality management) shows that no safety requirement is necessary. ASIL value A shows the lowest safety requirements and ASIL value D shows the highest safety requirements.
Safety element out of context (SEooC), introduced by ISO 26262, refers to an element that is not defined in the context of a special vehicle, but it can be used to make an item, which implements functions at vehicle level. SEooC is based on ISO 26262 safety process and information regarding system context such as interactions and dependencies on the elements in the environment should be assumed [33].
(a) Definition of the SEooC scope: assumptions related to the scope, functionalities and external interfaces of the SEooC should be defined. (b) Definition of the assumptions on safety requirements for the SEooC: assumptions related to item definition, safety goals of the item and functional safety requirements related to SEooC functionality, which are required for defining technical safety requirements of the SEooC should be defined. 2. Development of SEooC: based on the assumed functional safety requirements, technical safety requirements are derived and then SEooC is developed based on ISO 26262 standard. 3. Providing work products: work products are documents that show the fulfilled functional safety requirements and assumptions on the context of SEooC. 4. Integration of the SEooC into the item: safety goals and functional safety requirements defined in item development should match with assumed functional safety requirements for the SEooC. In case of a SEooC assumption mismatch, change management activity based on ISO 26262 standard should be conducted.
The process required for improving the intended functionality to ensure safety includes eight activities. Possible interactions between these activities and ISO 26262 activities and SEooC are shown in Fig. 4.
Safety process of the ISO 26262 standard starts with concept phase containing item definition, hazard analysis and risk assessment and functional safety concept [33]. An item implements a vehicle level function.   In item definition the main objective is defining items. Defining items requires defining the dependencies and interactions with environment. Then, related hazards are identified and functional safety requirements are obtained. In SEooC, assumptions related to system context are the main output of the concept phase. Functional safety concept includes providing functional safety requirements. Output provided by Functional safety concept is used by technical safety concept. Technical safety concept includes technical safety requirements and system design. Then, hardware and software development is done based on technical safety concept. HW/SW development is based on assumptions provided in concept phase. Next steps in the process are verification test, validation test and functional safety assessment. In SEooC, these steps require establishing validity of assumptions.
SOTIF process starts with functional and system specification, which includes functional description and considerations on system design and architecture. Then, potential hazardous events should be identified. If the harm is possible for the identified potentially hazardous events, then analysis of their triggering events should be conducted. Functional modification is the next activity for avoiding the hazards or for reducing the resulting risk. Next activities are verification and validation strategy specification and then in verification and validation activities arguments are provided to illustrate that the residual risk is below acceptable level by testing on various known and unknown scenarios.
Finally, evaluation on residual risk should be performed based on the verification and validation results and specified criteria.
Based on the taxonomy and definitions related to driving automation systems for on-road motor vehicles performing part or the entire dynamic driving task (DDT) on a sustained basis, there are six levels of driving automation. SAE level 0 refers to no driving automation and SAE level 5 refers to full driving automation [34]. These levels with description and example are shown in Fig. 5. Assessing human factor in driver-vehicle interface is not only important on lower SAE levels, but also on higher levels because of the importance of safe transition between automated and non-automated vehicle operation [35]. In order to improve safety, various scenarios of driver/vehicle interaction should be considered.

An integrated framework for assessing risk of AR-equipped socio-technical systems
Our provided framework for assessing risk of AR-equipped sociotechnical systems is based on our previously proposed modeling extensions and the Concerto-FLA analysis technique [14]. We name this framework FRAAR (Framework for Risk Assessment in AR-equipped socio-technical systems).   Our previously proposed modeling extensions on SafeConcert was recalled in Section 2.2. Concerto-FLA analysis technique [14] is also recalled in Section 2.4. Essentially, the added value with respect to SafeConcert/Concerto-FLA is the availability of modeling and analysis capabilities for modeling and analyzing various socio aspects, AR-extended human functions and AR-related influencing factors on human functions.
We use V-model structure to illustrate methodology of the provided framework. Different steps of the methodology are shown in Fig. 6.
As it is shown in Fig. 6, there are four main steps.
In the first step, we need to answer to the question of what are the involved entities in the system. Since we model the system as a component-based system, defining involved entities determines the composite components. In an AR-equipped socio-technical system, involved entities include technical (including AR) and socio entities.
In the second step, we need to identify important aspects of each entity. These important aspects are used to determine sub-components of each composite component. In this step, our proposed taxonomies and extended modeling elements explained in Section 2.2 can be helpful to have a list of important aspects. Based on scenario and the selected case study, required sub-components can be selected. For example, paying attention can be considered as an important aspect of a human driving a car. Not paying attention would lead to failure in deciding, which is a hazardous behavior that would lead to system risk.
Third step is to model the behavior of each sub-component, which should be done based on analysis of each sub-component individually. FPTC syntax explained in Section 2.3, can be used for modeling the behavior of each sub-component.
Finally, last step is analyzing system behavior, which provides system behavior based on the provided model. We do this step based on Concerto-FLA analysis technique explained in Section 2.4.
Based on the analysis results there would be feedback for changing the system design in order to decrease risk. This feedback can be suggestions for safety requirements or functional modifications.
Proposed risk assessment activities support several ISO 26262 and SOTIF development process activities, shown in Table 1. Defining involved entities in step 1 and important aspects of each entity in step 2 supports Item definition activity of ISO 26262 standard and functional and system specification of SOTIF standard. In step 1 and 2 of our proposed activities, components and sub-components are defined, Table 1 Risk assessment activities of our provided framework and supported ISO 26262 and SOTIF development process activities. which can support provision of items and functional specification. System model including all components and sub-components support provision of system specification. Provided component-based model in step 1 and 2 of our proposed framework can be used as work products expected by the standards.
Modeling important aspects of each entity, analyzing their behavior and analyzing system behavior supports hazard analysis and risk assessment (HARA) of ISO 26262 standard and SOTIF related hazard identification and risk evaluation and also identification and evaluation of triggering events of SOTIF standard. In hazard analysis and risk assessment of ISO 26262, the aims are to identify the hazards and formulating safety goals. Step 2, 3 and 4 of our proposed activities support hazard identification by modeling failure propagation and by providing analysis results of different scenarios. These results support formulating safety goals to avoid unaccepted risks. In SOTIF related hazard identification and risk evaluation, the aims are identifying and evaluating SOTIF related hazards and their consequences. Modeling and analyzing activities in step 2, 3 and 4 provide the support for identification and evaluation of SOTIF related hazards and their consequences. For example, failing to pay attention leads to deciding incorrectly, which is a SOTIF related hazard and it leads to executing incorrectly. The modeling elements, used in step 2 and 3, provide the possibility to model and analyze paying attention, deciding and executing functions. Analysis in step 4 also provides the consequences at system level. Provided model in step two, three and analysis results in step four can be used as work products expected by the standards.
Analyzing system behavior in step 4 also supports defining functional and technical safety requirements, which are used in functional S. Sheikh Bahaei et al. and technical safety concept of ISO 26262 standard and it also supports functional modification to reduce SOTIF risk of SOTIF standard. In addition, analysis results are based on considering various scenarios, which support verification test in ISO 26262 and verification of the SOTIF. Required work products for verification test in ISO 26262 and SOTIF standards can be prepared based on analysis results in step four of our proposed framework.

Case study design and execution
In this section, we design a case study to present the modeling and analyzing capabilities of the proposed framework that can be used to qualitatively analyze the risks for AR-equipped socio-technical systems.

Objectives
Our objectives include presenting the modeling capabilities and analysis capabilities of our proposed framework containing AR-related extensions. In other words, we aim at estimating the effectiveness of the provided framework in predicting risk caused by new AR-related dependability threats. In order to do that, we use an industrial case study from automotive domain to evaluate the proposed extensions. Analysis results can be used for defining related safety requirements

Research methodology
We use case study research methodology based on [36]. The steps carried out for the presented research are presented in Fig. 7. In the first step, objectives and the structure of the research are discussed.
In the second step, we asked Xylon Company for a case study in the context of augmented reality socio-technical systems. Surround view system as a case study was suggested by this company and a meeting was organized to decide about the collaboration. We also discussed about system description.
In the third step, system architecture was provided based on information provided by the company and it was reviewed in several iterations for improvement.
In the fourth step, analysis of the case study was provided based on Concerto-FLA analysis technique and it was reviewed in iterations for improvement.
In the fifth step, a discussion about results and lessons learnt was provided. Then, the results are reviewed and a discussion about validity of the work is provided.

Case study selection and description
The case study is conducted in collaboration with Xylon, an electronic company providing intellectual property in the fields of embedded graphics, video, image processing and networking.
In this study, we select as case study subject a socio-technical system containing the following entities: • Road transport organization (socio entity): representing the organization responsible for providing transport rules and regulations, proper road conditions and etc. • Driver (socio entity): representing a human who is expected to drive a vehicle and park it safely by utilizing augmented reality technology used in the surround view system of the vehicle. • Vehicle (technical entity): representing vehicle containing surround view system (a SEooC with the potential for using in vehicles with high levels of driving automation. However, currently it is used at driving automation level 0. It includes augmented reality technology to empower drivers).
Surround view systems are used to assist drivers to park more safely by providing a 3D video from the surrounding environment of the car. In Fig. 8, it is illustrated how the 3D video is shown to the driver. As it is shown in Fig. 8, driver can have a top view of the car while driving. This top view is obtained by compounding 4 views captured by 4 cameras mounted around the car and by changing point of view. It is like there is a flying camera visualizing vehicle's surrounding, which is called virtual flying camera feature. A picture of a virtual car is also augmented to the video to show the position of the car. Navigation information and parking lines also can be annotated to the video by visual AR technology. The current surround view system is a SEooC of driving automation level 0. However, Xylon plan to develop automated driving system features in higher levels for the future versions of the system.
Assumptions on the scope of the SEooC are: • The system can be connected to the rest of the vehicle in order to obtain speed information. In case of drawing parking path lines, steering wheel angle and information from gearbox would also be obtained to determine reverse driving.
Assumptions on functional requirements of the SEooC are: • The system is enabled either at low speed or it can be activated manually by the driver. • The system is disabled either when moving above some speed threshold or it can be deactivated by driver.
Assumptions on the functional safety requirements allocated to the SEooC are: S. Sheikh Bahaei et al. Fig. 9. Integration of the human, organization and vehicle effective aspects.
• The system does not activate the function at high vehicle speed automatically. • The system does not deactivate the functionality at low speed automatically.

Case study execution: System modeling
This subsection reports on how we model the described system in Section 4.3 using our proposed framework. Section 4.3 provides the required information for the first step of the risk assessment process defined in Fig. 6, which is identifying the entities for defining composite components. Based on the selected case study explained in Section 4.3, organization, driver and vehicle containing an automotive surround view system are three composite components of this system. In this subsection, we provide information for the second and third steps of risk assessment process.
Important aspects of each entity are modeled as sub-components of each composite component. For socio entities, the important aspects are selected from extended modeling elements explained in Section 2.2 and for vehicle, which is a technical entity the important aspects are based on system description.
• Important aspects of road transport organization (selected from Fig. 2): -Organization and regulation AR adoption: it refers to upgrading rules and regulations of road transport organization based on AR technology. -Condition: it refers to road condition.
-Monitoring and feedback: it refers to the monitoring task and feedback provided by organization.
• Important aspects of driver (selected from Fig. 1): -Surround detecting : it is an AR-extended function, because driver can detect surround environment through AR technology. -Supported deciding : it is an AR-extended function, because driver can decide with the support of AR technology. -Executing : it is human executing function.
-Interactive experience: it is an AR-caused factor, because AR provides interactive ways for enhancing user experience. -Social presence: it is an AR-caused factor, because AR may decrease social presence and lead to human failure.
• Important aspects of vehicle containing surround view system (selected based on system description received from Xylon Company): -A set of speed sensors: each sensor is a hardware for providing speed of the vehicle based on its movement. -A set of cameras: each camera is a hardware for providing raw data for a video receiver. Usually there are four cameras that can be attached to four sides of the car. -Switch: switch is a hardware for receiving on/off command from driver. It is also possible to send on/off command automatically based on driving requirement. -Peripheral controller: peripheral controller includes hardware and driver for receiving user inputs such as on/off command and speed and for sending them to user application implementation. -A set of video receivers: each video receiver includes a hardware and a driver. Its hardware is used for transforming raw data to AXI-stream based on the command from its driver implementation. -Video storing unit: video storing unit includes a hardware and a driver. Its hardware is used for receiving AXI-stream and storing it to the memory by means of DDR memory controller based on the command received form its driver. -DDR controller: DDR controller is a hardware for accessing DDR memory, which stores video in DDR memory and provides general memory access to all system IPs. -Video processing IP: Video processing IP includes hardware and driver for reading prepared data structures and video from memory, for processing video accordingly and finally for storing the processed video to memory through DDR controller. The prepared data is stored to memory by video processing IP driver based on the data structures received from memory. -Display controller: Display controller includes hardware and driver for reading memory where processed video is stored and for converting it in the format appropriate for driving displays. -Processing unit: processing unit includes hardware and software, which its software contains all the software and drivers of all other IPs. The software also contains user application implementation and video processing engine implementation. User application implementation receives inputs from peripheral unit and controls operation of all S. Sheikh Bahaei et al. Fig. 10. AR-equipped socio-technical system modeling.
IPs by means of their software drivers. Video processing engine implementation prepares data structures to be stored in DDR memory through DDR controller. Fig. 9 provides an overview of the integration of the human, organization and some of vehicle important aspects. In Fig. 10, we show how this AR-equipped socio-technical system is modeled using SafeConcert AR-extended modeling elements. Driver is composed of five sub-components. Driver has four inputs and two of its inputs are from system inputs with the names human detection input (HDI) and human communication input (HCI). Two other inputs are from organization and surround view system. We consider interactive experience and social presence as two sub-components of human component, which are influencing factors on human functions. Interactive experience affects on supported deciding and is affected by surround detecting. Social presence affects on human executing. Driver output, which is output of the system is human action shown by HA.
Organization and regulation AR adoption, condition and monitoring and feedback are three sub-components of organization composite component. Organization component receives input from system, which represents influences from regulation authorities on the organization (REG).
Vehicle is also modeled with three inputs including user command shown by CMD, vehicle movement shown by VMV and camera input shown by CAM. Green color is used to show the extended modeling elements used in this system.

Case study execution: System analysis
This subsection reports on the analysis of the system using ARrelated extensions, which refers to the last step of the risk assessment process defined in Fig. 6. We follow the five steps of Concerto-FLA analysis technique explained in Section 2.4 for system analysis. Fig. 10. We explained how the system is modeled in Section 4.4. In scenarios, we may change some components' failure behavior to source based on assumptions related to that scenario. For example, if we assume that an AR-related component is producing failure, then we need to change its failure behavior to source and update its FPTC rules. 3. Third step is to consider failure modes in inputs of the system to calculate failure propagation. In this case study, we inject noFailure to four inputs of the system, because we aim at analyzing system for scenarios that failure is originating from our modeled system and more specifically from our AR-related part of the system. 4. Fourth step is calculating the failure propagations. We consider three scenarios and show the analysis results in Figs. 15-17. 5. Last step is back propagation of results. Interpretation of the back-propagated results can be used to make decision about design change or defining safety barrier, if it is required.

Scenario 1:
• Description of the scenario: In this scenario, we assume that failure in the system is emanated from the technical part of the system. We assume video processing IP produces processed video incorrectly. For example, we assume that the expected visual mark for parking lot striping is assigned on an incorrect position (value failure mode). As a consequence, the driver cannot detect the surround environment correctly and decides and executes incorrectly (value failure mode). • Modeling failure behavior: We show the failure propagation with underlined FPTC rules, which are the rules that are activated, shown in Fig. 15. In this scenario, video processing IP behaves as source and while its inputs are noFailure, it produces valueSubtle failure mode in its output. This activated rule is shown on its sub-component. DDR controller, display controller and display sub-components behave as propagational and propagate valueSubtle from inputs to outputs. In this case, unintended displayed information is the identified hazard and the reason is failure in video processing IP. System failure in this scenario would lead to light accident and light injuries. The reason is that the speed is not usually high while parking the car. Based on the explanation in Section 2.5 and Fig. 3, severity in this case is S1. Class of exposure is E4, because probability of exposure is more than 10%. It means that it is more than 10 percent probable that a driver be exposed to parking S. Sheikh Bahaei et al.  situation while driving a car. Finally, class of controllability is C1 or normally controllable. It means that more than 90% of drivers can control this situation. Therefore, ASIL level for this case is A, based on Fig. 3. If we aim at overcoming risks with ASIL level A, then we should define safety goal, functional and technical safety requirements in order to overcome this risk. For example, for this scenario safety goal, functional safety requirement and technical safety requirement can be defined as follows to prevent failure in processing unit IP: -Safety goal: The driver shall be notified, if there is failure in processing. -Safety requirement: * Functional safety requirement: A monitoring component should be used to check the processing actively. * Technical safety requirement: Monitoring function should check the processing output every 10 ms.
After interpreting the results and providing safety requirements, system design would be updated. Then, failure behavior can also be updated and failure propagation analysis can be repeated for another iteration.

Scenario 2:
• Description of the scenario: In this scenario, we assume that the technical part of the system works without failure, but driver does not have interactive experience. For example, it is the first time driver is working with systems containing AR and he/she cannot understand the meaning of AR notations. Therefore, driver would decide and execute incorrectly. • Modeling failure behavior: We show the failure propagation with underlined FPTC rules, which are the rules that are activated, shown in Fig. 16. Surround view sub-components behave as propagational and propagate noFailure from inputs to outputs.
Interactive experience behaves as source and while its input is no-Failure, it has omission failure mode in its output. This activated rule is shown on this component. In this scenario, we considered failure in AR-related part of the system and since it refers to limitation in intended functionality (SOTIF related hazards), we do not determine ASIL level. If the expected severity and controllability of the scenario is higher than S0 and C0 respectively, we need to consider SOTIF safety process [40]. As we explained in the previous scenario, severity and controllability are higher than S0 and C0. Lack of interactive experience leads to system failure and incorrect deciding is the identified hazard. Safety goal and safety requirement can be defined as follows. Since the failure is not emanated from technical part of the system, we do not need to specify technical safety requirement: -Safety goal: Interactive experience shall be provided for the driver. -Safety requirement: The Company should provide a training video for all drivers at the first time of using the system.
After applying the requirements the behavior of this component would change from source to other types and analysis can be repeated. It is not possible to detect risk originated from failure in interactive experience, without using the proposed representation means, because using these representation means or modeling elements provide the possibility to analyze their failure propagation and provides the possibility to analyze effect of these failures on system behavior. Then based on analysis results decision about design change or fault mitigation mechanisms would be taken.

Scenario 3:
• Description of the scenario: In this scenario, we assume that road transport organization has not updated rules and regulations based on AR technology, which is a limitation in intended functionality. For example, parking lot striping is not updated to be used by AR applications and it affects on road condition, but monitoring and feedback component detect this problem and provide a feedback to driver. This feedback would be a visual text alarm showing that there is a problem in AR information. Therefore, driver will not depend on shown result and try to decide and execute correctly. • Modeling failure behavior: We show the failure propagation with underlined FPTC rules, which are the rules that are activated, shown in Fig. 17. Similar to the previous scenario, surround view sub-components behave as propagational and propagate noFailure from inputs to output. Organization and regulation AR adoption behaves as source and while its input is noFailure, it has omission failure mode in its output. This activated rule is shown on this component. Monitoring and feedback component behaves as sink and while its input is omission, it has noFailure in its output. • Analysis of system behavior: Omission failure mode propagates from organization and regulation AR adoption to condition and monitoring and feedback. In monitoring and feedback it will transform to noFailure. Then, noFailure is propagated from surround detecting to interactive experience, supported deciding and executing. • Interpreting the results: In this scenario, system output is provided without failure. Thus, there is no hazard and no safety requirement is required.

Compliance with ISO 26262 and SOTIF
Based on the explanation in Section 2.5, the first step of ISO 26262 safety process is item definition and the first step of SOTIF safety process is functional and system specification. In Fig. 10, we defined the components which are used for modeling items, their interactions and dependencies. We also specified system and functions through entities specification.
The second step of ISO 26262 is hazard analysis and risk assessment and second step of SOTIF is SOTIF related hazard identification and risk evaluation. Based on the interpreted results of each scenario, hazards are identified (if there are) and categorized based on ASIL level, if they are emanated from technical failures, otherwise they are evaluated qualitatively. If the hazard is emanated from a technical fault, functional safety is addressed by ISO 26262, otherwise it is addressed by SOTIF.
The third step of SOTIF is identification and evaluation of triggering events. Sub-components in Fig. 10 are the identified potential triggering events and failure behavior of each of these sub-components in Figs. 11-14 are evaluation of the triggering events.
The third and fourth steps of ISO 26262 are functional and technical safety concept and fourth step of SOTIF is functional modification to reduce SOTIF risk. The aim in the two steps of ISO 26262 is to define functional and technical safety requirements. Defining functional and technical safety requirements should be based on the analysis results as explained in the first scenario. Functional modification is also provided based on the analysis results as explained in the second scenario.
Finally, verification test of ISO 26262 and SOTIF includes considering several scenarios and verifying system functioning. This step is supported by the provided analysis results.

Lessons learnt
In this section, we present the lessons learnt while conducting the case study. As it is shown in the second and third scenarios in Section 4.5, in an AR-equipped socio-technical system, there are system failures which are not caused by technical entities of the system and new AR-related dependability threats are the reason for these system failures. These new AR-caused dependability threats are related to intended functionality of socio entities of the system. Our proposed framework provides the required means to take into account these new AR-related dependability threats. We can consider these extensions from two perspectives: • Augmented reality concepts coverage: from a coverage point of view, as shown in Section 4.4, modeling capabilities obtained by our proposed framework, allow architects and safety managers to model augmented reality effects on socio-technical systems by using modeling elements related to AR-extended human functions as well as modeling elements related to AR-caused faults leading to human failures. For example, in the second scenario, failure in interactive experience is considered as an AR-related dependability threat and its modeling element provides representation mean for taking into account AR effect as an AR-caused fault leading to human failures. In the third scenario, failure in updating rules and regulations based on AR technology is considered as ARrelated dependability threat and its modeling element provides representation mean for taking into account AR effects. It is also shown in Section 4.5 that analysis capabilities allow architects and safety managers to have at disposal means to reveal effect of AR-related dependability threats on system behavior. It is done by analyzing failure propagation that might be effective in emerging risks within an AR-equipped socio-technical system. • Expressiveness: Expressiveness refers to the power of a modeling language to express or describe all things required for a given purpose [41]. Set of symbols or possible statements that can be described by modeling languages can be used for measuring expressiveness. Statement means ''a syntactic expression and its meaning''. As it is explained in Section 2.2, the extensions on human modeling elements used to extend the modeling language is based on an AR-extended human function taxonomy (AREXTax [7]). This taxonomy is obtained by extracting human functions from about six state-of-the-art human failure taxonomies (Norman [20], Reason [21], Rasmussen [22], HFACS [23], SERA [19], Driving [24]). This taxonomy is also extended based on various studies and experiments on augmented reality. In addition, the extension for extending organization modeling elements is based on a fault taxonomy (AREFTax [8]) containing AR-caused faults leading to human failures. This taxonomy is gained by harmonizing about five state-of-the-art fault taxonomies (Rasmussen [22], HFACS [23], SERA [19], Driving [24] and SPAR-H [27]). The taxonomy is also extended based on various studies and experiments on augmented reality. According to the basis of the extensions and as it is also shown in Section 4.4, the extensions increase power of modeling language to express new AR-caused risks.
We used Concerto-FLA analysis technique as the basis of the analysis in order to disclose the advantages of the proposed AR-related extensions included in our proposed framework at analysis level. Concerto-FLA uses FPTC syntax for the modeling failure behavior of each component or sub-component, which includes defining FPTC rules for a component/sub-component in isolation. It is possible to define FPTC rules for the AR-extended modeling elements characterizing their behavior. In addition, as known, modeling the failure behavior can be challenging, because the number of FPTC rules grows exponentially with the increase of the cardinality of the input ports. It is important to consider possible failure modes for each input in a component/subcomponent and skip the others. It is not conspicuous in small and academic examples, but it is really challenging when we use an industrial case study.

Threats to validity
In this section, we discuss threats to validity in relation to our research based on best practices available in the scientific literature [36]. Validity of a study denotes to what extent the results can be trusted.
External validity refers to possibility of generalizing the findings. We provided a case study with three scenarios from automotive domain, but the proposed framework is not limited to specific scenarios and specific domain and the baseline for the included extensions, which are AREXTax and AREFTax taxonomies are attained from taxonomies in various domains. Thus, there is the possibility of generalizing the findings for automotive domain in general and also for other domains.
Construct validity refers to the quality of choices and measurements. In our case, we used SafeConcert, which is an accepted work, as the basis of our work. Proposed extensions are also based on stateof-the-art taxonomies (Norman [20], Reason [21], Rasmussen [22], HFACS [23], SERA [19], Driving [24] and SPAR-H [27] taxonomies) and studies and experiments for the new technologies. The modeling and analysis process is done based on standardized process to increase the repeatability of the work. However, it cannot be guaranteed that different people have same answer using our proposed framework, because it depends on the analyzer skills and ability for modeling and analysis.
In this paper we used a realistic and sufficiently complex case at a level that can be found in industry to verify our proposed framework including AR-related extensions. Although we were not allowed to access confidential information related to their customers, we have been able to model system architecture and failure behavior of system components using SafeConcert metamodel, its AR-extensions and FPTC rules.
In this case study, we illustrated the modeling and analysis capabilities of our proposed framework including AR-related extensions through three different scenarios with different assumptions about the AR-related components' failure behavior. We have not shown that the modeling elements are complete for modeling all possible scenarios. Instead, we have focused on the provided elements to check if they are able to capture new system failure behaviors.
The benefit of using our proposed extensions for a particular case depends on the ability to choose the best elements and the ability to establish failure behavior of the component related to that element. Still, this case provides evidence for the applicability and usefulness of our proposed framework. Further investigations are required to provide more beneficial results on limitations of modeling and analysis applications.

Discussion
Statistical information is used for determining exposure, severity and controllability of ASIL value of systems with SAE-levels 0-2. It would be possible to use the same statistical information for determining exposure and severity in AR-equipped systems with higher levels of automation, but controllability is a factor, which is affected by augmented reality used in higher levels of automation. Thus, it is required to model system and include effect of augmented reality on the model to be able to involve AR effect in specifying controllability factor of ISO 26262. For providing automated driving safety, Responsibility Sensitive Safety (RSS) standard [42] can be helpful. This standard provides formalization for safe decisions by self-driving cars in cases where machine learning mechanisms are used [43].
Surround view system can be mounted on vehicles with higher levels of automation (for example level 1-3) alongside more advanced systems for providing driver assistance functionalities. In these cases, driver is not supervising the car and controllability factor should be defined by modeling system as an AR-equipped socio-technical system. In [32], a controllability classification is proposed based on human takeover time and analysis of human driver models. The value of human action times, based on studies in literature, are used for predicting mean takeover times. Since classification of controllability according to ISO 26262 requires description of percentiles, normal distribution is assumed for each action time. Normal distribution can be obtained by mean value and its standard deviation. Based on the reaction times and distributions, it is possible to calculate controllability of the situation. The proposed modeling extensions included in our proposed framework provide the possibility to model effect of augmented reality on human and effect of augmented reality on influencing factors on human functions. Thus, mean takeover time and as a consequence controllability can be updated while using augmented reality by using the proposed extensions on humans and influencing factors modeling.
The generated model using our proposed framework and analysis results can be used to provide safety case for AR-equipped industrial products. Safety case contains arguments based on evidences to demonstrate that the system is acceptably safe to work on a given environment. However, it is required to provide also some documentation explaining the results and showing how the safety requirements are achieved. Goal Structuring Notation (GSN) [44] can be used for SOTIF argumentation [45].
Extended human modeling elements can be used for modeling integration of human aspects with interactive systems in system testing. For example, MIODMIT architecture [46] is a generic architecture for interactive systems. As it is discussed in [47], human aspects should be considered and integrated while testing. Using extended modeling elements for modeling different aspects of human as a user of an interactive system would be of value for the system testing.

Related work
A comparative study about architecture-based risk analysis techniques is provided in [48]. Specifically, in this work, authors compare: the modeling capabilities, process and tool support of various techniques. Traditional methods such as Fault Tree Analysis (FTA) [49] and Failure Modes and Effects Analysis (FMEA) [50] are manual analysis techniques. In comparison, there are also model-driven techniques, which provide the analysis results (semi-)automatically based on the system architecture and annotated failure behavior information. Modeldriven techniques such as Failure Propagation and Transformation Notation (FPTN) [51], FPTC, Hierarchically Performed Hazard Origin and Propagation Studies (HIP HOPS) [52] and techniques using Architecture Analysis and Description Language (AADL) and its technical error annex [53] are considered in this study. All these techniques consider risks emanated from technical parts. Human and organization are not considered as part of the system that would introduce risk.
A framework for construction safety management and visualization system (SMVS) is proposed in [54]. This framework includes a safety management process, which includes planning, education and inspection phases. A prototype system is also developed and tested. The results shown that this framework improves risk identification and communication between managers and workers in construction sites. Augmented reality is used for improving the safety management process. In comparison to our work, in this paper the proposed framework is specific to construction domain. AR is also used for safety management process improvement, but it is not considered as part of the system, which is going to be evaluated. Thus, risks emanated from AR and AR-related factors are not included in the process.
In risk analysis techniques for socio-technical systems, failures emanated from human and organizational factors are also considered in addition to technical failures. Human failure taxonomies provide the possible human failures while working in a socio-technical system.  There are also taxonomies on organizational factors that provide the factors influencing human performance. In [14], Concerto-FLA analysis technique is proposed based on SERA taxonomy including human failures and organizational factors. Human reliability quantification techniques can be used for quantifying human error probability and providing quantitative risk assessment. Expert judgment and analysis of accident reports can be used for determining likelihood. However, error likelihood estimation usually has low accuracy. We also do not aim at using quantitative assessment, because based on SOTIF standard, SOTIF related hazards require qualitative analysis.
A risk analysis technique for systems containing augmented reality, named Safe-AR, is proposed in [55]. Safe-AR integrates failures of AR/user interface at three levels: perception, comprehension and decision-making. Likely risks and their severity are based on reports available in literature. The proposed technique is shown on an AR leftturn assist app, which is an example from automotive domain. Human functions and failure modes in this study are limited to the provided example and a generalization is required to be used for other domains and more complicated case studies.
A framework for risk management in financial services is provided in [56]. The paper focuses on risk management from a human centered perspective. In comparison to our work, this paper is specific to financial domain and it does not provide a general framework. The proposed framework does not include modeling and analysis constructs to be used for risk assessment. The required activities in different steps for assessing risk are not defined specifically.
Human Functions in Safety (HFiS) framework is proposed in [57]. This framework focuses on the role of human in system safety in sociotechnical systems. Organizational factors are also considered in this framework. The output from applying the framework is a description of safety related activities through human functions, organizational goals and contextual factors. It is developed for railway context, but there are guidance for generic application of HFiS. In comparison to our work, in this paper there is no consideration on effects of new technologies such as augmented reality on human functions and organizational factors.
In comparison to the above-mentioned works, our framework provides more general risk assessment technique with the integration of risk emanated from human, organization and technology (augmented reality). In addition, effects of augmented reality on human functions and organizational factors are considered in our framework. We highlight the features provided by our framework and pre-existing related work in Fig. 18.

Conclusion and future work
In this paper, we provided a framework for assessing risk of ARequipped socio-technical systems. This framework provides the possibility to detect faults and failures leading to system risk and provides the possibility to model and analyze system behavior. In addition, we conducted a case study to illustrate how our proposed framework can be used for predicting risk caused by new AR-related dependability threats. The predicted risk can then be used as a basis for developing e.g., the safety concept in compliance with ISO 26262 and SOTIF related work products.
The framework includes extensions for modeling and analyzing AR effects on human functioning and AR effects on faults leading to human failures. We showed the analysis results by providing three scenarios. In two of the scenarios, the failure was emanated from the AR-related faults. We provided failure propagation manually and we showed that in some scenarios there would be no failure in technical entities of the system, but risk would be identified caused by non-technical AR-related faults. By implementing our proposed conceptual extensions for CHESS toolset, failure propagation calculation can be provided automatically to be used for AR-equipped socio-technical systems.
Our proposed framework supports ISO 26262 and SOTIF development process activities and can be used for providing expected work products by these safety standards. In addition, we discussed that the modeling capabilities within our proposed framework is helpful for determining ISO 26262 controllability. ISO 26262 controllability requires to be updated in order to be used for AR-equipped socio-technical systems, especially in higher levels of driving automation.
Further research is required to show the potential benefits of the proposed framework. Specifically, we intend to conduct case studies where there are scenarios with higher safety criticality. In addition, having two or more teams composed of three or four experienced analysts would help to have more advanced scenarios including more complicated propagation of failures. In future, we also plan to evaluate a safety-critical socio-technical system within the rail industry, the passing of a stop signal (signal passed at danger; SPAD) [58], to verify if the results are transferable to the rail domain.

Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.