A Review of Visualization Methods for Cyber-Physical Security: Smart Grid Case Study

Cyber-Physical Systems (CPSs) are becoming increasingly complex and interconnected as they attempt to meet the demands of evolving society. As a result, monitoring and maintaining them becomes a more complex and demanding task for control system operators and cyber defenders. While the literature on visualization techniques in the context of cybersecurity is extensive, the same cannot be said for studies on visualization for the security of cyber-physical systems. This paper aims to fill that gap by: 1) defining the main features of a visualizations workflow for security visualizations in cyber-physical systems. The workflow includes the acquisition of cyber and physical data, processing of data, selection, and configuration of both visualization tools and end-user interactions. 2) Providing an overview of cyber-physical security visualization systems, with a focus on smart grids as a case study. Finally, we use the perspectives gained from this analysis to provide insights and directions for future research and design of cyber-physical visualization techniques.

they use, which is beneficial to both the economy and the environment [3]. The smart grid is composed of two major parts: 1) power application and 2) supporting infrastructure [4]. The power application manages the main tasks of the smart grid, notably the generation, transmission, and distribution of electricity. The supporting infrastructure, on the other hand, is the intelligent part of the smart grid that is mainly concerned with managing and monitoring the key functions through the use of various pieces of hardware, computer programs, and communication networks.
Smart grids and their security and resilience have been at the core of critical infrastructures for many decades. Even though smart grid system connectivity has resulted in numerous benefits, many challenges have come up, including security, reliability, stability, maintainability, and safety [5]. Malicious attacks have disrupted system operations or resulted in the theft of sensitive data, and these attacks can start with an attack on either the cyber infrastructure or the physical components.
In 2007, the Department of Homeland Security of the USA staged a cyber-attack demonstration code-named ''Aurora'' resulting in the destruction of a diesel generator that was widely utilized across the US [6]. Recent incidents (e.g., Stuxnet [7], Ukraine power grid outage [8]) have shown that sophisticated and stealthy attacks (and faults) can result in catastrophic consequences to the economy, environment, and even human lives. Thus, it is paramount important to ensure the security of CPSs and smart grids.
A key aspect of detecting attacks is to provide enough information awareness to operators of control systems. To identify attacks and unexpected errors in smart grids and other CPSs an anomaly detection system (ADS) is required. Anomaly detection systems provide information about the detection of attacks, however, visualization of such information will facilitate more understanding of the attack to the system operator [9]. Effective visualizations would achieve a higher degree of operational assurance by providing explainable visual information on abnormal behavior and affected devices of the system. This will provide a better understanding for the system operator about what happened in the system and which actions to take. Therefore, we can infer that visualization tools play a major role in making smart grids secure and resilient. As the reported occurrences of cyber-physical attacks increase exponentially [10], it is imperative to develop and employ tool sets that enhance the user's ability to investigate, understand, and therefore mitigate the impacts of future events on smart grids and other cyber-physical systems.
Visualization tools are intended to be an effective means of communicating data to quickly spur insightful actions through visible patterns, trends, structures, and exceptions, resulting in improved system security [11]. In today's data-driven world, we can collect and use data to help decision-makers make informed and well-founded decisions. The push for renewable energy, increased demand for electric power, and upgraded intelligent power grid equipment (e.g., advanced grid meters, smart relays, etc.), have resulted in increased data streams and complexity of systems, as well as challenges related to controls of such systems [12].
The process of visualizing the state of the cyber-physical system is linked to the process of transforming raw unstructured data into information that can be represented using objects of varying shapes, colors, and dimensions. These objects should help understand the state of the system by facilitating the analysis of large amounts of data [13]. Figure 1 illustrates the various steps and actions involved in the visualization process. This diagram provides a comprehensive illustration of workflow throughout the process of transforming raw data into the final product, which is an interactive visualization designed to improve the security of the system [14].
Because of the clear lack of survey papers on the topic of visualizations for enhancing the cyber-physical security and resiliency of smart grids, we performed an analysis of related literature to provide a synthesized overview of the research VOLUME 11, 2023 59789 Authorized licensed use limited to the terms of the applicable license agreement with IEEE. Restrictions apply. on this critical topic. We hope to support both researchers and industry partners by offering an overview of the state-of-theart techniques and best practices in this field. Our aim is to facilitate a unified understanding of the use of visualizations in CPS security, ultimately promoting more effective and efficient security practices.
The main contributions of the paper are: • An analysis of the visualization workflow (desired features and characteristics).
• A review of existing work for smart grid security visualization.
• Analyze the challenges and key findings discovered during the review process regarding the visualization's design and implementation for the security of CPS.
• Identifying the potential future research directions for the successful use of the visualization tools.
The rest of the paper is organized as follows. Section II provides an overview of the related work in the field of data visualization. Section III presents in detail the main points and key features of the visualization workflow. An overview of different visualization methods for smart grid case study broken down into two classes (monitoring and planning and discovery) is covered in section Section IV. Section V discusses current challenges and key findings from the review of cyber-physical visualization research, as well as potential future research directions. The conclusions of the paper are provided in the final section.

II. RELATED WORK
In today's world, we generate and store vast amounts of unstructured data that may help us discover new knowledge and insights in cases we can aggregate into a meaningful picture [15]. The authors of [16] and [17] conducted a comprehensive survey on data visualization tools. This survey not only provided a taxonomy of data visualization papers but also highlighted the most effective techniques for optimizing the performance of data visualizations, as well as discussed the challenges and future opportunities in this field. Similarly, in [18], the authors performed a survey on techniques that can improve the efficiency of data visualization. In some reviews, such as those presented in [16] and [19], the focus is on analyzing the advantages and limitations of software tools used to generate visualizations.
The ability to process, store, and analyze vast amounts of data, commonly known as big data, is an interesting and hot topic in the research community [20]. Numerous surveys have investigated the challenges and solutions associated with visualizing big data [20], [21]. In [22], the authors address the problem of high-dimensional data and explore various solutions for visualizing it. Other papers focus on the visualization of specific types of data: ensemble data [23], event sequence [24], multivariate spatial data [25], graphs [26]. Furthermore, certain surveys concentrate on analyzing papers that focus on a specific domain or task: traffic data visualization [27], security [11], [28], [29], [30], Internet of Things [31], [32]. A promising research direction in visualization involves the integration of machine learning algorithms in the visualization process to automate the processing step of the data or recommend visualizations that are more useful for each user [33], [34], [35].
Given that cyber-physical systems involve a cyber component, an important area of interest is the representation of visualizations for cybersecurity. In [28], the authors presented a survey with a security visualization based on network logs. Similarly, the authors of [11] conducted an extensive review of the field and classified visualization tools into categories such as host/server monitoring, internal/external monitoring, port activity, attack patterns, and routing behavior. Jiang et. al. [29] performed a systematic review on cyber situational awareness visualization. In terms of physical security visualization, we can mention a review that has addressed the visualization of time-series data produced by sensors [36].
After conducting extensive research, it was identified that there is a lack of surveys on visualization methods specifically designed for cyber-physical systems security. This review will concentrate on visualization solutions that can be applied to smart grid security. While there are surveys that target the visualization and analysis of data in smart grids [12], [37], [38], they do not specifically review the security aspect of smart grids, but focus on monitoring other features such as carbon footprint. As a result, this review will rely on papers that provide visualizations for cyber-physical systems but can be applied to smart grids, as well as tools designed specifically for smart grid security. We will extract useful insights from papers that focus on cybersecurity or time-series data monitoring separately to provide a more comprehensive discussion section and to describe the visualization workflow in detail.
We conducted an extensive literature review by searching four databases: IEEE Xplore, Web of Science, and Scopus. We used the following search query ''Visualization AND (security OR cyber OR physical OR CPS)'' and ''Visualization AND (review OR survey)'' to select relevant papers on the topic. Our search yielded a total of 326 publications. From our analysis, we observed that the majority of the articles focused solely on cyber-security, and there is a limited number of papers that tackled the problem of security in cyber-physical systems. We could only identify a limited number of papers that had a finalized prototype of the visualization engine, and these papers were selected for review in Section IV.

III. VISUALIZATION WORKFLOW
The success of a security visualization system will be impacted by a multitude of factors such as the quality of available data and its attributes, the specific tasks that are associated with the visualization, the end-user preferences, habits, and expertise in the domain, the selection of visualization techniques that will be incorporated in the final visualization engine [39]. These steps are illustrated in Fig.1 and are part of a visualization workflow. In this section, we will discuss each of these factors in detail and provide an in-depth analysis of their main characteristics. Firstly, we will describe the different types of data and the techniques used for processing raw data, which are typically used in visualizing smart grid security. Then, we will present the most common visualization methods and tools, along with their typical use cases and the associated data. Furthermore, we will outline specific tasks that can be involved in enhancing the security of cyber-physical systems. Finally, we will discuss the adaptability of visualization to the end user, and the expected interactions from the end user.

A. DATA AND PROCESSING TECHNIQUES
Cyber-physical systems generate increasing cyber and physical data. The following paragraphs will define cyber and physical data and how the raw data is processed to visualize such systems.
Cyber data includes items, such as network packets, port numbers, system logs, network flow events, proxy logs, firewall logs, DNS data, DHCP logs, I/O data, user authentication events, CPU, RAM, and other information. Intrusion detection systems (IDS) and intrusion prevention systems (IPS) may generate additional cyber data, such as raw-log insights or specific pattern detection [40]. Moreover, supervisory control and data acquisition (SCADA) systems, Advanced Metering Infrastructure (AMI), Distribution Management Systems (DMS), and Cybersecurity tools such as firewalls are also sources of cyber data [41]. While this data is often monitored and reviewed by experienced analysts, cyber data is one of the most difficult types of data to visualize in real-time. In [42], authors remark that part of this difficulty stems from the technical factors (what mapping strategies are possible for the given data) and the human factors (what type of visualization makes sense to the user). The sheer amount of cyber data that is generated by network and system monitors (scalability) can also make visualization difficult [11]. Despite these challenges, there are a wide variety of tools available for cyber visualization including FlowTag, NVisionIP, security information and event management (SIEM) [43], IDS RainStorm, and Netvis [42]. The most common data types for this type of data are numerical (temporal and multidimensional) and categorical. The logs and reports will frequently imply analyzing text-based information.
Physical data can be defined as data that relates directly to the physical world, such as measurements on ambient temperature, humidity, wind speed, and direction, and geographical area, voltage, irradiance, and cloud cover [44]. Physical data also refers to the current state of a physical system based on the available internal sensor measurements, upon which people, like grid operators, are dependent to make judgment calls about their systems. For physical data measurements, they are often visualized in the system software (i.e. RTAC management software, MATLAB, etc.). This type of data will be described using numerical features in the case of sensor data and categorical features in cases where it is needed to identify the state of a component, e.g., if the switch is on or off. The main data sources are Smart meters, PMUs, Micro PMUs, Field measurement devices, Programmable thermostats, Sensors installed on grid-level equipment, (e.g. transformers, network switches), Programmable Logic Controllers (PLCs) [45]. Table 1 provides a summary of the data types produced by cyber-physical systems which can be included in the security visualization.
Data processing techniques: In this step, we are dealing with the problems related to big data [46]. And the main problem occurs in the cases of real-time security visualization (monitoring) or when the amount of data is so massive that it can overload the buffer. In this case, it is required to compress the size of the data or select the data that is most suggestive and remove the data that is redundant. Data normalization techniques are used to bring the data within a common scale to avoid any bias towards a particular feature. The most used techniques for data processing are feature selection and extraction, clustering, and dimensionality reduction [36], [47]. Moreover, the sliding window technique is an essential method for analyzing time series data [48]. It involves dividing a long time series into shorter, overlapping segments or windows, allowing for a more precise analysis of the data. One of the main advantages of using a sliding window is that it can capture local patterns or trends in the data that may VOLUME 11, 2023 59791 Authorized licensed use limited to the terms of the applicable license agreement with IEEE. Restrictions apply.
not be apparent when considering the entire time series as a whole.

B. VISUALIZATION METHODS
There are many types of visualization methods that can be used for displaying security-related data and the choice of these types of visualization depends usually on the message that needs to be communicated and the time constraints for the visualization. In this section, we will discuss the most common techniques, the strengths of each display, use cases, and how they are adapted to specific security needs [49].
The most common use cases and types of visualization tools are: Visualization of data trends: Most of the stored data from CPSs is continuous time-series data, so the most intuitive study that can be performed is noticing changes and changes over time. The most popular techniques are line charts and area charts. As well it is possible to visualize trends in multidimensional data using parallel coordinates [50] or radial displays [51].
Visualization of data correlations: These types of visualization are designed to emphasize the strength of the relationships between displayed variables. The most used visualization are scatter plots [52], bubble plots which represent the variation of scatter plots in three dimensions where the third dimension is the diameter of the bubble, and heatmaps, where the color intensity suggests the strength of the relationship.
Visualization of data distributions: The visualization of distributions is useful for better understanding the data and its characteristics, which can help identify outliers. The most common types of distribution visualizations are histograms for highlighting the frequency of an event [53], box plots for displaying the range of the data, and violin plots for visualizing the shape of the distribution.
Network visualization: This type of visualization is suitable for representing the complex structure of cyber-physical systems and how elements are connected. It is a very useful tool for tracking the overlapping physical and cyber layers of the system. The main idea behind these visualizations is to represent system components as nodes and relationships between components as edges [54].
Hierarchical visualizations: The visualization of this category display the data in a hierarchical structure, where each level represents a different level of abstraction. It can be beneficial for discovering interdependencies between components. Typical choices in this category are treemaps, which consist of nested rectangles of different colors and sizes, and each rectangle represents a node, and the color and size may suggest different properties of the node [55]. For highlighting dependencies between different levels tree diagrams and dendrograms can be used.
Iconic visualizations: An abstraction that aids in data analysis and comprehension is an icon-based representation of high-dimensional data. The attributes of the icons, such as color, shape, and size, are linked to the features of the high dimensional data [56].
Fusion of the methods: In this case multiple cases that were presented before are used simultaneously for providing multiple angles of the available data to transmit more useful information at the same time [57]. For example, a map that shows the status of various components of the system using icons [58]. Additionally, treemaps and icons can be used in combination to display additional information about a component rectangle. Using multiple types of data or visualizations in this manner allows for a more comprehensive understanding of the data.

C. SPECIFIC TASKS RELATED TO VISUALIZATION
It is clear that the main goal of the visualization mentioned in this paper is to enhance the security of the cyber-physical system, but at the same time, we can identify two main tasks that are directly related to this goal: monitoring of the system and situational-aware discovery and planning.
Monitoring: These visualization systems are designed to support the user's decision-making process in real time during the monitoring process. It is important to design the visual aspects of these tools with user efficiency in mind.
The data and visual tools should be selected in a way as the decision-maker may identify the affected portion of the system and the possible propagation of the attack through the system.
Discovery and planning: Based on the literature review we can define this task for two subcategories based on the temporal use-case: 1. preemptive use and 2. root-cause analysis. These tools are not subject to the same time constraints as real-time monitoring, data processing can be done offline, and more time is available for rendering complex visualizations. These tools heavily rely on the quality of data and algorithms used for extracting features that will facilitate developing hypotheses to identify vulnerabilities before they are exploited or to identify the root cause and develop a recovery plan after an accident happens.
1. Preemptive use: These types of visualizations are designed to assist in the prevention of attacks/events. They are extremely useful during the design process of cyber-physical systems, or vulnerability and/or resilient assessments. This type of visualization will commonly compute a lot of if/then scenarios, to identify vulnerabilities. By providing multiple perspectives, the decision-makers can better identify areas of improvement or weakness in the resiliency and security of the system.

Root-cause analysis:
This use case intends to aid the user in the investigation of cyber-physical events after they have occurred. The application of the visualization tools aims to provide insights and perspective through visual techniques displaying logged attack data. These insights and perspectives are used to assist the user with the understanding of the cyber or physical event and inform the decision-making process in taking steps to mitigate such events in the future.

D. END-USER EXPERIENCE
In this section, we emphasize the factors that can enhance the overall end-user experience when it comes to visualizations: interactions and adaptability. To ensure the success of a task based on a security visualization, it is important that the end-user can easily understand the information presented through the visualization. However, the time needed to comprehend the presented data can depend on their prior knowledge related to the domain and preferences. This poses a significant challenge, as there can be significant differences between the needs and preferences of cybersecurity specialists, engineers, or higher management who are responsible for the security of a cyber-physical system. To address this issue, end-users should be able to interact with the visualization and the most common interactions described in the literature are: zooming, filtering, details-on-demand, relate, history, extract [59].
Furthermore, the visualization can include the possibility to dynamically adapt to the user based on data gathered from previous interactions. For example, a machine learning recommendation system that uses data from users from the same categories as previous interactions could be implemented to provide a more personalized and efficient experience. By implementing these strategies, end-users can better perceive and understand the information displayed by the visualization.

IV. USE CASE: SMART GRID SECURITY VISUALIZATIONS
In this section, we will discuss the strengths and weaknesses of each type of visualization solution, as well as the important attributes of each analyzed solution. The reason for categorizing the solutions as real-time monitoring and discovery and planning is that the primary task for which the solution was designed, as well as time constraints, will have the greatest influence on the choice of processing techniques, visualization tools, and the amount of data that can be effectively visualized.
Subsection A describes the visualizations for monitoring tailored towards a short-term, immediate decision-making targeted approach (real-time). Subsection B presents the planning and discovery visualization may refer to the two-time categories related to anomalies: 1. visualizations that help to identify vulnerabilities during the design phase of the system and 2. visualization tailored towards forensics, i.e. understanding why certain events have occurred, and finding the reason that led to those events. Table 2 illustrates visualization systems reviewed in this paper.

A. MONITORING TASK
Monitoring is commonly associated with making short-term, real-time decisions. Monitoring visualizations should provide two main tools for the operators: 1) real-time visualizations for identifying anomalous behavior, and 2) meaningful information for fast, short-term decision-making for securing unaffected components of the system and rapidly solving the issue in the system. For example, when a grid is attacked by sensor spoofing, the operator should be able to quickly identify which part of the system is under attack and make decisions to prevent the spreading of the attack to the other parts of the system. Monitoring visualizations should be simple and interactive enough that system operators understand the system state to take rapid operational decisions. The main challenges for the tactical visualizations are 1) high volume and velocity data processing, 2) suggestive representation for taking rapid actions, 3) interdisciplinary representations, and 4) situational awareness.
By leveraging specific data characteristics in smart grid applications, we can create dynamic visualizations that adapt to real-time changes. For example, we can exploit the periodicity of sensor data, including amplitude or frequency, to develop a tool that can detect and highlight any deviations in these measurements. This approach was presented in Data Fusion Tool [60]. The visualization engine highlights the regions affected by cyber-physical events, as presented in figure 2, and assists in the discovery of meaningful correlations that provide operators with useful context.
Node-link diagrams are a powerful tool for visually representing relationships. The authors of [61] used this type of visualization tool to represent the complex interconnections between loads and generators. The main goal of the visualization engine is to draw the attention of the operator to the anomalous subsystems by highlighting the affected nodes (load or generator) in red. This tool facilitates the identification of alternative generators capable of supplying energy to the load.
The work of [63] is focused on using the PMU data for determining the stability boundary of the system and monitoring if the system is in safe working conditions. The visualization [62] determines the safe conditions using PMU data and random bits forest algorithm. The most relevant features VOLUME 11, 2023  are selected using bagging nearest-neighbor and Pearson correlation.
The Cyber Situational Awareness Visualization (Cyber-SAVe) [64] utilizes several algorithms and visualization techniques to detect various types of attacks on the smart grid.
This approach views the complex system as being composed of nodes. This visualization tool solves the problem of the high volume of data by fusing all available data and measuring against the single ''cyber trust'' metric in order to enable users to make critical decisions. Cyber trust at the level 59794 VOLUME 11, 2023 Authorized licensed use limited to the terms of the applicable license agreement with IEEE. Restrictions apply.  of nodes (sensors) refers to sensors' availability, ability to accurately detect measured values, produce few false alarms, and how trustworthy future measurements are predicted to be. The calculated level of trust for a particular sensor can be used to isolate a sensor targeted by an attack. Figure 3 demonstrates a time history plot of a single node (sensor) by displaying the three components of the cyber trust metrics (alarm, detection, and availability) and thus the overall trust of the node.
Situational awareness is suggested by the fusion of a geographical map and the nodes in the smart grid that provides information to the system operator about the impacted nodes and their interconnection to the other parts of the system. Individual nodes in the geographical system are represented using a color-coded format. This differentiation by color allows the operator to gain a quick insight into the trust of each individual node while also giving insight into the overall system state. Figure 4 illustrates the holistic view of the visualization tool used for smart grid security from a geographical perspective in the CyberSAVe system.
The Modular Analysis Hierarchical Intrusion Detection System Visualization Event (MAHIVE) [65] displays various data related to intrusion detection such as alerts, logs, and categories of events such as SSL, DNP3, or file-related alerts. The visualization dashboard for the MAHIVE Alert system uses scatter plots, bar charts, and numerical displays to present anomaly detection system-related data. While the presentation of this system is limited in terms of the current state of its visualization capabilities, the authors have identified various topics of future work for improving the system.  The significance of contextualized data in a humanreadable format was emphasized by [66], a human-on-theloop framework that allows the operator to customize the visualization. The presented framework has the main goal to contextualize data from automated detection, prevention, and mitigation tools. The framework consists of three modules: 1) data analyzer comprising Kafka, Apache Spark, and R, 2) classifier comprising a deep neural network, and 3) decision-maker with situational awareness and cognitive learning model. These modules are used to conceptualize data and then present it via the dashboard interface. The front end of the dashboard interface is segmented into four divisions that the operator can dynamically customize to show data based on system priorities and availability. Such dynamic customization facilitates adjusting to the point of view of a particular operator's background (discipline). The dashboard in Figure 6 presents a theoretical scenario where an operator is monitoring Grid-Tied 3 PV systems.  The Cyber-Energy Emulation (CEE) Platform [67] is used for real-time visualization of large-scale environments involving cyber-physical devices. This approach is quite unique to the field of cyber-physical visualization tools as this platform has the capability to include real and emulated physical devices. The emulated environment allows the user to test parameter changes without the risk of damaging physical equipment. This emulation feature combined with real-time visualization of network traffic, security alerts, and power system states allows the user to improve their overall system state awareness during system operation and parameter experimentation. The CEE real-time environment has the ability to display data from cyber and physical events in a 3D platform while providing the user with a beneficial perspective and utilities for in-depth analysis. An example of a 3D-rendered experimental environment with an ongoing intrusion detection alert can be seen in figure 6 with detailed logs and an interactive user interface.
Cyber-Physical Security Assessment (CPSA) visualization tool was proposed by Saxena et. al. [68] to promote situational awareness in the smart grids. This tool utilizes a Graphical User Interface (GUI) to assist operators in their decision-making process by identifying vulnerable states and measurement errors. The proposed visualization scheme for this tool uses a combination of line graphs, alphanumeric data fields, and the relative geographical location of the infrastructure. Moreover, CPSA can be used to improve understanding of power system monitoring, analyzing the nature of cyberattacks, malicious command insertion,affected devices, and understanding the impact of attacks on the operation of the power system. Thus, the operator is informed of potential and active threats, as well as the possible fallout from attacks on the power system.
The approach of Le Blanc et. al. [69] is focused on interviewing, observing, and learning about human experiences and interactions with control systems. By developing an understanding focused on human factors, the authors focus on improving user productivity and situation awareness. This paper presents an approach to creating suggestive and interdisciplinary representations by interviewing and including previous observations and experiences of the system operators to determine what information is important to the user.  Through observation of team-member interactions and data sharing between cyber and engineering teams, the authors identified informational context and other key features for informed system visualization. The preliminary design concept as seen in Figure 7 was created by observing the reactions of human subjects during the simulated cyber and physical attacks on the power grid.
The work of Scholtz et al. [70] focuses on enhancing situation awareness of individuals from non-overlapping roles to improve the coordinated response of control system engineers. An example of an alert on a remote terminal unit that has rebooted unexpectedly is illustrated by Figure 8. This proposed display allows multiple non-overlapping professionals to be able to provide insights during the monitoring of the system.
One of the inherent characteristic features of communication within operational technologies (OT) is the frequency with which communication occurs. Sensors within operational technology applications regularly send and receive sensor data at frequent intervals. The goal of the visualization system presented by Lohfink et al. [71] is to utilize these inherent features from sensors to provide insights into the available data using spiral plots. These insights aim to support and inform control operators and cyber defenders of the response strategy, regardless of their experience in the field. The main goal of spiral plots is to map the data dimensions 59796 VOLUME 11, 2023 Authorized licensed use limited to the terms of the applicable license agreement with IEEE. Restrictions apply.   to the width and color of the plot, making it more intuitive. The proposed system overview can be seen in figure 9. The final visualization is composed of the spiral plots and icons displaying alerts from the anomaly detection system.
The goal of the display presented in [72] is to aggregate meaningful information together, facilitating rapid operational decisions and complementary context for the roles, as the root cause of events can include both cyber and physical elements. Figure 10 shows an example of the visualization for the IEEE 33 bus system with several of the busses grouped into Aggregated System Resources (ASRs). Each of the different ASR units can be selected to drill down into lower levels that display the ASR's internal components, as shown in Figure 11, where each bus now possesses its own resilience icon. The user can easily locate relevant information by displaying information at a desired resolution. The interconnections between different elements are representing different states, such as normally closed, closed, normally open, or opened ( Figure 10).

B. PLANNING AND DISCOVERY TASK
Planning and discovery facilitate the adoption of a long-term plan for securing the cyber-physical system. In the case of these visualizations, the time scale is much more different than in the case of real-time visualizations. In this case, the efficiency of visualization is affected by the quality of the algorithms for data mining and integration of prior   knowledge [82]. Based on the time scale of the two types of visualizations, we can consider the following two scenarios: 1. Preemptive visualization for secure design of the system before any exploit occurred, 2. Root-cause visualization for securing the weak points of the system after an attack happened.

1) PREEMPTIVE VISUALIZATION
The secure design visualization can facilitate the secureby-design process and apply the zero-trust framework as well as identify risks associated with possible incoming data sources/events [83].
One of the most popular approaches for preemptive visualization is to compute attack graphs [76], [77], [78] in order to better understand the system's vulnerabilities before they are exploited by attackers. In [76], the authors proposed a visualization tool that uses Architecture Analysis and Design Language to model attacks, identify attack vectors, and visualize potential damage. As a result, system operators can use this visualization tool to identify weak points and develop effective solutions for their remediation. The visualization engine presented in [78] evaluates the effectiveness of potential mitigation plans and provides an overview of possible attack evolution by generating attack graphs for specific scenarios that can be simulated directly in the visualization tool.
The work presented by Wu et al. [74] is an example of a visualization system that provides applications for both the attacker's and defender's perspectives. When considering the attacker's perspective, there are multiple types of visualization techniques used for displaying pertinent data. Statistical analysis is performed to identify potential vulnerabilities and display their cause within a nightingale rose diagram as seen in Figure 13. The diagram is used to demonstrate the distribution of threats caused by a corresponding vulnerability. This threat and vulnerability data is used to link corresponding threats to commonly correlated keywords such as ''buffer overflow'' and ''memory''. These keywords are then displayed in a word cloud graph, giving the attacker a sense of potential attack vectors.
An example of a node-link diagram was given by the Bakirtzis et al. [73]. Here, the authors present an interactive security analysis dashboard that displays system requirements and associated attack vectors. As seen in the figure 12, this tool makes it possible to start analysis earlier in the system life-cycle. This system uses node-link diagrams to help inform system designers and security analysts where to improve collaboration during the system design process. The node-link diagrams display system specifications, potential attack vectors, and the resulting attack surface within the system's topology.
Another interesting example of visualization was the ''resilienceCube'' by Jovanovic et al. [75]. This paper presents data via line graphs, tabular data, 3-dimensional matrix cubes, node-link diagrams and maps. The framework was designed to accommodate a wide range of custom tools that monitor three aspects of infrastructure resiliency: Recovery/Adaptation, Robustness, and Preparedness. The ''cube'' plots each of these three values in a 3D space to provide a sense of how well or poor the system is in each area. This assortment of display tools is intended to aid in resilient management strategies by allowing the user to assess, monitor, and benchmark the resilience of a cyberphysical system. The proposed framework set allows for the modeling and analyzing of scenario-based threats, as well as stress-testing the intended system design. This method of identifying potential weaknesses in critical infrastructure allows for the optimization of the initial investment as the resilience of the system can be improved upon.

2) THE ROOT-CAUSE VISUALIZATION
This class of visualization methods addresses the need for forensics investigation and analysis. Such visualizations are intended to provide insight into the factors that led to a cyber-physical event. These visualizations enable users to learn from past events and come up with a plan to eliminate vulnerabilities that were exploited by the attackers.
Macedo et al. [80] presented a tool that aims to combine an intuitive user interface with machine learning forecasts that display the available data through time-series visualizations. The dynamic user interface of this tool is intended to display pertinent information in supporting the investigation of cyber and physical attacks that are presented in a user-friendly manner. This visualization tool uses line graphs, pie charts, bar charts, and numerical displays, as seen in Figure 14, to display the available data that is generated by threat detection tools.
The work of Yan et al. [79] aims to improve the understanding of how the power grid behaves under complex attacks by combining geographical details of substations with attack and defense data in a simulated environment. This creates a geographically accurate power grid system testbed for which the user can simulate several single-node (substation) attacks. The intent of this visualization system is to provide effective visualization capability of these cyber-physical attacks which will help inform the decision-making process of power system operators regarding their response strategies.
The geographical representations of the power grid during the simulation of an attack are able to display the cascading impact of a cyber-physical attack on the power grid, as seen in Figure 15. The ability to visualize the impact of a cascading attack and how it propagates through the power grid is intended to assist power engineers and operators in the efficient modeling and simulation of these attack scenarios. These scenarios utilize real power grid system data that can potentially help operators to understand grid behavior under certain types of complex attacks with the goal of improving defense strategies through a more informed decision-making process. Another visualization tool for visualizing the cascading failures is presented in [81]. The primary visualization method used is the node-link diagrams with specific node icons. The proposed solution includes separate displays for the cyber layer and physical hardware.

V. DISCUSSION
In this section, we will present the main challenges, findings, and future research directions for improving smart grid security visualizations.
The visualization system must be designed to aid in the solving of specific problems and tasks while remaining simple and intuitive to use [70]. By providing significant context to the mined data, it should improve situational awareness and facilitate faster decision-making.

1) CHALLENGES IDENTIFIED IN VISUALIZATIONS OF POWER GRID SECURITY a: COMBINING CYBER AND PHYSICAL
Major difficulties stem from the differences between cyber and physical data, acquisition process, and difficulties of synchronizing these types of data. Many distributed sensors are used in today's power grid, each using a different communication protocols, hardware, and software resources to generate multidimensional datasets with varying sampling rates, which are then transmitted to the central control center.

b: MISSING HOLISTIC VIEW
Currently, most research is focused on cyber security visualizations, with few proposed solutions for providing a holistic (cyber and physical data combined) view of the security of cyber-physical systems. The visualizations do not have any built-in tools for determining whether a cyber attack caused physical anomalies or whether physical anomalies disrupted communications between system components. As a result, it is entirely up to the end-user to identify cyber-physical causality by correlating the visualizations.

2) BEST PRACTICES
After performing the review, it has been observed that the majority of researchers are focusing their efforts on the development of real-time visualization monitoring tools for security, with relatively fewer efforts dedicated to visualizations designed specifically for tasks such as root cause analysis or system design.
Monitoring -This category of visualization tools should include features in their presentation format that allow the user to interpret them quickly. An effective example would be the ability to highlight important or high-priority alerts in the foreground of the display, while also taking into account the relative size, shapes, and colors in the context of the overall design, in order to quickly capture the attention of the user and provide a clear understanding of the alert's details, achieving a balance between focusing on critical components and comprehending the system as a whole.
Planning and discovery -This class of visualizations relies on prior knowledge and external sources to identify potential attack vectors in order to prevent future attacks. It is critical in this case to stay up to date on newly discovered vulnerabilities. The primary goal of these visualizations is to identify system weaknesses and areas for improvement in terms of resilience and security. The most effective approach is to look at the problem from both the red and blue teams' perspectives and to present information to the end-user via separate visualizations. This type of visualization necessitates analyzing ''if-then'' scenarios to identify vulnerable system components and predict potential attack outcomes, while also ensuring that a worst-case scenario attack does not cause cascading failures that bring the entire grid down.

3) TOP CHOICES FOR VISUALIZATIONS
The solutions examined in the paper are mostly dashboards that use multiple visualization techniques to provide users with valuable insight into available data from various perspectives. The most common techniques are traditional 2D representations such as line and bar plots. However, icon displays are also popular because they can condense a significant amount of information into a single attribute of the icon, thereby saving display space. Node-link diagrams are used to illustrate the interconnections of a system, where the color or shape of nodes and links may indicate different properties of the system components. Furthermore, most dashboards include a map to help with spatial understanding of the data.

4) INTERDISCIPLINARY ADAPTATION
An additional consideration in the design of visualization for power grids is interdisciplinary visualization. In the modern world of inter-connected and smart systems, users from many different roles as cyber experts, power engineers, etc. are collaborating to secure the grid. Supporting interdisciplinary adaptation and information sharing is imperative to help the end-users communicate and inform the decision-making process for their response strategies. These considerations also help find common ground among data and system feedback, as the available data may be used for many different considerations from role to role.
Allowing end-users to choose relevant information and how it is presented is an effective strategy for optimizing the adaptation process. Another option is to incorporate feedback mechanisms into the system design. The feedback gathered can be analyzed to determine the effectiveness of the visualization system and areas for improvement in future versions.

A. FUTURE RESEARCH DIRECTIONS 1) HOLISTIC CYBER-PHYSICAL HEALTH METRICS
By combining different streams of information into single metrics, the time required to achieve situational awareness can be significantly reduced by presenting single-number displays that holistically describe the system state. These metrics are presented at a level of abstraction that requires no prior knowledge or experience, making them easily adoptable by end-user with diverse backgrounds. Examples of such approaches are presented in [64], where the authors introduce the trust metric, which computes the metric for a single sensor, and [72], where the authors provide a score for the cyber/physical health and resilience of the system.

2) EMPHASIZING INTERDISCIPLINARY ADAPTABILITY
It is imperative to design a tool that does not add additional workload to the user and is capable of providing complete cyber-physical visualization that encompasses the entirety of the monitored system. This design consideration will allow the user to fully utilize the features of the additional data visualization tool while avoiding the idea of adding, ''yet another screen to monitor.'' While the goal of these visualization methods is frequently to provide the user with a better perspective, literature shows that these tools frequently encourage the integration of multi-disciplinary efforts with the common overarching goal of maintaining and protecting the power grid. The versatility of visualization tools makes it an appealing technology for researchers and industry professionals to pursue in order to further develop this standard.

3) EVALUATION IN REAL-WORLD SCENARIOS AND BENCHMARKING
Some of the solutions studied in this field are still in the early stages of development, and their effectiveness has been tested on experimental setups. As a result, additional testing and evaluation of these proposed solutions in real-world industrial scenarios are required to truly assess their effectiveness and highlight the benefits that they can provide to operators in achieving situational awareness. This testing will allow researchers and industry professionals to better understand the challenges of putting these solutions into practice, as well as identify areas for further improvements. Moreover, to evaluate these solutions effectively, there is a need to develop a benchmark that can help to identify unique criteria for evaluation [84], [85].

4) INTEGRATION OF AI/ML
Overall, the development of visualization tools based on machine learning has the potential to significantly improve the security of smart grids. Recent advancements in machine learning have opened up new horizons in the field of cyber-physical security visualization engines. Some of the benefits are a dynamic adaptable system based on an intelligent recommendation system, universal dimensionality reduction, feature extractors, and an effective intrusion and anomaly detection system for identifying and categorization of anomalies. Machine learning can be seen as a complementary tool for extracting insights from existing data streams.

5) CAUSALITY BETWEEN CYBER AND PHYSICAL
It may be extremely beneficial to shift away from the end-user effort in determining cyber and physical causality and correlation towards the use of algorithms to infer the causality of certain events detected in the system based on available data. This approach can significantly reduce the time required to comprehend the data because the visualization will clearly display whether changes in the system's subsystems or layers are correlated or causal.

VI. CONCLUSION
Visualization methods provide the ability to effectively organize and display complex data, allowing for effective and timely operator decisions, as well as running of ''if-then'' scenarios.
It is critical to detect anomalous behavior in the power grids as soon as possible, as any delay can have serious consequences to the security and safety of human life and physical assets. The visualization of security aspects of cyber-physical systems has a tremendous effect on the life cycle of complex systems. We looked at preemptive use visualizations that guide and assist users during the design process, as well as vulnerability and resilience assessments. Other solutions are designed to assist users in making real-time decisions during monitoring tasks by providing situational awareness and resolving the issue. Furthermore, some dashboards have been designed to assist users in investigating cyber-physical events after they have occurred and determine the root-cause analysis.
Over the recent years, one can recognize several growing interests when it comes to the visualization of cyber-physical systems with security in mind. The first one is incorporating machine learning algorithms to identify patterns, discover previously unknown relationships, automatically categorize and label incoming data, and reduce data dimensionality. The next one is a shift towards a user-centric focus (adapt to operator need in a given moment). Yet another direction is benchmarking of visualization mechanisms (efficiency evaluation in real-world industrial scenarios). BJORN VAAGENSMITH received the B.S. degree in engineering with an emphasis in electrical engineering from Dordt University, Iowa, and the M.S. and Ph.D. degrees in electrical engineering with an emphasis on electronic materials and devices from South Dakota State University. Currently, he is an Electrical Engineering Researcher with Idaho National Laboratory (INL), where he has continued electronic materials projects. He has over five years of experience in building and working with custom power systems data acquisition solutions and performing various power studies. While at South Dakota State University, he was awarded the Integrative Graduate Education and Research Traineeship Scholarship, in 2012, to conduct research on transparent photovoltaics and develop solution processed transparent electrodes. His current research interests include photovoltaics devices, new electronic materials, electrospinning, machine learning, electric grid hardening, transformers, cyber-physical systems, and power grid resilience.
VIVEK KUMAR SINGH (Member, IEEE) is a Senior Researcher with the Cybersecurity Evaluation and Application Group (CEAG), NREL's Energy Security and Resilience Center. He leads a commercialization project with an industry vendor, works on 5G-Securely Energized and Resilient (SER) and NREL's Clean Energy Cybersecurity Accelerator (CECA) Projects. He was with the Power and Energy Systems Group, Idaho National Laboratory (INL), from 2020 to 2021. He has published research papers, patents, software disclosure records (SDRs), and featured in several media articles. His research interests include cyber-physical security for smart grids, wide-area monitoring, protection, and control (WAMPAC), hybrid energy storage systems, and cyber-physical federation testbeds. He has received awards from the U.S. Army Research Laboratory, NASPI, and ISU. He also received awards at Resilience Week 2018-2020 and the 2020 Texas Power and Energy Conference (TPEC).
RUIXUAN LI received the bachelor's degree in safety engineering from the Civil Aviation University of China, the master's degree in industrial engineering from University at Buffalo, and the Ph.D. degree in human factors and ergonomics from the University of Minnesota. She is a Human Factors Research Engineer with the Human Factors and Reliability Department, Idaho National Laboratory (INL). She was a Postdoctoral Research Associate with INL, in August 2019. Prior to that, she was a Graduate Student Researcher with the University of Minnesota, where she helped to develop user-studies for evaluating human factors on motion sickness in virtual reality. Her areas of expertise include research and development, human factors engineering, simulation, motion capture, and ergonomics. Her research interests include affordance perception, system design, visualization, human-machine interaction, humanautomation interaction, and risk and resilience in cybersecurity.