Keywords

1 Introduction – Lab Paradigms to the Real World

The U.S. nuclear power plant (NPP) industry is interested in extending their current 40-year licensing periods by 20 or even 40 years to safely, reliably and securely meet U.S. energy demands. The Department of Energy (DOE) Light Water Reactor Sustainability (LWRS) program provides pathways for national labs to collaborate with academic and industry partners on R&D efforts to accomplish this objective [1]. Idaho National Laboratory (INL) and major US utility have arranged Cooperative Research and Development Agreement (CRADA) that is affiliated with the LWRS Program to support Main Control Room (MCR) modernization activities [2]. Fully modernizing a MCR in a single outage cycle is unfeasible. Utilities are therefore upgrading systems in a piecemeal fashion. INL’s role is to help formulate an end-state vision for the MCR, as well as conduct a series of operator studies to validate design concepts and operator performance using new systems. The utilities benefit from this arrangement by obtaining plant and system specific knowledge and guidance. DOE and other utilities benefit because DOE shares generalizable findings. Furthermore, conducting control room studies with licensed operators is an expensive and time-consuming endeavor. With the CRADA arrangement, the utilities contribute operator time and support to carry out running full-scale, full-scope scenarios.

DOE’s Office of Nuclear Energy (DOE-NE) has a Nuclear Energy Enabling Technologies (NEET) Program that aims to develop “crosscutting technologies that directly support and complement the Department of Energy, Office of Nuclear Energy’s (DOE-NE) advanced reactor and fuel cycle concepts, focusing on innovative research that offers the promise of dramatically improved performance [3].” LWRS focuses efforts on near-future challenges and objectives for existing NPPs. In contrast, NEET focuses more on future technologies that could have long-lasting benefits beyond our current fleet of light water reactors. The complementary nature of a LWRS-CRADA and an ongoing NEET funded project provided a unique opportunity to conduct an evaluation of a Computerized Operator Support System (COSS) with licensed operators in the summer of 2016. As part of a CRADA workshop to evaluate an end-state control room concept, we could conduct scenarios with operators using COSS prototype with advanced control schemes and fault diagnosis system. This iteration of COSS was implemented for the Chemical Volume Control System (CVCS) of a three-loop Combustion Engineering pressurized water reactor.

Our COSS prototype has gone through several design-evaluation iterations [4]. Previous iterations have examined how to integrate a fault diagnosis system (PRO-AID) and utilize the fault diagnosis information to benefit the operators by incorporating Computer Based Procedures with soft controls. Our previous concepts have functioned as digital only interfaces. Next-generation plants will feature mostly digital controls, but modernized control rooms are unlikely to have the same level of control automation or digital control.

U.S. utilities are modernizing their control systems to increase reliability and reduce operating costs, but are proceeding by keeping existing analog safety systems in place [5]. Re-working these systems is time-consuming and expensive. Modification of safety critical systems would also require extensive review from the Nuclear Regulatory Commission. With proper maintenance, these systems can remain in place and function for the extended lifetime of the plants. As a result, modernized control rooms will likely contain a mix of digital and analog controls (hybrid control boards). In our COSS evaluation, the Human Machine Interface (HMI) simulated hybrid control boards with both analog and digital instrumentation and control. The digital portions consisted of large digital overview and smaller touch displays, while the analog portion consisted primarily of the remaining safety indicators and controls.

To understand the context of COSS in the future of nuclear power we must understand that utilities are actively developing modernization roadmaps 20+ years into the future. Specifications that are being written today might take seven to ten years to come to fruition. The U.S. nuclear industry has a commendable safety track-record partially because the nuclear industry as a whole is conservative at adopting control technology. COSS concepts and their underlying engineering technologies are plausible but not yet commercially viable. Our goal with formative evaluations is to ensure COSS evolves into a product that would be of real-world value to operators and nuclear control operations. COSS demonstrations have a secondary effect in shaping how operations culture evolves to incorporate advanced control schemes and fault diagnosis systems.

2 Computerized Operator Support System Prototype

A COSS or Computerized Operator Support System is defined here as a conglomeration of traditional and advanced control system technologies and human factors interaction concepts that are designed to function as a whole to assist operators in monitoring, controlling, and managing control processes in normal and abnormal operating conditions.

The term designed is of critical importance to understanding the definition. A control system and human machine interface could incorporate the technologies in a haphazard fashion. The resulting product could, on paper, have the same functionality, but be suboptimal to plant operations and operator interactions. The COSS concept is philosophically distinguished by incorporating design thinking into the creation of the product. Design thinking is a synthetic inductive process (solution-focused) in contrast to traditional scientifically rooted human factors that tends to be analytic and deductive (problem-focused) [6]. The COSS concepts were conceived by thinking about what would be most ideal to the operators should a problem arise, and then fitting technology to the solution. In this manner it is a user-centered design process rather than an engineering driven design process.

This iteration of COSS was implemented for the Chemical Volume Control System (CVCS) of a three-loop Combustion Engineering (CE) pressurized water reactor (PWR). The CVCS is housed within containment and is part of the primary reactor coolant system. It serves a number of important functions necessary for running the plant for long-periods of time. It is responsible for maintaining the chemistry of the primary coolant by filtering out contaminants as well as controlling born concentration through addition and dilution. The CVCS also provides a high-pressure water supply for the reactor coolant pump seals, and is used to manage the inventory of primary coolant. The Human Machine Interface (HMI) simulated a hybrid control board with both analog and digital instrumentation and control such as those anticipated to be found in modernized Generation II NPPs. The digital portion of the hybrid CVCS consisted of two large digital overviews with two smaller touch displays, while the analog portion consisted primarily of safety indicators and controls. The prototype was deployed in the INL Human System Simulation Laboratory (HSSL) full-scale, full-scope, reconfigurable glass top nuclear control room simulator [7]. The HSSL simulator allowed the hybrid COSS control board configuration to be compared to a more traditional digital HMI as well as the existing analog configuration (see Fig. 1).

Fig. 1.
figure 1

The human systems simulation laboratory nuclear control room simulator. The hybrid COSS-CVCS is represented on the group of 3 bays depicted on the far left.

Full size image

The COSS prototype emulates several advanced technologies to help operators monitor and control the CVCS while also enhancing their ability to detect and mitigate faults. A control room can have over 2,000 analog indicators and controls in addition to indications from the plant computer and other sources. Operators must constantly monitor and integrate information across sources to assess the current state of the plant. As plants are modernized, digital infrastructure supplants existing analog systems. Digital infrastructure can be advantageous because it allows additional information to be provided to operators, but this extra information may also compete for the operator’s attention. One approach to organizing and prioritizing the available information is to use large overview displays to provide operators “at a glance” system status information. More detailed system and component level information is available by “digging” down through hierarchically organized displays.

Most NPP control rooms predate the existence of modern digital alarm list displays. With existing control rooms the alarms are grouped into windows at the top of each control board. This arrangement can be beneficial to operators because they can quickly assess the state of the plant on scanning the alarm tiles. The arrangement of the alarms is static and the operators can rely on their ability to recognize familiar or unfamiliar patterns. The CVCS prototype incorporated a like-for-like digital annunciator window replacement with an alarm list. The digital replacements offer lower maintenance and replacement costs compared to their analog counterparts. The like-for-like replacement maintains the operator’s ability to scan the alarm boards and to respond to incoming alarms with existing procedures. The annunciator windows also provide a means of grouping alarms and prioritizing their importance. Less critical alarms can be sent to the alarm list. While the alarm list may not facilitate rapid scanning it does have some unique advantages over the annunciator windows. The alarms in the list are time stamped and can be interactively filtered to identify critical information.

One of the core innovative components of our COSS system is an advanced online sensor validation and fault diagnosis system (PRO-AID) developed at Argonne National Laboratory. The PRO-AID system actively monitors plant sensors and components. When a fault occurs it can detect and inform operators to abnormal conditions before plant variables exceed alarm thresholds. Once a fault is recognized by PRO-AID, the HMI highlights what component(s) that may be at fault. Computer Based Procedures (CBP) integrated with an expert knowledge system provide operators actions to mitigate undesirable plant events and return the plant to a safe operating condition with the least amount of upset possible. This additional information could be sufficient to avoid the costly endeavor of taking the plant offline.

Here we conducted an interface evaluation workshop with licensed operators. The workshop was intended to accomplish several goals. The first was to assess whether the COSS concept could aid operators during abnormal events. Secondly, we sought to capture operator impressions regarding the acceptance of COSS-like technology in the control room. The COSS prototype provided higher levels of automation compared to existing control systems. Operators may feel uncomfortable relinquishing control to technological systems. Lastly, we the operators were used to identify potential shortcomings of the COSS concept and to ideate potential remediations and improvements.

3 Method

In August of 2016 two crews of licensed reactor operators visited INL to participate in a LWRS-CRADA workshop. Each crew consisted of three individuals, and the crews participated on consecutive weeks. This allowed us to capture unbiased first impressions from each operating crew. The HSSL nuclear control room simulator was configured to represent the control room of the visiting crews. Prior to the data collection the crews conducted a small loss of coolant scenario to familiarize themselves with the glass-top controls and to validate the indicators and controls functioned as expected in the virtual control room.

3.1 Study Design

Each crew conducted a fault scenario with three variations of a CVCS control board. As a control condition the conventional analog control board was represented. In this condition the board was represented as it currently exists in the operator’s plant. A second condition represented a hybrid analog/digital control board with large overview displays and a digital HMI. The second condition represented currently available technology would be commercial available for control room upgrades. The final condition incorporated advanced COSS concepts that are not yet commercially available. The following subsections describe these conditions in more detail.

Description of Conventional Boards.

In the conventional layout the controls are arranged as a process. The mimic depicts the major components of the CVCS and their arrangement and function in the system thus providing the organizational structure on the board. The piping of the mimic is color coded to segregate the CVCS into sub-systems (e.g., letdown path is orange, charging and seal injection is red). The benefit of a mimic format allows operators to identify the function and status of the component from its placement in the mimic diagram without having to rely on memory and the component’s labeling or looking back and forth between the board and documentation. The tradeoff to the mimic layout is that the spatial arrangement of the indicators and controls is not intuitive if operators are not familiar with the mimic’s layout. During normal operations, most of the activity is concentrated on the right half of the board. The left half represents I&C for boric acid recovery, boration, and dilution. The indicators and controls on the board with green and red labels are safety related.

Description of Digital HMI.

The digital HMI implementation consists of two large overview displays placed throughout on the vertical sections of the control boards to provide at a glance monitoring of plant subsystems and overall plant status. The large displays are intended be visible from across the control room. Below the large overview displays are four touch panel displays for reactor operators to interact with the CVCS. The remaining instrumentation was envisioned as a hybrid of analog and digital indicators and controls. The safety-related I&C along with a few additional non-safety-related I&C were relocated to the apron section of the panel. The layout of the controls was organized in the form of process mimic to maintain the positive aspects of the mimic organization while capitalizing on training carry-over from operator experience with the conventional board.

The left overview is for monitoring CVCS. The overview is organized as a piping and instrumentation diagram (P&ID) and is intended for monitoring during normal and abnormal operating conditions. The right overview is for monitoring Reactor Coolant System (RCS) coolant inventory. The RCS coolant inventory overview allows operators to monitor the reactor status, steam generator levels, RCS loop temperatures, and the pressurizer. The RCS and CVCS are tightly coupled. Pressurizer level is controlled through CVCS letdown, and the CVCS provides make-up for small RCS coolant losses. The RCS board resides to the right of.

Below the large overview displays are four touch panel displays. No other input such as keyboard, trackpad or mouse was provided. Therefore, all control functionality was implemented such that it could be performed using a touch interface. Buttons on the display were made substantially taller compared to an interface designed for cursor input. For numeric entry an onscreen numeric keypad was presented. The touch displays allowed operators to monitor and control sub-systems and components of the CVCS such as seal-injection, boration, dilution, and automatic makeup. For the study, digital HMI screens and controls needed for the test scenario were developed. These included screens for monitoring seal injection and makeup and screens for controlling letdown flow, temperature, and back pressure as well as charging pressure.

The physical size of the displays was taken into consideration when laying out the content for the screens. Because the overview displays are roughly twice the height and width of the smaller displays, it is possible to have four times the amount of legible content on the overview screens.

All the HMI screens were implemented in a style known as dullscreen. With the dullscreen concept the screens appear monochromatic when the annunciators and instruments are within normal operating ranges, allowing high-contrast and salient color indications to grab the operators attention should something unexpected or noteworthy happen. The conventional approach to designing HMI screens identifies the minimum allowable size for text and other graphical elements and then “consistently” uses these minimum allowable sizes throughout the entire interface to maintain legibility. The downside is that everything is equally illegible. By employing graphic design and visual perception principles to the design, information can be hierarchically prioritized, and more pertinent information can be made more salient and legible to distant observers. Graphic design has long known that slight variations of font size, font weight, white-space, and typeface, and kerning can produce drastic differences in how information is perceived [8]. Graphic design is the art of manipulating these variations to produce a design that conveys the intended message. The science of visual perception excels at understanding the basic principles of contrast perception, text legibility, saliency, but is lacking when it comes to understanding how multiple nuanced elements to produce the gestalt, the whole perception of the features.

Description of COSS.

Among the numerous goals of the workshop, we sought to evaluate how operators used the conventional boards, compared to the hybrid boards with the digital HMI described above, to an advanced concept digital HMI known as a The COSS builds on the digital HMI by conceiving additional control technologies and user interactions. The COSS implemented Type 2 computer based procedure (CBP) system that provided real-time variable status embedded in the procedure and guidance for selection the appropriate path (see Fig. 2).

Fig. 2.
figure 2

The COSS features a type 2 computer based procedure, a fault diagnosis system and continuous monitoring of important plant variables. The purple lines are auto-scaling trends. (Color figure online)

Full size image

The COSS also behaved as if it had an underlying prognostic diagnosis system known as PRO-AID developed by Argonne National Laboratory. PRO-AID is capable of determining system faults such as leaks and blockages from available sensor data. The spatial sensitivity of the diagnosis is dependent on the richness of the available instruments. One of the unique features of PRO-AID is that it only requires defining the system at the P&ID level. The PRO-AID system then trains from steady-state data to be able to recognize faults. This is significantly more feasible than having to develop a first principles model representation of the system.

Fault detections from PRO-AID are conveyed to operators through the HMI screens using a highly salient and distinct yellow-green color. The CVCS overview is organized as a P&ID to support conveying fault diagnostics from PRO-AID. The fault diagnostics require highlighting sections of piping and components to show operators the location of a detected fault. The COSS would then diagnose and alert operators to the problem. The CBP was used to guide operators down the appropriate course of action.

Fault Description.

During normal operation some of the high-pressure RCS coolant is diverted the CVCS. The flow that is diverted and reduced in pressure through the letdown path. Then the flow passes through demineralization tanks and into a volume control tank (VCT). The VCT supplies coolant to charging pumps that increase the pressure so that coolant can be pushed back into the RCS. The fault scenarios caused letdown flow to become isolated or stop. In all scenarios, the letdown isolation event was produced by an instrumentation and control malfunction but the exact cause varied slightly to keep the operators from becoming complacent with the scenarios.

With the conventional boards, the letdown isolation resulted from a setpoint failure of a temperature controller that varies shell side component cooling water (CCW) flow to maintain the temperature of the letdown flow. The setpoint fails high with a ramp of 100 s which causes the CCW flow to increase briefly before closing to its minimum value of 20%. The reduced cooling flow through the letdown heat exchanger results in the letdown flow temperature increasing. A temperature interlock linked to a temperature controller downstream of the letdown heat exchanger was then triggered to close to prevent melting the resin in the demineralizers.

With the digital HMI and COSS conditions the letdown isolation is caused by a failure a temperature sensor failing high, then jumping around for a few seconds, then failing low. The COSS implementation also conveyed the fault diagnosis on the CVCS Overview (see Fig. 3). The cascade of events is the same as previously described.

Fig. 3.
figure 3

CVCS large overview display post fault condition resulting in letdown flow becoming isolated. The large overview was implemented with a dullscreen concept that increases the saliency of the alarms. (Color figure online)

Full size image

4 Results

Licensed reactor operators are a rare and expensive commodity when it comes to conducting human factors studies. As a consequence traditional quantitative performance measures are of limited validity due to sample size constraints. Here we relied on qualitative methods to elicit and capture operator feedback. Following each scenario an independent human factors consultant with 30+ years of experience in nuclear human factors engineering led a semi-structured discussion. The format presented the operators with the same set of questions for each condition. The semi-structured format allowed for additional follow-up questions and discussion. During discussion several human factors practitioners took notes. After the workshop these notes were compiled and several themes emerged with content relating to: layout, controls, automation, and COSS functionality. The high-level points pertaining to the CVCS COSS are summarized below.

4.1 HMI Layout and Style

Hierarchical Organization.

Operators expressed preference for hierarchical organization with task based displays. The displays should normally be dedicated to a single screen or set of screens belonging to a single subsystem even if it is possible to bring up a screen from any subsystem. It was recommended that the overview displays be as large as possible to permit the information on the display(s) to be readable from a distance. The overview screens should provide a holistic and rapid depiction of the system. Operators prefer use of graphical representations, mimics, colors and other coding techniques to facilitate recognition of important information. The overview screens should be intended for monitoring only and should not contain any soft controls. Task based displays should tailor the available indicators to the task. The presentation scheme needs to clearly differentiation between controls and indicators. Operators are trained to look for confirmatory indications after performing control actions. Operators would like to have feedback indicators co-located with controls. The tags and labels presented in the interface should be identical to the procedures.

P&ID Layouts.

P&ID layouts should resemble plant engineering and training materials. For example, if charging pumps are presented C, B, A from top-to-bottom in training materials, they should be represented in that same order in the HMI.

Display Clutter.

Operators are sensitive to the amount of information on a display. Detailed information should be available but should normally be hidden from view and made accessible as pop-up windows that only appear on demand. Operators preferred pop-ups to dedicating faceplate space for detailed panels. They felt dedicated screen space would be wasted when the faceplates are being used. Operators expressed that the use of trends should be carefully considered. In the correct context the trend indicators provide valuable information permitting operators to better to predict future states. But, too many trends can be overwhelming and could lead to what the operators called “death by information.” Operators had mixed feelings regarding auto-scaling Tufte styled very small line charts (sparklines [9]). Some operators expressed that they wish they could set the axis limits.

Use of Color.

The study found operators strongly disliked the dullscreen implementation of the HMI in which the use of color is reserved exclusively to convey only important information. Operators strongly preferred the traditional red and green valve status indicators, even when compared to high contrast monochromatic indicators within the dullscreen implementation. The interface incorporated black and purple trend lines to distinguish multiple axes. The operators thought that more contrast was needed between the two colors.

4.2 Controls

Maintaining Hard Controls.

Operators thought it was important to keep hard controls (analog buttons, dials, switches, etc.) for critical and time sensitive actions such as tripping a turbine or scramming the reactor.

Soft Control Accidental Activation.

Operators expressed anxiety about soft control buttons being accidently being clicked because of user error or spurious touch panel input. Operators suggested that certain control actions need to have confirmation dialogs to prevent control actions from taking place from accidental input. Operators even suggested that buttons should remove focus to avoid being accidentally triggered and that the cursor should automatically move away from clickable button if it is left on top of a button for a set period of time.

Touchscreen Reliability/Secondary Input Device.

Operators were concerned that a touchscreen failure could interfere with operations and suggested that a backup input device such as trackpad should be provided. In the event of a screen failure it should be possible to quickly and easily reconfigure what is shown on the displays. Operators also noted that they can use the mouse pointer to indicate what they are looking at on the screen to support peer checking.

Some operators are shorter, making it difficult for them to operate touchscreens. A trackpad on the apron would provide an ergonomic solution for these operators.

Ergonomic Considerations.

Standing workstations present ergonomic considerations to maintain touchpanels in the reach envelope of 5th percentile females to 95th percentile males by stature. In accordance with NUREG-0700 the font on the displays need to maintain at least 16 min of arc across individuals [10].

4.3 COSS Functionality

Computer Based Procedure.

They liked the capability to show plant data linked to a procedure step (decision aiding automation), which is defined as a Type 2 CBP system. They did not want the CBP system to take actions automatically to control the process (defined as a Type 3 CBP system) without their permission. They commented that the CBP system decision aiding automation was very useful, but suggested that problem diagnosis decision aiding also would be very helpful. They said that this capability should permit early identification of a developing problem and permit them to take earlier actions to mitigate the developing problem.

The CBP guided operators through procedures by highlighting the current step, as well as providing plant variable values within the procedure step itself. The COSS also provides contextual information within the procedure steps. Specifically, the COSS displays trend information within the procedure step to provide historical information about the relevant variable so that operators can assess abnormal fluctuations. Both crews noted the significant improvement with this integrated information.

Operators wanted the ability to be able to look ahead in the procedure. They also wanted the current step to be more apparent by being stylized differently.

An ad-hoc scenario variation where the reactor operators were using CBPs at the control boards versus at the senior reactor operator (SRO) workstation revealed operators might respond more quickly when CBPs are available at the board, but operators reported concern with a keyhole effect where they are inclined to focus too much on the CBP. A suggestion was that the SRO should have the ability to monitor their progression through the procedure from their workstation so that the SRO can maintain broad situational awareness of the plant and the information that the ROs are actively viewing.

PRO-AID Fault Detection.

The plants current instrumentation may not be sufficient for increased levels of automation or for diagnostic systems like PRO-AID. Plants may need to consider upgrading instrumentation to realize automation benefits.

Operators emphasized the importance of the interface to provide a transparent view of PRO-AID systems functioning so operators can validate the diagnostics and build trust in the system. The PRO-AID fault diagnosis system monitors sensors to detect faults and determines faults using logic that operators might use.

5 Conclusions and Discussion

The operators who participated in this study lacked familiarity with modern DCS capabilities and digital HMI concepts and functionality. We must remember most nuclear control rooms were originally designed and implemented several decades ago. The nuclear industry has an aging workforce. Control system upgrades are being implemented with more recent technology. In comparison to decades past, there are many cases were control automation can handle tasks at least as reliably as human operators. When control automation is adopted the role of operators shifts from continuously manipulating controls to monitoring and anticipating the automated system. In some circumstances this may be a philosophical departure from current operations. In particular, the COSS implemented higher levels of automation than operators were accustomed to, but operators expressed a desire for this automation after sufficient familiarity was attained and if sufficient reliability could be established. The information obtain from the evaluations will be incorporated into the CVCS-COSS and evaluated with licensed crews. The roadmap from prototype to actual control technology is long and arduous but we hope our operator centric design approach influence control room modernization in the short-term and lead to next-generation advanced control systems in the long-term. The PRO-AID fault diagnostic system plays an important role in making COSS technologically feasible, though there is still work to be done in regard to the underlying technology that would drive an actual COSS implementation. Our work here lays the design concept groundwork for how to integrate these technological systems once they mature.

6 Disclaimer

This work of authorship was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government, nor any agency thereof, nor any of their employees makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately-owned rights. Idaho National Laboratory is a multi-program laboratory operated by Battelle Energy Alliance LLC, for the United States Department of Energy.