A Hybrid Qualitative and Quantitative Diagnosis Approach

Parametric faults are detected and isolated using parameter tracking algorithms based on optimization algorithms or filtering techniques (e.g., Kalman filter, particle filter). Online, simultaneous tracking of all parametric faults can fails since there may be too many combinations of parameter values that explain the observed behavior. Hence, a correct diagnosis solution is not obtained. An alternative in the single fault case is to track separately each parametric fault in parallel and choose the one that best explains the observed behavior according to some chosen metric (e.g., mean square error). This approach is feasible but computationally expensive, since there may be too many tracking algorithms running in parallel. We propose using analytic redundancy relations (ARRs) to reduce the number of parametric faults that are tracked simultaneously. ARRs qualitatively point to a set of possible explanations but usually require a large number of sensors to achieve good isolability of fault causes. They induce a fault signature matrix (FSM) that can be derived offline. The parameter tracking algorithms will be instantiated for the faults in the set of possible explanations produced by the ARRs. By combining ARRs with online parameter tracking algorithms we can obtain a good tradeoff between computational effort and fault isolability. We demonstrate our approach by diagnosing faults in a rectifier circuit.


INTRODUCTION
Our goal is to design a model-based diagnosis engine for detecting and isolating parametric faults in a dynamical system.The diagnosis engine is provided with a model of the system, nominal values of the parameters and values of some of its inputs and outputs over time.There is a rich literature on model-based diagnosis results proposed independently by the artificial intelligence (de Kleer, Mackworth, & Reiter, 1992) and control (Gertler, 1998;Isermann, 2005;Patton, Frank, & Clark, 2000) communities.Data-driven approaches are widely used as well.They use statistical models (classifiers or regressors) trained with labeled data that describe the behavior of the system under various fault modes.They usually require a large amount of data to build statistically meaningful models.This requirement is not always easy to satisfy, since system are designed not to fail.In addition, we may lose explainability of the root causes.
In this paper we focus on parametric faults.A significant change in a parameter value as compared to its nominal value indicates a fault in the component associated with the respective parameter.There are several approaches for parameter tracking.Filter based approaches such as the Kalman filter (Kalman, 1960) and its extensions to non-linear systems (extended and unscented Kalman filters) are an option.They do not always perform well on nonlinear systems and non-Gaussian noise.Alternatively, we can use a particle filter algorithm (Arulampalam, Maskell, & Gordon, 2002).The computational effort required to implement such a filter may be prohibitive though.
Filters track the state of a dynamical system and hence the state needs to be augmented by including the system parameters.Tracking the state and all parameters simultaneously does not guarantee accurate results since a multitude of parameter value combinations may explain the system behavior.Another option for tracking the system parameters is to use an optimization based approach.Time series reflecting the system behavior are compared with the simulated behavior, and the model parameters are adjusted until the real and simulated behaviors match.Typically, the parameters are adjusted as part of solving a nonlinear least square problem.
Tracking multiple parameters simultaneously faces the same challenges as in the case of the filtering methods.Since the least square cost function typically depends nonlinearly on the system parameters, the more parameters we have the more local minima exist.Hence, unless we use global optimization algorithms, which are slower, we cannot guaranteed convergence to the global minimum.
When dealing with single fault cases, each parameter is tracked using an instantiation of an optimization algorithm.For large number of parameters this may prove computationally costly.Therefore we need a procedure to select a subset of these parameters.We propose using analytic redundancy relations (ARRs) to implement this procedure.
ARRs are constraints that when not satisfied indicate the presence of a fault.They induce a fault signature matrix (FSM) that shows which parameters are sensitive to the variables in the ARRs.It encodes signatures in the observations associated with the faults.We cannot guarantee that the signatures are all different.That is, several faults may have the same signatures.
The FSM structure depends on the number of sensors and what variables are actually measured.We use the fault signatures to derive a set of possible faults, and use optimization (or filtering) methods to estimate the parameters associated to a particular fault signature in parallel.This way we limit the computational effort by tracking only a subset of parameters.In this paper we track the parameter by applying optimization algorithms.The approach can be easily adapted to work with filter-based methods.
Paper structure: We start with a description of the problem in Section 2. In Section 3 we introduce background material on ARRs and optimization-based parameter tracking.In the same section we discuss how the two approaches can be combined to improve the computational efficiency of the parameter tracking procedure.In Section 4 we demonstrate our approach when diagnosing faults in a rectifier.

PROBLEM SETUP
The objective is to detect and isolate faults in dynamical systems expressed as a set differential algebraic equations (DAEs) of the form: where x is the state vector, u is the input for the system, y is the vector of output measurements and θ is a vector of system parameters.Often, we can separate the DAE into an ODE and a set of algebraic equations of the form where we have a new vector z called vector of algebraic variables.This is the type of DAE we will use in the example section.In addition, the sensing model is based on measuring a subset of the state and algebraic variables x and z.Time series of the input and output vectors are periodically produced and used by a diagnosis engine to detect and isolate a fault, if present.Such time series are of the form u T1:T2 = {u t k } N2 k=N1 , y T1:T2 = {y t k } N2 k=N1 , where For linear minimum-square problem, convergence can be tested by computing the rank of a particular matrix.For nonlinear cases, it is rather challenging to specify when we have enough information since we would actually need to know the local minima.We are left with choosing "long enough" time series and making them longer in case convergence is not achieved.Focusing on parametric faults only is not as constraining as it may seem.In our previous work we showed how we can introduce a variety of parameterized fault models based on the physics of failure.Under this approach the initial nominal model is augmented with fault modes, each mode depending on one or more parameters.The type of faults introduced are domain dependent.We cover electrical (short, open connections, parameter drifts), mechanical (broken flanges, stuck flanges, torque losses due to added friction, efficiency losses), or fluid (blocked pipes, leaking pipes) domains.
Fault augmented models enabled us to execute a number of system analytics tasks, ranging from fault diagnosis (Minhas et al., 2014), reliability analysis (Honda et al., 2014) to maintenance scheduling (Saha et al., 2014).Hence parametric faults cover a significant number of possible faults that can affect a system.
The example on which we test our approach is shown in Figure 1.It is a rectifier circuit powered by an AC source, with a smoothing capacitor and a resistive load.The diodes' behavior follow the ideal nonlinear characteristics where i s is the reverse bias saturation current, v T is the thermal voltage and n is the ideality factor.As we will see in the example section, the behavior of the rectifier is described by a DAE rather than an ordinary differential equation.
We are interested in detecting faults in the diodes represented as open connections, and drifts in the parameters C and R.

ARRS AND OPTIMIZATION BASED APPROACH FOR PARAMETER TRACKING
In this section we briefly describe how ARRs are used for fault diagnosis and how optimization algorithms are used to track parameters of dynamical systems.In the last part of this section we combine the two approaches.More details on the use of ARRs for fault diagnosis can be found in (Staroswiecki & Comtet-Varga, 2001), (Staroswiecki, 2000), (Samantaray & Bouamama, 2008).

Analytic redundancy relations
The mathematical representation of the system model is a set of differential equations that are used to produce ARRs.The ARRs represent various constraints among a set of known process variables (Staroswiecki & Comtet-Varga, 2001).To account for the modeling uncertainties and noise the thresholds δ i need to be adaptive.Typically, function Φ is of the form Φ(r) = |r|.This defines a basic change detection scheme.A smoothing of the change detection scheme be applied by using a moving average scheme to compute were the time is assumed to be discrete.Other change detection schemes can be employed, such as the cumulative sum control chart (CUSUM) test.We can define the cumulative sum quantities where d is the size of the shift to be detected.They can be used in a CUSUM based coherence vector definition: A fault is detected if c = [0, 0, . . ., 0], that is, at least one element in the coherence vector is non-zero.A fault is isolated by matching the coherence vector to a binary fault signature matrix (FSM).The FSM denoted by S embeds the structural sensitivity of each residual to the faults in the system components (Staroswiecki, 2000).The entries of the FSM are defined as A fault in a component is detectable if at least one residual is sensitive to it.Fault isolation is performed using the FSM.Note that if there are too few sensors, the resulting ARRs may not be able to differentiate among some faults.The ideal case is when each residual is sensitive to only one fault.This type of residuals are called structured residuals.If we try to monitor more faults than the number of sensors, we cannot construct structured residuals.A significant of ARRs is the fact that we do not need to explicitly include the fault models.A fault appears when a significant change in the variables and parameters of a component takes place.Therefore, a residual is affected by a component fault if the variables contributing to the computation of the residual are sensitive to the respective fault, ARRs can be determined using a structured approach, where a bipartite graph describes the dependence between the variables and the constraints that define the system behavior (Staroswiecki & Comtet-Varga, 2001).If a variable appears in a constraint then an edge in the graph exists between them.The structural analysis is done through matching on the bipartite graph: system variables are associated with the constraints from which they can be computed.ARRs are generated from over-constrained subsystems and are completely expressed when only known variables appear (inputs, outputs and parameters) (Staroswiecki, 2000).
The elimination of unknown variables is done iteratively.At the first iteration, we find the variables that can be computed directly from the current set of initial set of known variables.These variables are added to the known variables set which are used to express additional unknown variables at the next iteration.The algorithm stops when all variables are expressed in terms of inputs, outputs and parameters.The constraints that were unused in this process become the ARRs.We emphasize that this determines a structural dependency between known and unknown variables.This dependency may be broken for particular choices of system parameters.More details on the ARR-based diagnosis and other modelbased diagnosis techniques can be found in (Samantaray & Bouamama, 2008).In Section 4 we show how we can construct ARRs for the rectifier circuit.

Optimization-based parameter tracking
The goal is to estimate the parameters of the system using field data.The field data is represented by input and out-put time series u T1:T2 and y T1:T2 .The output time series are used to construct the least square cost function , where ŷk (θ) is the simulated sampled output having as input the time series u T1:T2 .The parameter estimates are computed as solution of the nonlinear least-square problem where Θ is a constraint set.The constraint set ensures that the system parameters take values that make physical sense.For example, the resistance of a electrical resistor must be positive.For box type constraints, that is , where θ i is an entry of θ, we can transform the constrained optimization problem ( 9) into an unconstrained one by applying the transformation θ i = θ min i + sin( θi ) + 1 (James & Roos, 1975), and minimizing J ( θ).Similar transformations can be used when the parameters are only partially bounded.Namely for −∞ < θ i ≤ θ max i we have . This is useful since typically, unconstrained optimization problems are simpler to solve, and there are more optimization algorithms available for such problems.
We can use both gradient-free and gradient based optimization methods to solve the unconstrained optimization problem.Both methods rely on model simulations.In the case of gradient-based methods, in the best case scenario we can symbolically obtain the gradient vector.When this is not the case possible options include automatic differentiation or numerical approximations.
We usually prefer to avoid approximating the gradient numerically, since the process introduces numerical errors that tend to accumulate.One avenue for automatic differentiation is using the deep-learning platform such as Tensorflow (Abadi et al., 2015) or Pytorch (Subramanian, 2018).This will require building customized objects that described the behavior of the system.If the system is dynamic, solving the optimization problem requires solving an ODE (or a DAE).Tensorflow already includes a numerical solver for ODEs.Alternatively, we can implement explicit numerical solvers such as the Runge-Kutta algorithms, the simplest of such solvers being the Euler first order approximation.Implementing such solvers is equivalent with designing a "custom" recurrent neural network (RNN), where the number of identical cells is given by the length of the input and output time series.Reasonable choices for gradient-based optimization methods are part of the class of nonlinear least square algorithms such as trusted-region-reflective or Levenberg-Marquardt (Kanzow, Yamashita, & Fukushima, 2004), which take advantage of the cost function's structure.
For large scale optimization algorithms, first order methods such the gradient decent algorithms and its extensions are the state of the art.Gradient-free methods include Nelder-Mead (Powell, 1973), pattern search (Hooke & Jeeves, 1961), or Powell (Powell, 1964) algorithm with its different versions (Powell, 1964).The optimization problem ( 9) is usually nonlinear and non-convex.Hence there are no guarantees that the optimization algorithm converges to the global minimum.There is the option of using global optimization algorithms (e.g., particle swarm optimization, (Kennedy & Eberhart, 1995), genetic algorithms, (Mitchell, 1998), evolution strategy) (Beyer & Schwefel, 2002)) but they are usually too slow.The optimization algorithm uses the model to generate the simulated output.If the model is dynamic, it requires the initial state for simulation.If the initial state is not available, we can treat its entries as optimization variables and estimate both parameter and initial conditions.
Note that estimating the parameters using an optimization based procedure is one option among many.Alternatively, we can use filtering based techniques based on the extended Kalman filter or particle filter (Arulampalam et al., 2002).

ARRs enabled parameter tracking
In the previous subsections we discussed how ARRs are used for diagnosis and how optimization algorithms can be used for parameter tracking.The ARRs approach gives an qualitative diagnosis, that is, it provides a list a parameters that are the cause of the abnormal behavior.Parameter tracking provides quantitative values for the parameters, which gives an idea about the severity of the fault.The isolability property of the ARR approach depends on what variables can be measured and how many.We have at most 2 m signatures with m being the number of sensors.Hence with two sensors we can encode in the best case scenario at most four modes, nominal mode included.Even if a signature corresponds to a list of parameters, rather than a single one, this list is typically a subset of the set of all parameters.Hence we need to track fewer parameters.In other words, the ARRs based diagnosis is used to filter out parameters that are unlikely to be responsible for the abnormal behavior.
Figure 2 depicts the architecture behind the combination of ARRs and optimization based parameter tracking.First the input and output time series are used to evaluate the ARRs and generate a coherence vector c.This vector is checked against the FSM which produces a list of possible parameters {θ i1 , . . ., θ i M } responsible for the abnormal behavior.Next, for each parameter θ ij , an optimization procedure is instantiated which estimates the parameter value θij that best explains the observed behavior.Finally, the likely parame-ter responsible for the faulty behavior is selected by choosing the one for which the cost function is the smallest, that is, θi * , where i * = arg min θ∈{θi 1 ,...,θi M } J(θ).

Figure 2. ARRs enabled parameter tracking architecture
The optimization blocks in Figure 2 can be replaced with filters that track each parameter θ ij , if filtering based techniques for parameter tracking are used.Since filters require the statistics of the initial state, we would need to track the state of the system until an abnormality is detected, and the statistics of state estimate at the respective point will be used as initial state for the bank of filters.

ILLUSTRATIVE EXAMPLE
In this section we demonstrate the ARRs enabled parameter tracking approach for parametric fault diagnosis.The schematic of the rectifier was introduced in Figure 1.The input for the rectifier is an sinusoid signal E = E a sin(2πf t).
The nominal parameter values of the rectifier's components are shown in Table 1.We consider two types of faults: open connections in the diodes and parameter drifts for the capacitor and resistor.The open connections are simulated by making the saturation current i S zero.
We first construct a set of ARRs and the FSM.We assume we measure two variables: potentials v 2 and v 3 .The behavior of the rectifier is described by the following DAE system: To solve the above DAE, it is needed to solve, at each timestep, the optimization problem where ( 20) is subject to the following constraints: This problem can in fact be simplified to an iterative Newton-Raphson algorithm where the iteration variable is the potential v 2 .Once v 2 is solved, the rest of the unknown variables can be computed.
To find the ARRs we apply the structured approach.Since we have two sensors we can generate two ARRs.Note that there is no unique set of two ARRs.In fact, any two equations from the constraints (10) -( 19) can serve as ARRs.However, they will induce the same fault signature matrix.The structured approach uses a greedy procedure to compute the unknown variables.In the case of derivatives, explicit equations are added.For example for v3 we add the equation This is a causal equation though, meaning that we can compute v3 from v 3 but not the other way around.The "computation" is in fact an approximation since we have samples of v 3 in time.Hence we have 11 equations and 9 unknowns.From the measurement variables v 2 and v 3 , in the initial step we can find solutions for the unknowns v 1 , i 2 , i R , i 4 and v3 from constraints 10, 15, 17, 14 and 27, respectively.In the second step, we can derive the unknowns i C , i 1 and i 3 from the constraints 19, 12 and 11, respectively.Finally, in the third step we can compute i E from constraint 13.
We have solved for all the unknowns and we are left with the constraints 16 and 18.They are the ARRs for our example, resulting in the residuals r The next step is to construct the FSM.This involves tracking the dependence of each variable in the constraints 16 and 18 on the parameters or the components of the system.In particular, we will track the dependance of the variables on the four diodes and on the parameters C and R. For example, i 2 depends only on the state of diode D 2 while i E depends on the state of diodes D 1 and D 3 .We repeat the dependency analysis for all variables involved in the ARRs, resulting in the FSM depicted in Table 2.
Table 2. Fault signature matrix We discover that we can differentiate between three pairs of components {D 1 , D 2 }, {D 3 , D 4 } and {R, C}.The detection thresholds can be chosen based on the desired sensitivity of the ARRs.For example if we want to detect a ±25% change in the parameters R and C, we would need to choose a threshold of maximum 0.01 for the average residual values.This value was derived by ignoring any noise in the measurements.It would need to be revised to accommodate for the noise when used on a real system.
Once a fault is detected using the ARRs, a list of possible causes is generated.For each possible cause, an optimization algorithm is instantiated to find the best explanation.Using the magnitude of the ARR residuals to draw a conclu-sion may be misleading.For example a short in resistor R generates residual values comparable in magnitude with the residual values generated in the case of an open connection in D 1 .For the resistor and capacitor we track the parameters R and C. In the case of the diode faults, we track the saturation currents i S .
We used Powell's gradient free algorithm for estimating the fault parameters.With this approach we do have to worry about the gradient approximation.Since the number of optimization variables is small, the algorithm runs fast enough.
Tables 3 and 4 show the results when the ARRs indicate that either R or C may be faulty.The first column of the table depicts the real mode.This means that the second row corresponds to the resistor having drifted from 100 Ohm to 150 Ohm.We estimate each parameter at a time (columns 2 and 3) and the results when R and C are estimated simultaneously.
Table 3 presents the parameter estimates while Table 4 shows the cost values computes at the estimate values.First we note that the cost estimates can indeed be use to differentiate between the faults.We also note, that when estimating the two parameters simultaneously, the results are not correct although the resulting cost value may be small.
We repeat the same steps for differentiating between faults in diodes D 1 and D 2 .
Table 5. Optimization results for faults in D 1 and D 1 -saturation currents estimates mode 1e-22 7.57e-13 1.05e-22, 1e-12 îS2 = 0[A] 7.19e-13 9.99e-23 9.99e-13,1e-22 We note that in this case even when estimating simultaneously the saturation currents in the two diodes we get correct results (recall that the nominal value for the saturation current is i S =1e-12 A).Since the optimization problem in nonlinear and usually non-convex, we cannot guarantee convergence to  the global minima.We can imagine an architecture where different types of optimization algorithms initialized at different initial conditions are instantiated.We can then select the parameter estimates based on their cost functions evaluated at the estimates.The approach discussed above can be applied online as well, by using a moving window idea to generate time series.This provides the advantage of a good guess for the initial conditions of the optimization variables under the assumption that there are no sudden changes in the parameter values.Some of the simulated residual values take values unrealistically large.In a real system we would expect these value to be between some physically realizable values.Still, we expect them to be significantly large to induce correct values in the coherence vectors.

CONCLUSIONS
We discussed an approach under which qualitative based diagnosis is used to improve the numerical efficiency of an opti-mization based approach for parameter estimation.The qualitative diagnosis uses ARRs to generate a list of candidate components/parameters that would explain an abnormal behavior.It is based on a FSM derived offline.This list is used to instantiate a set of optimization algorithms that learn separately the values of the parameters in the list.We demonstrate our approach for diagnosing faults in a rectifier circuit.
As a future work we plan to extend our approach to handling non-parametric faults as well.Parasitics, for example, is a growing problem in modern integrated circuits and plan to revise our approaches for diagnosing these kinds of problems.
with h the sampling period.The diagnostic engine processes the time series and produces a diagnosis result reflecting what parameter has drifted and by how much.The engine processes the time series all at once unlike the filter based implementations that can process data one sample at a time.The length of the time series should sufficiently large to ensure convergence of the optimization algorithm.This is equivalent to ensuring the invertability of the Hessian of the cost function at a local minimizer.

Figure
Figure 1.Rectifier circuit

Figures
Figures 3a-3b depict the residuals r 1 and r 2 over a time interval of 0.1s, and their average values, in the nominal model.As we expect, their values are close to zero.Figures 3c and 3d depict the residuals r 1 and r 2 over a time interval of 0.1s, and their average values in the case diode D 1 suffers from an open connection fault.The simulated residual values are unrealistically large, but it has the correct signature.

Figure 3 .
Figure 3. Residuals r 1 and r 2 for various nominal and fault modes

Table 3 .
Optimization results for faults in R and C -parame-