Event-driven adaptive intermittent control applied to a rotational pendulum

Intermittent control combines open-loop trajectories with feedback at discrete time instances determined by events. Among other applications, it has recently been used to model quiet standing in humans where the system was assumed to be time-invariant. This article expands this work to the time-variant case by introducing an adaptive intermittent controller that exploits the well-known self-tuning architecture of adaptive control with a Kalman filter to perform online state and parameter estimation. Simulation and experimental results using a rotational inverted pendulum show advantages of the intermittent controllers compared to continuous feedback control since the former can provide persistent excitation due to their internal triggering mechanism, even when no external reference changes or disturbances are applied. Moreover, the results show that the event thresholds of intermittent control can be used to adjust the degree of responsiveness of the adaptation in the system, becoming a tool to balance the trade-off between steady-state performance and flexibility against parametric changes, addressing the stability–plasticity dilemma of adaptation and learning in control.


Introduction
Event-driven intermittent control (IC) [1][2][3][4] has been proposed as a model that explains the underlying control mechanisms of human balance. At its core, the fundamental property of alternating between open-and closed-loop configurations provides two main advantages: (1) the open-loop evolution reduces the overall bandwidth of the controller while providing resources to perform state-dependant optimisation 5 and (2) serves as a mechanism to modulate the relationship between exploration and exploitation or, in other words, the stability and plasticity trade-off that is critical for adaptation and learning. 6 While IC has been used in the physiological control context, 7-12 its engineering origins come from the practical implementation of model-based predictive control (MPC) in the presence of hard constraints 13 and the observer-predictorfeedback architecture, 14 supported by experimental results on physical real-time systems. 15,16 While a variety of different IC implementations exist, 3 the form of IC considered in this article relies on the principle of continuous observation and intermittent action, meaning that the states are monitored all the time, but only used to recalculate the control signal at discrete points in time (or events) which are determined by a triggering mechanism. Between events, the controller applies an open-loop control signal that is generated by a generalised hold and evolves using its own internal states. When an event occurs, the controller uses observed states instead to produce a new control signal for the next open-loop interval.
IC has been extended recently to include the multivariable case, with an emphasis on multi-link unstable models of human standing, 4,6,17 where it has been argued that the open-loop intervals and the impulse-like control signals in IC could benefit adaptive schemes in situations where model uncertainties are present or when the system parameters vary with time. This is based on the idea that the variability caused by the open-loop interval results in events and corresponding control actions that excite the system more often than a controller with less sensorimotor variability, facilitating system identification. This hypothesis was tested in the context of human control, 18,19 verifying that an intermittent strategy allowed subjects controlling a virtual inverted pendulum with a joystick to perform better against parameter variations compared to subjects that used a continuous strategy. Gawthrop et al. 4 applied adaptive intermittent controllers to simulate human balance control in single-input single-output scenarios, using a formulation that solved the online identification problem via state-variable filters, using a non-minimal state-space representation of the system. A self-tuning architecture was used, 20,21 which separates the algorithm in an online system identification stage followed by a controller redesign.
In this article, an adaptive intermittent controller is introduced by combining joint Kalman filters that perform online state and parameter estimation in a single routine, using the aforementioned self-tuning architecture. In addition, this controller uses a closed-loop representation of the overall system dynamics to generate open-loop control trajectories and is based on earlier initial results. 22 This particular type of hold is known as the system-matched hold, 23 and it provides a refer-ence model of the ideal behaviour of the system under the influence of a control signal. The framework is not limited to this type of hold; for instance, an intermittent tapping hold 24 could be incorporated to produce the inter-sample behaviour. In IC, an event triggers the use of the observer states for feedback; however, since the joint Kalman filter also estimates a vector of timevarying parameters, these values are used to recalculate the different components of the intermittent controller before applying the final control law, providing an adaptation layer to IC.
The next section of the article introduces our adaptive IC architecture. Thereafter, the simulation results are presented, followed by experimental results, leading to the conclusions drawn from this work.

Continuous control
The adaptive intermittent controller presented in this article is based on a continuous predictive controller (CC) which is shown in Figure 1(a). This model assumes that the controlled system can be written as a linear state-space model of order n, as follows The quantities y(t), u(t), and w(t) represent outputs, inputs, and reference, respectively. The product x ss w(t) is the vector version of w(t), and the observed states are defined by x o (t). The Predictor generates future states x p (t i ) to compensate for the time-delay t d . The state-feedback block labelled as State FB represents the gains that are used to compute the control input u(t) continuously. (b) IC uses a Hold to produce internal states x h (t) which are compared against x w (t) by a Trigger mechanism. If the difference exceeds a predefined threshold q, then an event is created at time t i . The hold states x h (t) are used to compute the control signal u(t) between events which evolves in an open-loop configuration, until it is reset at times t i by the Predictor. The dashed lines represent quantities that are defined only at event times.
where x 2 R n , y 2 R n y , and u 2 R n u are the system state, output, and input, respectively, and t corresponds to continuous time. A is a matrix with a dimension of n 3 n, B is n 3 n u , C is n y 3 n, and x 0 is an n 3 1 vector of initial conditions. The dynamical model in equation (1) is represented in the diagram by the block labelled as System. The Observer estimates the system states x o (t) continuously, which are passed to the rest of the structures in the feedback loop after a reference input The steady-state component x ss is calculated offline, and it is part of the design process described in the next section. The Predictor compensates for possible delays in the loop by generating the future states x p (t). To generate a control law that stabilises equation (1), we could resort to an underlying continuous design (UCD) stage, which involves the design of a state-feedback controller, state prediction, and the introduction of steady-state components.

Underlying continuous design
A state-feedback controller with gain k is used to formulate a control law of the form which is used to stabilise the system in equation (1).
Here, w(t) is an external reference signal, t d is a constant time-delay, and x p (t) is the predicted state vector at time t + t d based on the data available at time t. 2,14 The delay t d can be included to account for transmission and computational delays in the human operator, which is an assumption derived from human motor control and physiology. 2 This implies that the delay is part of the controller in the feedback loop. A standard linear quadratic regulator (LQR) approach 25 is used to obtain k, this involves the minimisation of the LQR cost function and the solution of its associated algebraic Riccati equation. Both x(t) and u(t) in equation (3) are weighted by the diagonal design matrices Q c (an n 3 n matrix that must be positive semi-definite) and R c (an n u 3 n u positive definite matrix). The state vector x p (t) used in equation (2) is calculated via a state-predictor of the following form The reference w(t) is introduced by considering that the system has reached the steady-state where x f , u f , and y f are the steady-state versions of the states, inputs, and outputs, respectively. The goal is to ensure that y f = w(t) at all times, which can be achieved by substituting the expressions x f = x ss w(t) and u f = u ss w(t) in equation (5) and cancelling the common factor w(t), resulting in the following system from which x ss and u ss can be obtained Both x ss equation and u ss can be computed offline.

Intermittent control
The general IC architecture, 2,4,16 shown in Figure 1(b), builds up on the previously introduced CC by adding two fundamental components: The generalised Hold uses the predicted states x p (t i ) to create an open-loop hold state x h , which is used to calculate the control trajectories via the state-feedback gain k. These control trajectories are applied between the events imposed by the Trigger mechanism at discrete points in time t i . The trigger mechanism involves continuously comparing the Predictor states x p (t) with the state and x w (t) triggering an event if the difference between them exceeds a predefined threshold q. This action closes the feedback loop at t i .

Intermittent control time frames
The different time frames that are used in IC are defined as follows: 1. Continuous time (t): time that describes the system evolution. 2. Discrete-time (t i ): instances of time that indicate when an event has been generated, shown as subscript i. The elapsed time between consecutive events is the intermittent interval D ol i = t i + 1 À t i . 3. Intermittent-time (t): when an event is generated at t i , the continuous-time variable t is reset using t = t À t i . A lower limit D ol min . 0 is established for every intermittent interval such that D ol i . D ol min . The lower limit D ol min is commonly known as the minimum open-loop interval.

The system-matched hold
The open-loop behaviour of IC is dictated by the states that the hold produces, which are used to generate a control input of the following form where x h are the hold states, which evolve in time according to t, with dynamics generated by an autonomous system as follows The dynamics of equation (8) are determined by A h . To produce hold states that are an approximation of the real system states (in the absence of disturbances or noise), A h can be matched to the behaviour of an ideal delay-free closed-loop system of the following form with dynamics dictated by the closed-loop matrix A c = A À Bk. This choice of hold, where A h = A c , is known as the system-matched hold 23 and provides a suitable reference model for the detection of trigger events. Its simplicity makes it particularly attractive from an implementation point of view, and for this reason, it has been used widely to implement intermittent controllers. Alternative versions are also possible, such as the tapping hold. 24 At the start of each intermittent interval, t = t i , the hold states are reset to the predicted state Details on how to obtain the predicted states x p in an intermittent context are given in the following section.

Intermittent prediction
To compensate for a possible time-delay t d , the following dynamical system, based on the system matrices from equation (1), can be established during the inter- and evaluated at t = t d . Combining equations (11) and (8) yields the following extended system where X is defined as X takes the following form The solution of equation (11) at t = t d yields Thus, the 2n 3 2n matrix E = e A ph t d can be used to obtain the predicted states x p (t i ) at every intermittent interval, as discussed in Gawthrop et al. 4 Event detection Event-driven IC generates an aperiodic sequence of events. This process starts with the continuous comparison between the estimated states x w (t) and the predicted states x p (t), to compute the error e x = x p (t) À x w (t). Events are produced when e x exceeds a threshold q; this relationship is introduced with a quadratic criterion as follows The positive semi-definite matrix Q t selects which states contribute to the event generation process.

Thresholds and open-loop intervals
The threshold value q and the minimum open-loop interval D ol min are two of the most relevant parameters in IC. In our framework, setting the threshold to q = 0 generates events at constant time intervals (clock-driven or timed mode) 4 that are equal in length to the minimum open-loop interval D ol min . For q = 0, the response of IC converges to the response of the equivalent continuous controller if D ol min approaches zero, implying that CC is included in IC as a special case. The examples by Gawthrop et al. 4 provide an in depth explanation of the effects of these parameters on simulated systems.

Adaptive control
The continuous and intermittent controllers from Figure 1 can be extended to account for time-varying system parameters by adding an adaptation layer. In this work, this is achieved by using a self-tuning architecture 20,21 that runs a state and parameter estimation algorithm continuously, which tracks both, the evolving states and the system parameters, to then redesign the controller based on the updated parameter values. In the adaptive continuous controller, this redesign is executed at every iteration, whereas in the adaptive intermittent controller, this is done only once an event is detected, that is, at t i . These concepts are illustrated in Figure 2, where the block diagram of a general adaptive intermittent controller (Figure 2(b)) is presented alongside its continuous counterpart (Figure 2(a)). The online estimation problem implies the use of a recursive algorithm capable of tracking parameter changes in real-time using only the control input u(t) and the system output y(t). This can be done using techniques such as recursive least squares; 21 however, the controllers introduced in this article exploit the flexibility of joint Kalman filters 26 to estimate not only system parameters but also the states, in one single routine. We will start by establishing the main aspects of the redesign stage for both controllers, to then focus on the estimation algorithms.

Adaptive continuous control
The entries in matrices A, B, and C in equation (1), are linear expressions of the system parameters. If a parameter estimation routine is used to update specific entries in these matrices, then a controller redesign can take place. Assuming that these parameters are obtained in real-time by a recursive estimator, a new set of system matrices can be computed as follows The vector u m (t) contains the selected system parameters to be estimated continuously. The recomputation of matrices A and B for the adaptive continuous controller takes place at every iteration; this operation can be considered as the initial step of the redesign process. Once the new system matrices are obtained, the quantities established in the underlying continuous design stage have to be recomputed. This involves solving the system in equation (5) to obtain x ss and u ss , followed by a new state-feeback gain k generated via the LQR approach and the computation of the predicted states x p (t) using equation (4). The final control law that is applied to the system is given in equation (2).

Adaptive Intermittent Control
The basic approach of adaptive IC is the same as for continuous control, with the fundamental difference that adaptive IC has to go through the redesign cycle less often than adaptive CC, which is an advantage from a computational point of view: the set of system parameters u m is estimated continuously but is only used at event times, t i . Based on this, the estimated matrices A and B are defined as This means that A i and B i stay constant until the next event occurs at t i + 1 , that is, during the open-loop interval D ol i . With the new system matrices defined at t i , the re-computation of the steady-state components and state-feedback gains can be performed giving x ss (t i ), u ss (t i ), and k i = k(t i ), using the same methods as before.
Adaptive IC requires the definition of additional quantities that depend on the newly obtained k i and The events from the standard IC are used not only to sample the states x w (t) but also the estimated system parameters u m (t) at t i . This yields a set of redesigned controller values u c (t i ). Grey blocks show the components that provide adaptation capabilities and the dashed lines are quantities defined only at t i .
the system matrices A i and B i . One of them is the hold mechanism, A h , defined in equation (8), which dictates the behaviour during the open-loop interval D ol i and is matched to the closed-loop matrix A c when an event is generated, that is Expression (20) implies that A h stays constant throughout the open-loop interval. To simplify notation, we will write A h i = A h (t i ) to represent the hold at event times t i .
The aforementioned hold mechanism uses the predicted states x p (t i À t d ) at t i to reset its internal state as shown in equation (10). The operations involved in the calculation of x p depend on the dynamical system defined in equation (12), which has a solution at t = t d given by equation (16). This solution relies on matrix A ph , which needs to be re-calculated when the parameters are updated at t i Finally, an IC law is established as follows where the hold states x h (t) are reset according to equation (10) when there is a feedback event.
The stability of the proposed controllers is guaranteed by the underlying continuous design methodology that serves as a basis for IC, as it defines the closedloop performance and the characteristics of the response. In this work, the LQR approach 25 was used to obtain values of the feedback gain k that result in stable closed-loop behaviour (see equation (9)). Other design approaches could also be used such as directly assigning the location of the closed-loop poles via poleplacement methods. A more detailed treatment of the stability properties of the system-matched hold version of IC is given in Gawthrop and colleagues. 4,23 The general IC framework can be designed in such a way that constant disturbances affecting the system are accounted for, for example, by including a disturbance observer as a part of the state estimation process. 2,4 In order to simplify the analysis for this article, no external disturbances were added to the simulations or to the real-time experiments, and a disturbance observer has not been included in the system.

State and parameter estimation
As discussed above, the general structure of the Kalman filter allows it to be implemented as a joint estimator, which tracks both system states and parameters in a single algorithm. 26 Let us assume the following model for the system The process and measurement noise vectors, v x 2 R n and v y 2 R n y respectively, are considered Gaussian with zero mean, with covariance matrices Q and R that are known. In a Joint estimation process, the state vector x(t) is augmented with the selected system parameters as additional states, making the assumption that they do not change through time. If the augmented state vector is defined as x aug (t) = x(t) u m (t) ½ T , then the updated model can be written as follows The Kalman filter formulation is then applied to the model defined in equation (24). In this work, a nonlinear version of the filter known as the unscented Kalman filter (UKF) 27 is implemented, where statistical transformations are used to avoid the linearisation process involved in linear formulations of the filter, such as the extended Kalman filter (EKF). 28 The implementation details of the UKF are given in Appendix 1.  Table 1. To design the adaptive controllers for the rotational pendulum, first the augmented model that adds the system parameters as extra states to the original state-vector needs to be derived. These system parameters will be estimated in order to update the corresponding controller in the redesign stage. For this system, the statevector is defined in equation (47), which needs to be augmented by the system parameters to obtain the form given by equation (24). The arm and pendulum angles are the outputs of the system and can be defined in vector form as y(t) = ½ u a T . Considering the case where the mass of the pendulum m p and the length l p are time-varying parameters, u m (t) = ½ m p l p T , then the augmented state vector upon which the UKF is designed can be written as

Augmented rotational pendulum model for parameter estimation
where u and a are the arm and pendulum angles, respectively. By dropping time dependency notation for simplicity, establishing u = x 1 , a = x 2 , _ u = x 3 , _ a = x 4 , m p = x 5 , and l p = x 6 , and with the help of equation (25), it is possible to write where z = ½ € u € a T and f(Á) being The function f( Á ) corresponds to the solution in terms of the angular accelerations based on x aug and the torque u, which serves as the only control input in the system. The matrices M and C, as well as constants D r and D p , are defined in Appendix 2.

Controller design and UKF parameters
The controller and joint Kalman filter parameters that were used for the simulations are shown in Table 2. In the first section, the timing and threshold parameters are shown (applicable only to IC), followed by the LQR controller design matrices R c and Q c which are defined offline to determine the state-feedback gain k. 25 The UKF algorithm parameters (described in Appendix 1) that need to be initialised before execution are shown in the final section of Table 2, including the initial error matrix P o , the process noise Q, and measurement noise R covariance matrices. The off-diagonal elements of all of them are 0. Both P o and Q have a dimension of 6 3 6 due to the UKF being applied to the augmented state-vector defined in equation (25). The values of the parameter that controls the spread of the sigma points involved in the unscented transformation, a ukf , and the a-priori state distribution, b ukf , are shown for reference (Wan and Van Der Merwe). 29 The time delay, t d , was set to a very small value (i.e. 0:003 s), as the delays are thought to be negligible.

Simulation scenarios
In this section, simulations with time-varying system parameters are evaluated. Both adaptive CC and IC were implemented using the UKF as a combined stateparameter estimator, which was started from the following vector of initial conditions x o (0) = ½0 5:15 0 0 0:4 0:5, this assigns 5.15°to the pendulum angle u and the values of 0.4 kg and 0.5 m to m p and l p , respectively. As shown in Figure 2, v u 2 R n u and v y 2 R n y correspond to the input and measurement noise vectors, respectively; since, there are two outputs in the system (n y = 2), then v y is also a two-dimensional vector. These signals were defined as randomly seeded Gaussian noise with jv u j = 0:01 and jv y j = 0:001 as the respective amplitudes.

Simulation results
Continuous vs Intermittent Control. In the first simulation, the pendulum arm angle u follows the reference w(t), which initially switches between 10°and 210°with a period of 10 s, while keeping the pendulum angle a as close as possible to 0°. After 30 s, the reference for u becomes 0, cf. Figure 3(a, b). The pendulum length, l p , and its mass, m p , are varied during the simulation, which effectively changes the dynamics of the plant at different times. The nominal value of m p is 0.127 kg, which is used to start the simulation. At t = 7:5 s, it is increased by a factor of 2. The nominal starting length is l p = 0:168 m, which is increased by a factor of 2 at t = 17:5 s. After 32.5 s, m p is returned to its nominal value while l p returns to its starting value at 42.5 s. Finally, these two parameters are doubled once again at 52.5 and 62.5 s, respectively (cf. Figure 3(c) and (d)).
The results of the simulations are shown in Figure 3, where the estimated anglesû andâ are shown in (a) and (b), the estimated parametersm p andl p in (c) and (d), and the control input u(t) as well as t (which applies to IC only) in (e) and (f). The results in red correspond to CC and the ones in green are for IC. The controllers started using the estimated parameters at t a = 3 s, shown by a grey vertical line. From Figure 3(a) and (b), we can see that both angles follow closely the corresponding reference. During the initial part of the simulation, where the system is excited due to periodic changes in the arm angle reference, both CC and IC estimate the updated values of m p and l p well ( Figure  3(c) and (d)); however, IC does this faster than CC, continuously, without depending on the excitation due to the change in reference angle (between 5 and 10 s for m p and between 15 and 20 s for l p ). In contrast, CC only adjusts to the correct parameter value when the reference w(t) changes at t = 10 s and t = 20 s.
Interestingly, when the reference w(t) switches from a tracking regime to a regulation case (after t = 30 s), the parameter estimates for IC in Figure 3(c) and (d) converge to their expected values, whereas CC is not capable of estimating the parameters correctly with the estimation error growing after t = 50 s. This is likely due to a lack of excitation in the CC control signal u(t) (cf. Figure 3e). On the contrary, IC generates a control signal that is higher in amplitude, compared to CC, ensuring persistent excitation which benefits the parameter estimation process.
The effect of the event threshold. The event threshold q has a critical role in IC, defining how sensitive the system will be to errors and therefore establishing if feedback is used more or less often. In the context of parameter adaptation, the threshold has the added benefit of acting as variable that we can manipulate to control how much the system is excited, which affects the parameter estimation rate of adaptive IC. To demonstrate this effect, we evaluated the adaptive IC in a regulation case (w(t) = 0 for both angles). Two sets of parameter changes for m p and l p were established, the first is applied between 0 and 50 s, followed by a second change between 50 and 100 s. Both of these changes were equal in terms of amplitude, doubling the nominal value of the parameter when the change was applied. During each of these time intervals, a different value of the threshold q was used, starting with 0.1°(i.e. less than in the previous simulation) and switching to a threshold of 1.5°(i.e. larger than previously). All other simulation parameters remain unchanged (cf . Tables 1  and 2).
The results of this simulation are shown in Figure 4 which displays the same quantities as in Figure 3. The moment in time when the value of the threshold changes is indicated by a green dashed vertical line at 50 s, denoted as t q . The estimated parametersm p and l p , shown in Figure 4(c) and (d), exhibit different estimation rates directly after their true values change. For a threshold of 0.1°(t \ 50 s), the estimated parameters slowly converge to the true values. Once the second threshold of 1.5°is enforced at t q , the estimates converge at a significantly faster rate, that is, almost instantly form p and in around 3-4 s forl p .
These different parameter estimation rates can be explained by the effect of the threshold q on the system excitation through the control input u(t), which is . Responses for the simulation study, using adaptive IC, when the thresholds are varied throughout the simulation: (a) estimated arm angleû, (b) estimated pendulum angleâ, (c) estimated pendulum massm p , (d) estimated pendulum lengthl p , (e) control input u(t), and (f) behaviour of the intermittent time t. The time when adaptation is enabled, t a is represented by a grey vertical line at 3 s. The reference w(t) is 0°for both the arm and pendulum angles. The threshold q takes a value of 0.1°from 0 to 50 s, and after 50 s, its value is set to 1.5°. The time when the threshold changed is indicated with a green dashed vertical line and labelled as t q .
visible in the estimated anglesû andâ (cf. Figure 4(a), (b) and (e)). Increasing the threshold value results in larger variations in the output, associated with a larger control input amplitude.

Real-time experiment
An experiment was performed using a real-time platform that includes a servo motor (SRV-02) that provides a torque to a rotating arm (ROTPEN-E), both modules are made by Quanser, see also Appendix 2, Figure 6. The two outputs (angles of the arm, u and pendulum, a) were measured by two incremental optical encoders and collected for processing with a National Instruments acquisition card (PCI-6024E). The arm angle u was regulated via the voltage that the DC motor in the SRV-02 module produces. All controllers were implemented in MATLAB (Mathworks Inc.) using the Real-Time Windows target toolbox and zero-order hold approximations with a 1-ms sample interval. The values of threshold q, the delay t d , and the minimum open-loop interval D ol min , that were used in the experiment were unchanged from the simulations and are defined in Table 2.
Both CC and IC were used to stabilise the pendulum angle a, and a periodic square signal w(t) was used as a reference for the arm angle u, ranging from 210°to 10°. In this experiment, only the pendulum mass m p was estimated as a system parameter; as a consequence, the last element of x aug in expressions (25) and (26), which corresponds to the length parameter l p , is removed. The total duration was 60 s, starting with 30 s of evolution based on a design that used a pendulum mass value of m p = 0:07 kg, which is different compared to the nominal value in Table 1. At t a = 30 s, the controllers were allowed to use the estimated parameters for redesign purposes, and this feature remained active until the end of the experiment. Figure 5 shows the outputs u in Figure 5(a) and a in Figure 5(b), the estimated parameterm p in Figure 5(c), as well as the control input u(t) in Figure 5(d) and the open-loop time t in Figure 5(e). While oscillations around the reference w(t) for u can be observed throughout the experiment, these decrease after the redesigns take place at t a = 30 s. In terms of amplitude, the oscillations are similar for CC and IC. The pendulum angle a, in Figure 5(b), also shows a slight reduction in amplitude when adaptation is enabled.

Results
The parameter estimates (shown in Figure 5(c)) for CC start deviating from the true value after t a ; while the IC estimates stay close to m p for the entire duration of the experiment. The values of t in Figure 5 Figure 5(d) shows clearly the effect of the parameter mismatch (before t a ), where high control values are produced early in the experiment, these being higher for IC. When adaptation is enabled, the amplitude of the inputs reduces significantly because the estimates of m p , which are closer to the nominal value, are used to improve the controllers.
Comparing the results shown in Figure 5 with those of the simulation studies in Figure 3 and 4, it can be observed that the arm and pendulum angles for both IC and CC exhibit significantly larger oscillations in the case of the real-time system when compared to the simulated model. This is a result of un-modelled dynamics of the physical system, such as friction and gear backlash. Similarly, the open-loop periods (t) in Figure 5, show a different pattern compared to the ones observed in simulation, as a result of the induced oscillations. Both u and a follow their respective references; however, since the oscillations push these angles out of the no-triggering region imposed by the threshold q, the open-loop intervals in t exhibit a quasi-periodic cycle of slow and fast triggering (corresponding to low and high values of t).

Conclusion
The results of this article show the feasibility of adaptive intermittent controllers based on state and parameter estimation using joint Kalman filters. In our Kalman filter framework, an extended state-vector is used which is augmented with time-varying system parameters as additional states which are to be estimated. The estimated parameters are then used in a redesign stage to update the controller, leading to a self-tuning architecture. The adaptive intermittent controller was compared in simulations with an equivalent adaptive continuous controller. The adaptive intermittent controller was further evaluated on a real-time experimental control scenario. The results demonstrate how these adaptive controllers are useful in cases where the parameters of a system have a time-varying nature or when the model parameters assumed for control design do not approximate reality. The following list includes important remarks about IC and CC in an adaptation environment: 1. The use of IC results in persistent excitation of the system, which ensures that parameter changes are detected almost immediately and that they are tracked even if the system is not otherwise excited by a change in reference (cf. Figure 3). In contrast, CC requires external excitation (through a change in reference) to detect parameter changes. In the real-time experiment (cf. Figure 5), the pendulum mass estimate drifts away slowly from its true value due to missing excitation when CC is used.
2. The threshold q can be used to adjust the degree of adaptation (i.e. the sensitivity to parameter changes). Smaller thresholds generate less excitation and consequentially slower adaptation (cf. Figure 4). 3. In adaptive IC, the controller redesign only happens at event times t i , which provides a computational advantage compared to CC where the redesign is done every iteration.
The adaptive IC introduced in this article relies on the concept of combining feedback and open-loop control. Opening the feedback loop as a mechanism to discover the causal links relating state variables and inputs has been mentioned before, 18,30 an idea that results from the observation that when a system operates under the influence of continuous feedback control, it becomes more difficult to clearly understand the effect of external components affecting the system such as noise or disturbances, as well as system parameters that change through time. On the contrary, small instances of open-loop control would result in rapidly growing errors, which reveals the causes that lead to them and clarifies the subsequent correcting actions.
A list of the main contributions of this article, which revolve around the aforementioned ideas, is provided in the interest of clarity: Adaptive IC is introduced as a framework that provides an acceptable steady-state response through a control signal that also generates enough excitation to detect system changes. This ability can be understood in terms of the stability-plasticity dilemma, 31 which raises the question of how a control methodology can be designed with the goal of remaining plastic in the presence of changes or uncertainty, while maintaining stability performance levels that have been achieved in part from past experience. Special emphasis is placed on the threshold parameter q. In the adaptation context, this parameter not only determines when the events are generated, but it also sets the rate of adaptation by regulating how much error is allowed in the specified outputs, that is, how much the system can be excited. In practice, this becomes relevant since a closed-loop system could potentially be designed with this principle in mind: use a small threshold when precision control is needed, uncertainty is low, and where the main goal is to reduce variability; conversely, use large thresholds if the goal is to detect parametric changes in the system and redefine the control parameters to compensate accordingly.
A natural limitation of the proposed adaptive intermittent controller is that if due to design constraints, the outputs of a particular closed-loop process must have very small steady-state error levels, then the ability of using the threshold to regulate the adaptation rate could potentially be limited.

Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors would like to thank the financial support provided by the Consejo Nacional de Ciencia y Tecnologıá (CONACyT, Mexico), the Secretarıá de Educacio´n Pu´blica (SEP, Mexico), and the EPSRC grant Closed-loop Data Science (EP/R018634/ 1). P. J. Gawthrop was supported by the University of Melbourne Faculty of Engineering and Information Technology through a professorial fellowship.

ORCID iD
J Alberto Á lvarez-Martı´n https://orcid.org/0000-0002-4494-8405 1 2(n + l) , for i = 1, . . . , 2n 3. Collect the sigma points 4. Prediction transformation: propagate the sigma points x through the system model, calculate the a-priori-predicted state meanx À k and the a-priori state covariance matrix P À k x À k = f(x, u kÀ1 ) ð33Þ 5. Observation transformation: propagate the a-priori sigma points x À k through the observation model, calculate the predicted output meanŷ À k , its covariance matrix P yy k , and the cross-covariance matrix P xy k c À k = h(x À k , u kÀ1 ) ð36Þ 6. Improve the estimates with measurements: calculate the Kalman gain K k . Improve the a-priori state estimates with measurements y k to generate the a-posteriori estimatesx + k . Then calculate the a-posteriori error covariance matrix P + k to be used in the following iteration K k = P xy k P yy k À Á À1 ð40Þ The a ukf parameter defines the spread of the sigma points. A small value of it means that the selected points are closer to each other. The b ukf parameter conveys information of the a-priori distribution, and a value of b ukf = 2 is considered optimal for Gaussian distributions. The k ukf parameter was defined as 0.