pyvrft: A Python package for the Virtual Reference Feedback Tuning, a direct data-driven control method

In this paper, the pyvrft , a Python package for the data-driven control method known as Virtual Reference Feedback Tuning (VRFT), is presented. Virtual Reference Feedback Tuning is a control design technique that does not use a mathematical model from the process to be controlled. Instead, it uses input and output data from an experiment to compute the controller’s parameters, aiming to minimize an H 2 Model Reference criterion. The package implements an unbiased estimate of the controller for MIMO (Multiple-Input Multiple-Output) processes using both least-squares and instrumental variable techniques. The package also provides accessory functions to import data and to perform MIMO systems simulations, together with some examples


Motivation and significance
Data-driven control methods emerged in the literature on the early 40's, with the work of Ziegler and Nichols [1], which provided practical formulas, based on simple experiments, to tune PID controllers. However, most of the data-driven control methods appeared and attained more visibility after the 90's, and they still are being researched and developed on the control systems community until the present day. The main objective of these techniques is to tune a predefined and fixed order controller for a dynamic process, using batches of input and output data, without the necessity of the process' mathematical model [2]. Therefore, since the task to obtain a good and reliable process' model can be very expensive and time-consuming, data-driven * Corresponding author. control became a very attractive matter for a wide range of practical and industrial applications [3].
It is usual, among data-driven control literature, to classify the methods on two distinct groups: the group of iterative methods, which use several experiments to update the controller's parameters iteratively, and the group of direct methods, that require only one or two batches of data to tune the controller. The most popular iterative methods are the Iterative Feedback Tuning (IFT) [4], the Frequency Domain Tuning (FDT) [5] and the Iterative Correlation-based Tuning (ICbT) [6] and the most popular direct methods are the Virtual Reference Feedback Tuning (VRFT) [7], the Non-iterative Correlation-based Tuning (CbT) [8] and the Optimal Controller Identification (OCI) [9]. Within the group of direct methods, the VRFT is the most researched and disseminated one, and it possesses several extensions, applications, and analysis around its properties on the literature. So, to show this high popularity and the significance of the method, some theoretical contributions and some recent papers of VRFT applications are described in the sequence. One of the first extensions for the VRFT was proposed on [10], which introduced the possibility to design two degree of freedom controllers. Another relevant contribution to the method was its extension for multivariable (or MIMO) control systems, demonstrated on [11]. The MIMO scenario was also investigated on [12] and on [13], where the latter introduced an unbiased formulation. The VRFT approach for nonlinear control systems was presented on [14], and on [15] the method was applied to design adaptive PID controllers. As more recent contributions, the work [16] addressed the problem to deal with non-minimum phase plants, and on [17] a solution with asymptotically guaranteed stability was formulated. Also, the work [18] introduced the use of regularization to enhance the method's statistical properties, and the paper [19] formulated a new setup of the VRFT to perform load disturbance rejection.
Concerning the applications of the VRFT, there is also an extensive literature on the subject. For example, on [20], the VRFT was employed to a classic benchmark problem, where it was used to tune high order controllers for an active suspension system. On [21], the method was used to design feedback controllers for knee joint movement using functional electrical stimulation. The VRFT was also used to tune multivariable controllers in a simulation of a wastewater treatment plant on [22], to design the active braking control of vehicles on [23], to control a nonminimum phase level plant on [24], to control the attitude of a quadcopter on [25], to control MIMO tank systems on [26] and on [27], and to control cavity tuners in particle accelerators on [28].
Accordingly, given the importance of data-driven control in the literature and on practical and industrial applications, the main idea of this work is to introduce a free and open-source package, developed in Python, that implements the unbiased VRFT method for MIMO processes [13], which is, as stated before, the most popular direct data-driven method. It is important to emphasize that there is also a Matlab toolbox for the VRFT, described on [29]. Both the Matlab toolbox and pyvrft implement the standard VRFT, with the option to use the least-squares or the instrumental variable to tune the controller. Besides, there are some differences between both software. The Matlab toolbox gives additional options, as to tune nonlinear and two degree of freedom controllers. Those options are not available on pyvrft. On the other hand, pyvrft implements the unbiased MIMO version of the VRFT, while the Matlab Toolbox was developed only for the case of SISO (Single-Input Single-Output) systems.

Theoretical background
The core idea of the unbiased MIMO VRFT is to tune a controller for a linear time-invariant (LTI) discrete-time process, which can be represented by the following equation: where q is the forward-shift operator, G(q) is an n × n rational transfer function matrix, u(t) and y(t) represent, respectively, the control input and output of the process, both described by a ndimension column vector. The noise vector v(t) can be written as with w(t) being a white noise n-dimension column vector, where each of its elements has variance denoted by σ 2 w i , i = 1, . . . , n and H(q) is an n × n stable transfer function matrix.
The controller to be tuned is also an n×n linear time-invariant system, which belongs to a predefined (user-specified) class of transfer function matrices. The controller is parameterized by a parameter vector ρ ∈ R p so that the control action u(t) can be written as where r(t) is an n-dimensional column vector that represents the reference signal. The MIMO structure of the controller is given by where ρ = [ρ 11 ρ 12 . . . ρ n1 . . . ρ nn ] T and it is assumed that each subcontroller has a linear parametrization, i.e. they can be written as withC ij (q) being an m-dimension column vector of fixed causal rational functions. Each subcontroller can have a different structure, provided that they are linear in the parameters. Finally, the equations that describe the system (1)-(3) under closed-loop control, are where the dependence of the output on the controller's parameters was made explicit.
With the VRFT method, the objective is to tune the parameter vector ρ to achieve a desired closed-loop performance for y(t, ρ), which is specified through a transfer function matrix, denoted by T d (q) and also known as the reference model. The reference model defines the relationship between the reference signal r(t) and the desired output of the closed-loop system, denoted by y d (t): Then, the VRFT designs the controller in a Model Reference (MR) framework, where the goal is to minimize the L 2 norm of the difference between y(t, ρ) and y d (t). This problem can be expressed The optimization problem described on (10)-(11) is non-convex, and from (8), observe that T (q, ρ) depends explicitly on the process' transfer function G(q) that is usually unknown by the user.
On the other hand, the VRFT proposes a convex optimization problem that does not depend on the process' model but uses only input and output data to tune ρ. It assumes that a batch of input/output data is collected from the process: and, then, the controller's parameters are computed by solving the following optimization problem: The filter L(q) is inserted on the method as an extra degree of freedom that can be used to improve the properties ofρ VR under non-ideal conditions. The choice of the filter is discussed with more depth and formality at [7,13], where the authors demonstrate that an approximation for the optimal solution can be achieved by for both SISO and MIMO cases, with Φ r (e jω ) and Φ u (e jω ) being the power spectrum of r(t) and u(t) respectively. The convex optimization problem introduced on (13)-(14) has a closed-form solution, given by the least-squares equation. Moreover, under ideal conditions, both J MR (ρ) and J VR (ρ) have the same global minimum, such that the unbiased MIMO VRFT can be successfully used to solve the MR problem [13]. Under non-ideal conditions, when the complexity of the controller is restricted, and the desired performance cannot be achieved, the VRFT proposes the use of the filter L(q) to shape the method's criterion, aiming to approximate the minimum of both cost functions. Also, when the signals are corrupted by noise, an instrumental variable can be used to provide unbiased estimates for ρ. In this case, a second batch of data that is uncorrelated with the first one must be collected, either by running a second experiment or by simulating the process with an identified model [7,13].

Practical aspects and preprocessing the data
When the input/output data are collected from a real process, which possibly is nonlinear, presents high frequency measurement noises, drifts or offsets, missing data, periodic disturbances, and other undesirable effects corrupting them, they are not likely to be directly used on system identification [30] or on datadriven control methods, such as the VRFT. Hence, to improve the efficiency of pyvrft on real applications, a preprocessing step on the data is highly recommended and it can done using basic operations that are also known and employed by the system identification community and described on its classic books [30,31]: • Remove sample means [30]: if the data presents offsets or drifts, the user can subtract the mean value of the samples of y(t) and u(t), assuming that the system is around an equilibrium point and reducing the impact of this undesirable effect; • Prefiltering the data [30,31]: prefiltering y(t) and u(t) with the same filter will not change the shape of the VRFT criterion. So, the user can prefilter the signals with a low-pass filter (to reduce high frequency noise/disturbances), a highpass filter (to reduce low frequency noise/disturbances, or even drifts and offsets), or a band-stop filter, like a notch filter (to reduce the effects of a disturbance with a specific frequency on the data).
It is important to reiterate that those are just basic and simple recommendations to preprocess the data in order to reduce the most typical undesirable effects that appears on the data. However, the user can also apply more sophisticated and complex preprocessing procedures, as those discussed with more depth at [30,31].

How to use the software
The first step to use the software is to install the package, which can be made via pip, running the command: pip install pyvrft. The pip command also installs the package's prerequisites if they are not present in the Python environment: numpy for numerical computations, scipy for signal processing and matplotlib to create graphics.
To run the main function of pyvrft, namely vrft.design(), to tune the parameters of an MIMO LTI controller, it is necessary to collect input and output data from the process (in an openloop or closed-loop experiment) and organize them as a matrix (numpy.ndarray) with dimension (N, n): Besides, if the signals are corrupted by noise, the user may collect output data from a second experiment to use the instrumental variable technique: After collecting and organizing the data, the user must specify the desired closed-loop performance through T d (q), the controller structure that will be tuned, and the VRFT filter L(q). All these quantities are transfer matrices (or MIMO transfer functions) and, since there is no structure of such type on the traditional Python packages for signal and numerical processing (numpy and scipy), in pyvrft it was decided to organize them as nested Python lists, as described below • T d (q): nested Python list with two levels. The first one represents a line of the transfer matrix T d (q) and the second one represents a column of T d (q). Each element of the list has a specific variable type, which is the signal.ltisys. TransferFunctionDiscrete (the scipy variable type for discretetime transfer functions); • C (q, ρ): nested Python list with three levels. The first level represents a line of the transfer matrix C (q, ρ) and the second represents a column of C (q, ρ). Yet, the third level represents each element of the m-vectorC ij (q). Each element of this list is also a signal.ltisys.TransferFunctionDiscrete variable; • L(q): has the same structure of T d (q), which is a nested Python list of two levels and each element is a signal.ltisys. TransferFunctionDiscrete.
The following code exhibits an example for the design of an SISO controller: To assist the user with the definitions of these quantities on Python and how to use them on the software, there are a few examples of controllers design within the package. Also, a detailed description of each function is available and can be accessed with the help() command on the Python environment.

Software description
The pyvrft is a Python package that implements the unbiased MIMO VRFT method, to tune an MIMO (or SISO) LTI controller, with the possibility to use the least-squares or the instrumental variable to solve the problem. To familiarize the user with the software's architecture and its particularities, they are described on the following subsections.

Software architecture
The package contains 3 main modules, namely control.py, csvfunc.py and invfunc.py. Each one of these modules has its purpose inside the package and its functions that are described in the sequence.

control.py
This module implements the main algorithms of the package, that are used to compute the controller's parameters. It possesses three functions: • design(): this is the main function of the pyvrft package, the one that implements the Unbiased MIMO VRFT method. It calls several functions from different modules of the package as filter(), colfilter(), mtf2ss() and stbinv(); • filter(): implements a multivariable filtering operation, which is essential for the main function of the package. This function can also be used for simulations and validation of the controllers; • colfilter(): implements a column filtering operation, which filters every column of a matrix of signals with the same filter.

csvfunc.py
This module is used to import csv data into Python. It possesses just one function, that is • datafromcsv(): provide the option to read the data from a csv file. The csv file must have a strict organization of its columns, such as [y 1 , y 2 , . . . , y n , u 1 , u 2 , . . . , u n ].

invfunc.py
This module contains the functions that are related to the algorithm that performs a stable inversion of a linear system, which is used on the VRFT to calculate the virtual error, i.e., the signalē(t) = (Td(q) −1 −1)y(t). The algorithm was firstly presented on [32], and it was implemented on pyvrft through the following functions • invredc(): implements one step of the inversion algorithm, which is the system reduction step. This function is called at each iteration of the stable inversion algorithm; • stbinv(): implements the inversion of the system, calling invredc() to perform the system reduction step. It also calculates the conditions for the stable algorithm to be successful. Otherwise, it produces a warning to the user; • mtf2ss(): transforms a MIMO transfer function on a statespace representation. The algorithm that was employed is a simple one, that does not intend to produce a minimal realization. This procedure is important because the VRFT method was implemented with a transfer function notation (as it is usually done in the literature of this theme). Although, the stable inversion algorithm is implemented using a state-space representation, as presented on [32].

Software functionalities
The main functionality of the pyvrft package is to tune an MIMO LTI controller for a dynamic process with the unbiased MIMO VRFT method, using batches of input and output data. This can be achieved with the function design(), described above.
Besides, as some secondary functionalities, it is possible to use other functions of the package for different kinds of applications.
For example, filter() can be used to simulate MIMO LTI systems, or equivalently, to filter a vector of signals with an MIMO filter. The stbinv() function can be applied to calculate the  inverse of an MIMO system, i.e, the signal u(t) = G(q) −1 y(t), which can be useful on some circumstances, and mtf2ss() can be used to transform a MIMO transfer function representation (done with the nested Python list structure) to a state-space model.

Illustrative examples
To motivate the application of pyvrft on real control systems, this section demonstrates an experimental example, where the main objective is to tune a full PI controller for an MIMO level control process. This example, along with two others, are also provided on the package. The process that is considered in this example belongs to a pilot plant, which was fully described at [26] and used to compare different control strategies for multivariable processes. The plant possesses a Two-Input Two-Output (TITO) process, where the outputs y 1 (t) and y 2 (t) are the levels of water on two distinct tanks and the inputs u 1 (t) and u 2 (t) are the percentage opening of two globe valves that are used to regulate the water flow of the system.
To collect the data from the process, two distinct pseudorandom binary sequences (PRBS) were applied as input signals on the process in an open-loop experiment. Fig. 1 exhibits the input signals, and Fig. 2 exhibits the output signals obtained. These data were stored on a csv file, that was read with the pyvrft datafromcsv() function.
and the controller structure to be tuned was a full PI: The filter of the VRFT was chosen as L(q) = T d (q)(I − T d (q)), and the pyvrft was used to estimate the controller's parameters. The controller designed with the software was which is practically the same obtained on [13,26]. This controller was implemented on the pilot plant, and the achieved closedloop performance is exhibited on Figs. 3 and 4. Fig. 3 shows a comparison between the output signals, the reference signals, and the desired outputs on the closed-loop experiment and Fig. 4 exhibits the control signals in this experiment. Notice that the process' outputs, obtained on the closed-loop experiment, were very close to the desired ones. Also, the control system almost achieved a perfect decoupling between the loops, where one reference/output signal does not disturb the other, which is a significant result for a multivariable control system.

Impact
The pyvrft package intends to impact the industrial users, researchers, and developers of data-driven control methods, providing for these groups, a free and open-source software that allows them to use the VRFT method on practical situations and to develop and test further theoretical contributions. As far as the authors know, this is the first implementation of the unbiased MIMO VRFT that is free and open for the community, as well as peer-reviewed. From now on, new users of VRFT do not need to implement the algorithms from zero and by themselves. Moreover, the package was developed in Python, which is the largest growing programming language used by scientific communities, and that provides free and open-source algorithms that can be verified, corrected, and expanded by other users.

Conclusions
The pyvrft is a Python package for designing feedback controllers using the Virtual Reference Feedback Tuning method. The package computes unbiased estimates for MIMO and SISO controllers. Both classical least-squares and instrumental variables are available to be used. The package simplifies the task of calculating feedback controllers since it implements a data-driven solution that does not depend on a mathematical process' model. The architecture of the package makes it simple to use, and it is a timesaving tool for control designers. The pyvrft package is open-source, and it is distributed under an MIT license.

Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.