Identifying Odd/Even-Order Binary Kernel Slices for a Nonlinear System Using Inverse Repeat m-Sequences

The study of various living complex systems by system identification method is important, and the identification of the problem is even more challenging when dealing with a dynamic nonlinear system of discrete time. A well-established model based on kernel functions for input of the maximum length sequence (m-sequence) can be used to estimate nonlinear binary kernel slices using cross-correlation method. In this study, we examine the relevant mathematical properties of kernel slices, particularly their shift-and-product property and overlap distortion problem caused by the irregular shifting of the estimated kernel slices in the cross-correlation function between the input m-sequence and the system output. We then derive the properties of the inverse repeat (IR) m-sequence and propose a method of using IR m-sequence as an input to separately estimate odd- and even-order kernel slices to reduce the chance of kernel-slice overlapping. An instance of third-order Wiener nonlinear model is simulated to justify the proposed method.


Introduction
Living systems usually exhibit complex and nonlinear behaviors [1][2][3], which can be characterized by a mathematical model carefully tuned to represent the relationship between the input and output data. A linear system is capable of determining the input and output relationships through an impulse response function; however, for a nonlinear system, a higher order transfer function has to be used for this purpose. A nonlinear system can be typically modeled by Volterra or an equivalent Wiener series expansion, in which the Volterra or Wiener kernels to be estimated can fully define the system characteristics [4][5][6].
The kernel estimation for such nonlinear system usually requires the input signal to be a long Gaussian white noise to completely activate the underlying system. Under such conditions, Lee and Schetzen proposed a convenient cross-correlation method widely used to estimate the kernel functions [7,8].
In several circumstances, particularly for a variety of living biosystems, input signals are constrained as a series of impulse trains instead of continuous signals, such as the Gaussian white noise [9,10]. For instance, the auditory system is usually studied by stimulating the ear with a series of click sounds to activate the corresponding neurons in the cochlea and neural pathway to evaluate the hearing integrity [11,12]. A well-studied impulse train for the input is a pseudorandom binary sequence called maximum length sequence (short for m-sequence), which has an important role in nonlinear system identification. The correlation property of an msequence is analogous to a Gaussian white noise such that to model the system by borrowing the idea of the crosscorrelation method for Gaussian white noise input is possible. Hence, the binary kernels are defined using cross-correlation method for m-sequence inputs [3,[13][14][15][16][17][18].
Using the m-sequence approach, Sutter studied the binary kernel for identifying multifocal retinosystem through electroretinography and explained the visual function using the binary kernels [15]. Shi and Hecox transferred the msequence into an m-impulse sequence in a study on the nonlinear properties of the auditory system by measuring the electrical response from the scalp [16]. In a study on the dynamic characteristics of the primate retinal ganglion cell, Benardete and Victor developed a hybrid m-sequence allowing the summation of multiple m-sequences as input to estimate the main diagnostic kernel slice [17].
However, the binary kernel slices-derived by making use of the shift-and-product property of the m-sequence-are all laid in the first-order cross-correlation function between the m-sequence input and the system response, that is, the observed output. The specific location of any kernel slice in the cross-correlation function is determined through a complex shift function that cannot be explicitly determined. If the kernel slices are improperly arranged such that overlaps among neighboring slices occur, then the kernel estimation is inevitably distorted. A straightforward approach to solve this problem is to multiply the length of the input msequence, which is unfavorable for living systems with more or less time-varying property. Another approach to alleviate the overlap issue is to sparsify the impulse train of the msequence at risk of suffering the underestimation caused by the reduced number of available kernel slices [18].
In this study, we addressed the overlap problem through a new strategy using an inverse-repeat (IR) m-sequence. We will drive the estimation equations for the binary kernel slices corresponding to the IR m-sequence, through which the oddand even-order kernel slices can be separately estimated and thus reduce the chance of slice overlapping. Last, a thirdorder nonlinear system is simulated to demonstrate the process of the proposed method.

The Properties of an m-Sequence. An m-sequence [ ]
consisting of digit −1 and +1 that pseudorandomly occurred can be generated through the output of a linear circular shift register. The structure of which is determined through a primitive polynomial of degree , which is also the degree of the periodical m-sequence [19,20]. And the period or length of the m-sequence is = 2 − 1. An m-sequence is called a balanced sequence because the number of −1 is 2 −1 , which is just one more than the number of +1, that is, 2 −1 − 1. Two crucial properties of the m-sequence used in the present study are as follows.
(i) Shift-and-product property where = ( 1 , . . . , −1 ) is referred to as shift function representing the circular shift lag of the m-sequence [ ]. The exact value of a shift function depends on the shifting lags of the m-sequences to be multiplied. This property indicates that the product of the m-sequences with different circular shifts is also the same m-sequence circular shifting to a lag determined by a shift function that is unknown a priori. For 1 = ⋅ ⋅ ⋅ = −1 = 0, that is, the product of the same m-sequences, it yields where {1} denotes an all-one sequence-all members of the sequence are 1s. Equation (2) provides an exception for the shift-andproduct property that an all-one sequence instead of an msequence is produced under a certain condition when multiplying the same m-sequences. When more than two different m-sequences are multiplied, we have another exception for the shift-and-product property: Equation (3) implies that when dealing with higher-order kernels ( ≥ 3), the m-sequence should be selected with caution in case invalid results occur for the kernel estimation.
(ii) The autocorrelation function [ ] is a periodical real value function with the same minimal period of ; that is, In analogy to the Gaussian white noise method, this autocorrelation property of (4) is important to account for the derivation of the m-sequence in identifying a nonlinear system.

From Volterra Kernels to Binary Kernels. The output
[ ] of a general th-order nonlinear dynamic system in response to the input [ ] can be modeled by a Volterra series expansion [5,21], where H is called the th-order Volterra operator which is defined as where represents the memory length of the dynamic system, and ℎ represents the th-order Volterra kernel.
To estimate the Volterra kernel for Gaussian white noise, input is not theoretically feasible for the difficulty of nonorthogonality. Instead, it is preferred to estimate the Wiener kernel after the Gram-Schmidt orthogonal process on the Volterra series expansion [21]. This method can be extended to deal with m-sequence input yielding the so-called binary kernel estimation [13,16]. A th-order binary kernel is given by Computational and Mathematical Methods in Medicine Figure 1: Illustration of overlapping condition for two arbitrary kernel slices. A piece of cross-correlation function contains th-and thorder kernel slices, that is, and , respectively. Corresponding shift functions and designate the beginnings of kernel slices. The duration of is dependent on memory length and sum of shift lags ( 1 + ⋅ ⋅ ⋅ + −1 ). According to (14), nonoverlap and overlap cases are presented in (a) and (b), respectively.
where the th-order cross-correlation of input [ ] and output [ ] is Let = 1 and = +1 − , and then (7) and (8) become [ , + 1 . . . , According to the shift-and-product property, (10) becomes which transfers the multivariable correlation function [ , + 1 , . . . , + 1 + ⋅ ⋅ ⋅ + −1 ] into a single variable crosscorrelation function [ + ]. Substituting (11) to (9) yields Given 1 , 1 , . . . , −1 , (12) presents a portion of the kernel function values along the diagonal and subdiagonal dimensions and is called binary kernel slice. Considering the confinement for the independent variables for , the kernel slice is probably unable to completely cover the true binary kernel along this dimension. Given the memory length similar to (6), all variables for th-order kernel slice must be in the range of the memory length ; hence, suggesting that is defined through the cross-correlation function between and +( − 1 −⋅ ⋅ ⋅− −1 ). Therefore, if the shift functions of two neighboring slices satisfy that is, the interval between an arbitrary kernel slice of order and another kernel slice of order is less than the length of the prior slice, then a slice overlap occurs. The kernel slices overlapping condition is illustrated in Figure 1.

Estimate Binary Kernel Slices Using IR m-Sequence
where [ ] = {(−1) } is a square wave-like function with alternates 1 and −1. Several useful properties of an IR msequence are given as follows [22,23].
If the minimal period of the m-sequence is odd, then [ ] = − [ + ]; that is, [ ] is also inversely repeated. The proof of property (iv) is given in Appendix A.
(v) IR m-sequence [ ] also has the following shift-andproduct property, The proof of property (v) can be found in Appendix B. In reference to Volterra series expansion for a nonlinear system, this property for an IR m-sequence implies that the higherorder Volterra operators can be separated into odd-and evenorder cases.

Odd-and Even-Order Kernel Slices for IR m-Sequences.
Based on the above properties of the IR m-sequences, we can derive the binary kernel slices in response to an IR msequence. Let [ ] be the input of the system, and then the output of the th-order nonlinear system expressed in terms of Volterra series expansion is Again, let = 1 and = +1 − , and, using IR m-sequence property (v), the Volterra operator on the input [ ] becomes Exploiting (20) for the property (v) of the IR m-sequences, we can derive that the cross-correlation function between the msequences [ ] and odd-order Volterra series term H [ ],

( = odd), is zero and that the cross-correlation function between IR m-sequences [ ] and even-order Volterra series term H [ ] ( = even) is also zero; that is,
The proof for (23) is given in Appendix C.
We can derive the th-order cross-correlation for IR msequence input [ ] with the same approach as that of getting (7)- (12) to give the expression of the th-order kernel slices as where, according to the definition of cross-correlation function, the right side member is [ + ] , = even.
Therefore, (24) becomes [ ], or the even-order Volterra kernels are also excluded from the odd-order binary kernel slices. The similar argument can be obtained for the case of even-order binary kernel slices.

Nonlinear Wiener Model Generation.
A general nonlinear dynamic system can be represented by a Wiener model consisting of two subsystems in cascade form [24]. This system consists of a dynamic linear subsystem [ ] followed by a static nonlinear subsystem [⋅], as shown in Figure 2. The output of the dynamic subsystem [⋅] is where represents the memory length of the system. For simplicity, we simulate a three-order nonlinear system expressed in the second module [⋅]. Then, the output is Therefore, the output of the whole nonlinear system is expressed as a high-order convolution of the input and the Volterra kernels as From (3), the following relationships between Volterra kernels and the parameter model hold (31) Let = 1 , 1 = 2 − 1 , and 2 = 3 − 2 , and then the binary kernel slices can be associated with the Volterra kernel for this model [12] which is (32)  In the present study, the memory length = 10, impulse response function [ ] = − /2 ⋅ sin(2 /5), and the thirdorder polynomial coefficients 0 = 0, 1 = 1.4, 2 = 5, and 3 = 10 are set.
We assume that represents the total length of all the kernel slices spread along the cross-correlation function, as illustrated in Figure 1. To avoid the overlap among these slices, at least the length of the cross-correlation function, (= 2 −1), should be larger than , indicating that the degree of the m-sequence should satisfy the condition, where ⌊(⋅)⌋ denotes the integer part of (⋅). According to (13), the shift lags are set to start with minimal values min( ) = 1, and we can obtain the total slice length for this three-order nonlinear system as = 175 for = 10, and the degree of the m-sequence should be ≥ 8 to avoid the slice overlap.

Comparison of m-and IR m-Sequence Inputs.
We select an arbitrary eight-degree m-sequence in this identification instance. Figure 3 shows the binary input m-sequence [ ], output signal [ ], and cross-correlation [ ]. The estimated kernel slices are plotted in solid trace superimposed on the cross-correlation signal (in dashed trace). The red trace segments indicate the occurrence of slice overlaps. According to the overlap condition defined in (14), a total of 10 overlaps occurred in the cross-correlation function.
The kernel slices estimated for the IR m-sequence input [ ] are shown in Figure 4. The odd-and even-order kernel slices are separated into two traces of the cross-correlation, as indicated in (27). All slices are notably separated without any overlap.
The distortion severity due to overlapping does not display in Figure 3 without the true kernels. Therefore, we extract and compare the estimated and the true kernel slices for the second-and third-order systems in Figures 5 and 6, respectively. These are two-dimensional illustration, where the kernel slices with different lags are plotted by projecting them to the main-diagonal direction of that order. Notably, the length of the kernel slices decreases with increasing values  of shift lags. The kernel slices with the length less than four data samples were excluded in Figures 5 and 6 for less practical significance.
Six second-order kernel slices estimated by the msequence (in read traces) and proposed IR m-sequence (in blue traces) methods are shown in Figure 5. The true or theoretical kernel slices are also given as a benchmark. Four kernel slices determined using the m-sequence evidently deviated from the true values; however, no such distortion was observed for IR m-sequence method.
A series of third-order kernel slices are shown in Figure 6. Similar to Figure 5, IR m-sequence method shows more consistent results with the true values. However, the thirdorder kernel slices for 1 = 4 show a slightly opposite behavior that needs to be accounted for, because no overlap occurs at these affected kernel slices. This inconsistency is only observable at a significantly small scale (see the amplitude scales in Figure 6) caused by approximating the orthogonality of the IR m-sequence and has no appreciable effects upon practical application. Alternatively, increasing the length of the m-sequence will alleviate this problem. Further analysis of this intrinsic property is beyond the scope of the present study and more details can be found in [25,26].

Conclusion
The shift-and-product property of the m-and IR m-sequence is crucial in the derivation of new properties to address the overlap problem for short-length m-sequence. It is apparently not beneficial to use a redundantly higher order m-sequence or to sacrifice the number of the estimated slices using sparse m-sequence. In this study, we alternatively propose an approach by introducing the IR m-sequence. We provide and prove several relevant properties of the IR m-sequence allowing the estimation of the binary kernel slices. By examining the special shift-and-product properties of the IR msequence derived for odd and even shifts, the odd-and evenorder kernel slices can be separately represented in the crosscorrelation functions, such that the chance of overlapping significantly decreases. Furthermore, this separation will be useful in some special applications where only odd-or evenorder kernel slices might be of significant interest.

Appendices
A. The Proof of (19) Given an IR m-sequence [ ], using (15) (20) is proved. (23) According to the shift-and-product property of IR msequence in (20), the th-order Volterra series term becomes