An Improved Toeplitz Measurement Matrix for Compressive Sensing

Compressive sensing (CS) takes advantage of the signal's sparseness in some domain, allowing the entire signal to be efficiently acquired and reconstructed from relatively few measurements. A proper measurement matrix for compressive sensing is significance in above processions. In most compressive sensing frameworks, random measurement matrix is employed. However, the random measurement matrix is hard to implement by hardware. So the randomness of the measurement matrix leads to the poor performance of signal reconstruction. In this paper, Toeplitz matrix is employed and optimized as a deterministic measurement matrix. A hardware platform for signal efficient acquisition and reconstruction is built by field programmable gate arrays (FPGA). Experimental results demonstrate the proposed approach, compare with the existing state-of-the-art method, and have the highest technical feasibility, lowest computational complexity, and least amount of time consumption in the same reconstruction quality.


Introduction
Compressive sensing (sampling), as a novel sampling technique, which under certain conditions aims to project a highdimensional random signal onto a lower-dimensional space with high probability compared to conventional Shannon's sampling rate, is put forward first by Donoho et al. in 2006 [1, 2]. Measurement matrix as the key part of CS research which guarantees the excellent projection effect has recently gained more and more attentions in various fields of applied mathematics, electrical engineering, and remote sensing.
Random measurement matrix is not just a simple random variable. It should satisfy restricted isometric property (RIP) condition. The most classic random measurement matrix is Bernoulli and Gaussian matrix [3], as well as Sub-Gaussian Random Projection [4] and over-sparse random projection [5]. According to RIP rules, partial random measurement matrix is only part of the random matrix, which has some different ways: Sub-Gaussian Random Projection [4], oversparse random projection [5], partial Fourier matrix [6], partial Hadamard matrix [7], partial orthogonal matrix [8], and so forth. It is clear that randomness plays a key role in the design of the measurement matrix. Thus random measurement matrix is universal in the sense, regardless of whether it meets RIP with high probability in most situations.
When the input signal is -sparse, an × i.i.d. Gaussian random measurement matrix requires generation of ( ) random elements and implementation of ( ) independent operations. However is large enough to delay data acquisition and reconstruction time, which is rarely realized in VLSI digital hardware, and led to particularly trouble for large-scale applications.
While the deterministic measurement matrix has specific matrix value and fixed position, including Toeplitz and circulant matrices [9][10][11][12], sparse binary matrices [13], structurally random matrices [14], Chirp sensing codes [15], random convolution matrices [16], finite fields [17], and second order Reed-Muller [18], the most classic is Toeplitz matrix, which uses fast Fourier transforms (FFT) and, as a consequence, requires only ( ) independent variables and ( log 2 ( )) operations. More importantly, there are matrix-vector multiplication routines, which can be exploited in recovery algorithms. Furthermore, they arise naturally in certain applications such as identifying a linear time invariant system.
However, deterministic measurement matrix's unchanged element and fixed position result in three main limitations [19]. The first point is that incoherence criterion is hardly met. The second disadvantage is that the reconstruction precision of deterministic measurement matrix is obviously inadequate. Matrix building time being unbearable is the third failing.
To overcome those limitations, our intuitive idea is discrete chaotic system functions to generate a series of random elements in Toeplitz matrix. Then, a circulant/blockdiagonal splitting structure is applied to speed up convergence rate. Based on these improvements, an improved Toeplitz-structured matrix is proposed to both improve reconstruction precision and reduce cost time.
Key contributions of this work can be elaborated as follows.
(1) Discrete chaotic system function is proposed to generate a series of pseudo-random numbers. Based on those elements, Toeplitz-structured chaotic sensing matrix (TSCSM) is produced to guarantee the incoherence criterion.
(2) To reduce the building time of TSCSM, circulant/block-diagonal splitting structure is attached on TSCSM. Although about one-third of matrix values are eliminated, ITMM is proved to satisfy Johnson-Lindenstrauss lemma (JL). We can take advantage of the equivalent relation between theorem JL and restricted isometric property (RIP); ITMM achieves the goal of satisfying RIP.
The rest of this paper is organized as follows. The mathematical model and analysis are described in Section 2. Based on the previous analysis, an improved Toeplitz-structured matrix is proposed in Section 3. Several comparative experiments are conducted for the conventional algorithms and the proposed method in Section 4. Discussion and conclusions are summarized in Section 5.

Mathematical Model
From the viewpoint of mathematics, CS is to consider the problem with far incomplete measurements as follows: If ∈ R ×1 is a -sparse one-dimensional signal and Ψ is the orthogonal basis being known, then finding a suitable Φ ∈ R × ≪ , which is the so-called CS measurement matrix, to obtain the goal of minimizing × 1 linear measurements , as Figure 1 shows [1].
The ill-posed problem appears from ≪ and unknown elements of measurement matrix Φ. So finding a proper measurement matrix Φ is quite a challenging problem. To solve the problem, Candès et al. [2] have proposed that if is -sparse and the locations of the nonzero coefficients in are known, then for any vector V sharing the same nonzero entries as and some > 0, be the smallest constant satisfying following inequality, The sufficient condition (2) is called restricted isometric property (RIP) [1].
That is to say, once orthogonal basis Ψ is known, CS measurement matrix Φ must be irrelevant to Ψ. Candès et al. [2] have demonstrated that an × i.i.d. Gaussian matrix Θ = ΦΨ = Φ can meet RIP with high probability when Ψ = and ≥ log( / ), where is a small constant. Specifically, assuming is independent and identically distributed (i.i.d.) random sequence from Gaussian probability density function with mean zero and variance 1/ , then the × partial Toeplitz measurement matrix Φ has the following form: ] . (3) Using fast Fourier transforms (FFT), Φ requires only ( ) independent variables and ( log 2 ( )) operations instead of ( ) random elements and ( ) independent operations from Gaussian measurement matrix. Consequently, it is often easily (or even naturally) realized in VLSI hardware.
Toeplitz matrix verifying RIP has two main limitations. One is a large amount of calculating the spectral norm of  submatrices built by selecting arbitrary columns from dimensions. The other is difficult to validate these submatrices one by one whether satisfies RIP bounded condition.
Further research is being carried out to be closely linked to the famous Johnson-Lindenstrauss lemma [20,21], which is extensively utilized in numerical linear algebra, machine learning, or other areas requiring dimension reduction.
Theorem (J-L Embedding with Beta-like Matrix): given a Beta-like matrix Φ = R × , there exists a constant > 0, such that, for any vector ∈ R , the following inequality Here ( ) is a constant related to . By the Egodicity property of the chaotic system, the theorem has been verified in [22]; different constant ( ) is equal to with Gaussian matrix.

The Proposed Approach
The challenge of optimizing Toeplitz measurement matrix is how to both improve its reconstructed precision and shorten its processing time. Based on above mathematical analysis, an improved Toeplitz-structured matrix is presented in this paper, as shown in Figure 2. In the first step, pseudo-random chaotic elements in a Toeplitz measurement matrix are selected to replace random elements. Next, circulant/blockdiagonal splitting structure is employed as discussed in Section 3.2. Finally, an improved Toeplitz measurement matrix (ITMM) is shown in Figure 2 and proved that this matrix also retains the RIP property with overwhelming probability. See more details in Section 3.3.

Toeplitz-Structured Chaotic Sensing Matrix.
In order to achieve the purpose of random Toeplitz matrix, pseudorandom chaotic sequences have been entirely replaced by random ones because of better randomness. Logistic map  [22,23], as the simplest dynamic systems evidencing chaotic behavior, is described as follows: where ∈ (3.5699, 4] ⊂ is the discrete state. When parameter is 4, the sequence ( ) satisfies beta distribution with = 0.5 and = 0.5; the next probability density function ( ; 0.5, 0.5) = (1/ )( − 2 ) −1/2 has been used for simulation.
Here (7) is called the Beta-like matrix.

4
International Journal of Distributed Sensor Networks According to (7), set one initial condition ∈ (0) ∈ R, and generate a sequence ∈ R in the chaotic system. Then Toeplitz-structured matrix Φ = R × is constructed in the following form: ) . (8) Here √8/ is for normalization and Φ is called Toeplitzstructured chaotic sensing matrix (TSCSM), which meets J-L theorem.

Circulant/Block-Diagonal Splitting
Structure. If independent random elements are cut one third from either bevel edge band or column band in the measurement matrix, the overall amount of calculation can be saved more than onethird.
In [24,25], Φ satisfies J-L theorem, by the Egodicity property of the chaotic system; J-L condition can replace RIP condition. So Φ satisfies RIP property with overwhelming probability. According to the linearity property of Toeplitz measurement matrix, Φ 1 satisfies J-L property with overwhelming probability. So Φ 1 also satisfies RIP property with overwhelming probability.
A fused optimization method (ITMM + StOMP) is now presented. These main steps are as follows. Firstly, by selecting some particular elements in Toeplitz measurement matrix, discrete chaotic system function is employed to generate a series of random sequence. Secondly updating Toeplitz-structured matrix by reducing variable numbers, an improved Toeplitz-structured matrix (ITMM) is presented to find the sparse approximation solution and faster converge to this value in compressive sensing. Finally, under the samesparse degree, reconstructed 1D signal or 2D image is built up by stagewise orthogonal matching pursuit (StOMP) [32]. As is shown in Algorithm 1.

Experiments and Result Analysis
In this section, all existing state-of-the-art measurement matrix (Gaussian, partial Fourier, Hadamard, circulant, and Toeplitz) size is turned into 64 × 256 pixels, signal sparsity method is DCT, and reconstruction algorithm is StOMP. We first carried out 1D random signal experiments to investigate the efficiency of the proposed approach. Subsequently, we report the results for the classical 2D test images. Finally, prototype onboard recovery system is stimulated by the proposed approach. All these algorithms are implemented using Matlab 7.10 on a laptop with Athlon, Processor 1.60 G HZ, 1 GB RAM, and Windows XP operation platform.

The 1D Random Signal Experiment.
If the 1D signal amplitude becomes 0.3 sin(2 1 ) + 0.6 sin(2 2 ) + 0.1 sin(2 3 ) + 0.9 sin(2 4 ), where size varies from 50 Hz to 400 Hz in intervals of 50, sampling period is 1/800, and sampling signal number is 256, all testify experiment results of recovery 1D signals by StOMP and their amplification part are shown in Figure 3. Figure 3 features the recovery performance for 1D random signal by different measurement matrix. It is clear from Figure 3 that recovery curve from Gaussian (black line), partial Fourier (blue line), Hadamard (red line), circulant (yellow line), and Toeplitz (green point line) has a great deviation from the original curve (red block line). All recovery curves from the proposed approach (black point line) have both black point fallen into red block from the original curve and occupied the original curve accurately. Furthermore, the objective evaluation, including signal-to-noise ratio (SNR) and cost time (Time), is shown in Table 1. Table 1 shows detailed recovery performances and cost time for all of the six measurement matrixes. In Table 1, we show the recovery performance for ITMM is better than that obtained by the other five measurement matrixes.
International Journal of Distributed Sensor Networks 5 Input: (1) Number of iterations to perform; (2) Threshold value ; (3) ∈ R and the measurement matrix ∈ R ;

Output:
∈ R Such that is -sparse and = Φ .   In the experiment, we can see that the performance of Hadamard measurement matrix is inferior to the other measurement matrixes for all five classical test images. The difference between the performances of circulant and Toeplitz measurement matrix is not salient. The performance of Gaussian measurement matrix is surprisingly close to that of our proposed method. It is far from enough to only rely on the experiences of the individual but not on systematic, objective, and accurate method. The objective evaluation is necessary to correctly estimate these various measurement matrixes, including mean squared error (MSE) and cost time (CT). See more details as Table 2 shows.
As shown in Table 2, the objective evaluation criteria (MSE and CT) are achieved by several state-of-the-art methods and our approach with Lena, Cameraman, Couple, Face, and Brain data sets. From the bold numbers, we can draw the conclusion that our approach method outperforms the existing state-of-the-art method.

Prototype Recovery System Experiment.
A prototype onboard recovery, which is only fabricated by Toeplitz and its improvement, is one of the applications from CS Technology. Figure 9 shows the prototype recovery engine (RE) board and final recovery result. It is composed of 8 * 16 standalone REs each with the ability to recover spectral vectors in parallel. These are autonomous devices programmed to perform recovery in continuous mode, subset by subset. An RE is composed of a field programmable gate arrays (FPGAs) chip. It took a half second for the prototype to recover, while a powerful computer platform would take a few minutes to reconstruct. The throughput can be increased by using more CEs in parallel. The experimental results showed that MSE and fidelity achieved by the prototype recovery are similar to those obtained by software simulation.

Conclusions
In summary, an optimization method is proposed on Toeplitz-structured chaotic sensing matrix with circulant/block-diagonal splitting structure. It has retained the RIP property with overwhelming probability and decreased time consumption to the largest extent. We demonstrate that the proposed method has the most significant recovery performance in 1D and 2D input signal sources. And prototype recovery system is built by FPGA, whose fidelity is similar to those obtained by software simulation. Therefore, we believe that this framework gives an unconventional way to solve the optimization problem of deterministic measurement matrix and will vastly and widely benefit large-scale applications and is realized in VLSI hardware.