Introduction to the discrete Fourier series considering both mathematical and engineering aspects-A linear algebra approach

Abstract: The discrete Fourier series is a valuable tool developed and used by mathematicians and engineers alike. One of the most prominent applications is signal processing. Usually, it is important that the signals be transmitted fast, for example, when transmitting images over large distances such as between the moon and the earth or when generating images in computer tomography. In order to achieve this, appropriate algorithms are necessary. In this context, the fast Fourier transform (FFT) plays a key role which is an algorithm for calculating the discrete Fourier transform (DFT); this, in turn, is tightly connected with the discrete Fourier series. The last one itself is the discrete analog of the common (continuous-time) Fourier series and is usually learned by mathematics students from a theoretical point of view. The aim of this expository/pedagogical paper is to give an introduction to the discrete Fourier series for both mathematics and engineering students. It is intended to expand the purely mathematical view; the engineering aspect is taken into account by applying the FFT to an example from signal processing that is small enough to be used in class-room teaching and elementary enough to be understood also by mathematics students. The MATLAB program is employed to do the computations.

ABOUT THE AUTHOR Ludwig Kohaupt received the equivalent to Master's degree (Diplom-Mathematiker) in Mathematics in 1971 and the equivalent to PhD (Dr.phil.nat.) in 1973 from the University of Frankfurt/Main. From 1974 to 1979, Kohaupt was a teacher in Mathematics and Physics at a secondary school. During that time (from 1977 to 1979), he was also an auditor at the Technical University of Darmstadt in Engineering Subjects, such as Mechanics, and especially Dynamics. As of 1979, he joined the Mercedes-Benz car company in Stuttgart as a computational engineer, where he worked in areas such as Dynamics (vibration of car models), Cam Design, Gearing, and Engine Design. Then, in 1990, Dr. Kohaupt combined his preceding experiences by taking over a professorship at the Beuth University of Technology Berlin (formerly known as TFH Berlin). There, he taught primarily Mathematics and sometimes Mechanics for Engineers. He retired on April 01, 2014.

PUBLIC INTEREST STATEMENT
The discrete Fourier series is a valuable mathematical tool allowing engineers to transmit digital signals more efficiently than analog signals. The present paper gives an introduction to the subject for pedagogical purposes at a college level. It considers both mathematical and engineering aspects and employs a linear algebra approach that enables one to treat related problems in a unified way.

Introduction
Often, there is a strong interaction between the development of the mathematical sciences, on the one hand, and the application of mathematical tools in other sciences such as in physics, medicine, or engineering, on the other hand. An example for this is the Fourier series, first introduced by Jean Baptiste Joseph Fourier in the solution of a physical problem, namely in the problem of the propagation of heat in solid bodies. Here, a physical problem led to the development of a new mathematical tool. Sometimes, first a mathematical tool is developed and only later, applications are discovered. An example for this is the introduction of complex numbers by Cardano in the sixteenth century and the development of the theory of functions in the nineteenth century by Gauss, Cauchy, and others, on the one hand, and the pertinent applications, for example, in the calculation of alternating current networks in the area of electrical engineering, on the other hand. The benefit of complex numbers can also be seen in Section 8. In short, mathematics and its applications are often tightly interweaved.
The subject-specific aim of the present paper is to introduce the discrete Fourier series taking into account both mathematical and engineering aspects.
In the mathematical aspect, we adhere mainly to the book of Hainer (1982, 1980) since they treat the problem of approximation of periodic functions by trigonometric polynomials in a more general setting that allows one also to handle the problem of approximation of a sufficiently smooth function by polynomials. In other words, the presentation by Stummel and Hainer is such that it unifies problems and their solutions that are, at first glance, different. This is an important task in mathematics, in general. In order to achieve this, Stummel and Hainer first treat the best approximation in the sense of least squares of a vector in a finite-dimensional vector space with scalar product by a linear combination of linearly independent vectors and obtain the best approximation of a periodic function by trigonometric polynomials and the best approximation of a sufficiently smooth function by polynomials as examples. In this way, both the continuous and the discrete cases may be investigated. For instance, in the continuous case, the best approximation of a periodic function defined on a whole interval by trigonometric polynomials can be obtained, and in the discrete case, the best approximation of a periodic function defined or sampled on a set of discrete points by trigonometric polynomials restricted to the discrete point set can be obtained, the last one being the discrete Fourier series. This unified treatment is possible since the scalar product on IR n in the discrete case is also the scalar product in the continuous case for the used vector space of trigonometric polynomials or polynomials.
Of course, the discrete Fourier series may be studied merely from a mathematical point of view. But, it is also of interest to know where it originated or where it is applied. As far as the author is aware, it originated in telecommunication engineering in connection with the digitalization of data (signals) in order to transmit them. Formerly, signals were transmitted in the form of waves that were a superposition of continuous periodic time functions. Nowadays, the signals are digitalized that is to say, they are sampled at discrete points and are then transmitted. In this context, the fast Fourier transform (FFT) allows one to transmit the digitalized (discrete) signals much faster than the pertinent analog (continuous) signals. Thus, there is a very important application of the discrete Fourier series.
But, the introduction of the discrete Fourier series by investigating such an interesting complex engineering problem would be beyond the scope of an article that is intended to serve as a presentation for educational purposes, especially for mathematics students. Therefore, we use a simple example on signal processing from the book of Mohr (1998) that is simple enough to be understood also by mathematics students.
Besides the subject-specific aim, the paper pursues also some non-subject-specific aims.
First, as already mentioned, it wants to give an introduction to the discrete Fourier series in a way that emphasizes a unifying approach to problem solving by beginning with a more general problem in a vector space endowed with a scalar product. Of course, it is also possible and sometimes even advisable to study the approximation of periodic functions by trigonometric polynomials on an interval or a discrete point set as well as the best approximation of sufficiently smooth functions first, and afterwards investigate the more general case of the best approximation of a vector in a finite-dimensional vector space by a linear combination of linearly independent vectors.
The second non-subject-specific aim of this article is to make the case for the application of mathematics to other disciplines, for instance, to engineering. The application aspect is not only interesting on its own, but also has practical advantages. For example, if one seeks to earn money with mathematics, that is to say, if one wants mathematics to be one's profession, then this is only possible in combination with other disciplines. For instance, if one wants to become a mathematics school teacher, as a rule, one will only succeed in combination with a pedagogical formation or, at least, with a natural talent to teach. Other combinations are with information sciences, with chemistry, biology, medicine, or economics.
The third non-subject-specific aim of the article is to invite engineering students and students of other disciplines to extend their knowledge in mathematics. Sometimes, simple mathematical operations will lead to new results that are also of utmost importance in engineering. A good example for this is the derivation of the fast Fourier transform from the discrete Fourier transform by just simplifying powers of certain quantities as done in Section 8. This is a really motivating example, especially for engineering students. Here, one can see that mathematics actually can help with only a little effort. More generally, with a sound and broad enough basis in mathematics, it will be possible for students of other disciplines to be more competitive in their own area than it would be with poor knowledge in mathematics.
The paper belongs to the category of College Education and derives the formulas in all details. It is assumed that the reader is familiar with the continuous Fourier series and elementary linear algebra.
The paper is organized as follows. In Section 2, some mathematical preliminaries from linear algebra are collected. These consist of the orthogonal projection and the application of the orthogonal projection to the solution of overdetermined systems of linear equations. Section 3 gives the basic definition of the continuous and discrete Fourier series. In Section 4, the regression problem with trigonometric polynomials for discretely defined or sampled periodic functions is posed leading to an overdetermined system of linear equations. In Section 5, examples of discrete trigonometric basis functions are given. In Section 6, the discrete Fourier series is obtained as the solution of the regression problem posed in Section 4. In Section 7, the discrete Fourier transform (DFT) is defined and in Section 8, as an algorithm for calculating it, the FFT. The last one is obtained from the first one using the complex form and simplifying the coefficient matrix, whose elements consist of powers of a certain complex quantity. Section 9 presents a functional analysis-oriented form of the discrete Fourier series. In Section 10, remarks on the used references are made. In Section 11, conclusions are drawn. Finally, an Appendix A contains supplementary material.

Some mathematical preliminaries from linear algebra
In this section, we collect some mathematical preliminaries from linear algebra. They consist in the notion of orthogonal projection and its application to the solution of overdetermined systems of linear equations. This approach allows one to treat the continuous and discrete Fourier series in a unified manner and, in addition, to pave the way to more advanced subjects treated in functional analysis courses such as the convergence of the continuous Fourier series in the mean for functions that are measurable and square integrable in the sense of Lebesgue. http://dx.doi.org/10.1080/2331186X.2015.1064560

Orthogonal projection
We need the following theorem.
Theorem 1 (cf. Stummel & Hainer, 1982, Section 7.1) Let V be a vector space over IF = IR resp. IF = I C, equipped with the scalar product (⋅, ⋅). Then, the following statements are valid.
(i) For every finite-dimensional subspace M ⊂ V and every vector u ∈ V, there is a unique vector p ∈ M such that or, equivalently, such that (ii) For any orthonormal basis w 1 , … , w m in M, the orthogonal projection p and perpendicular q of u onto M can be expressed as Theorem 1 is illustrated in Figure 1(compare Stummel & Hainer, 1982, Section 7.1, Figure 10).

Application of the orthogonal projection to overdetermined systems of linear equations
Let the following system with N linear equations and n unknowns be given, where N ≥ n: An equivalent expression with the column vectors a 1 , … , a n of A is given by

Uniqueness of the solution
There exists at most one solution, if one of the following equivalent conditions are fulfilled: (1) �u, M�: = min h∈M ‖u − h‖ = ‖u − p‖, (3) (U1) The column vectors a 1 , … , a n are linearly independent.

Existence of a solution
There are two cases.
x k a k .
x k a k . This means that Ax = b is overdetermined.

Solution of an overdetermined system of linear equations by the method of least squares
• One considers a scalar product (⋅, ⋅) with associated norm ‖ ⋅ ‖ on IF n with IF = IR resp. IF = I C and for arbitrary vectors x ∈ IF n the residuals • Then, one seeks a solution x = z ∈ IF n with • Theorem 1 delivers an equivalent property to Equation 6, namely The results are summarized in the following theorem.

Let the column vectors of the matrix A be linearly independent.
Then, there is a unique solution z ∈ IF n of the overdetermined system of linear equations in the sense of least squares. This vector can also be characterized, equivalently, as the solution of the following system of inhomogeneous equations (called normal equations) As a consequence, the trigonometric interpolation polynomial needs not be investigated separately from the trigonometric regression polynomial (as done in Mohr, 1998).

Basic definition of the Fourier series
In this section, we give the basic definition of the Fourier series in an informal way, following essentially the book of Mohr (1998, Section 5.5).
Besides power series, in mathematics as well as applications, another sort of infinite series plays an important role, namely the Fourier series. Here, a periodic function is approximated by a sum of trigonometric functions.
A real function u has period p if In this paper, we restrict ourselves to 2 -periodic functions, that is, with p = 2 since the case of a general period p can be reduced to this by a simple transformation.
Under mild conditions, e.g. if it is piecewise continuous, a 2 -periodic function u(x) can be decomposed in a Fourier series,

with the Fourier coefficients
This representation is intuitively imaginable since the harmonic functions cos kx and sin kx are linearly independent and periodic with period 2 ∕k.
A finite sum of harmonics, is called a trigonometric polynomial. The formulas for the Fourier coefficients can be derived from the least square method: one seeks the coefficients 0 , … , m ; 1 , … , m such that the mean square error gets minimal. A necessary condition for this is Elementary calculations show that the minimum is attained by the Fourier coefficients, that is, for j = a j , j = 0, 1, … , m and j = b j , j = 1, … , m. This approach to the Fourier series is usually chosen in Analysis. Now, we want to carry over this approximation principle to the case when the periodic function (signal) in the interval [0, 2 ) is sampled only at N ≥ n: = 2 m + 1 places. For this, the equally spaced nodes x j = 2 N j, j = 0, 1, … , N − 1 are chosen. The sampled values are to be approximated by a trigonometric polynomial Q(x) of the above form on the discrete points by a discrete least square method which is left here as an exercise. Thereby, one obtains the discrete Fourier series.
What is the approximation good for?
Usually, the approximation will be better when n gets larger. If one chooses only the coefficients with large modulus, then often a good approximation is obtained with a few terms of the Fourier series. The Fourier coefficients with large modulus are, as a rule, the first ones.
We refer the reader to Appendix A.1 for more details on this.
The special case N = n leads to the uniquely determined trigonometric continuous or discrete trigonometric interpolation polynomial.

Regression problem with trigonometric polynomials for discretely defined or sampled periodic functions
In this section, we pose the regression problem with trigonometric polynomials for discretely defined or sampled 2 -periodic functions and illustrate this by an example of Mohr (1998). First, the real form of the regression problem is studied and then the complex form. The complex form often is mathematically simpler than the real form.

Real form
We start with the following problem: Assume • that a real, piecewise continuous function u(x) on [0, 2 ), shortly u ∈ C pw [0, 2 ), is given and periodically continued with period 2 outside of the half-open interval [0, 2 ) • and that the function u(x) is sampled at the N uniformly distributed nodes x j = 2 N j ∈ [0, 2 ), j = 0, 1, … , N − 1 resp. is only defined at these discrete points spaced h = 2 N from each other.  This is illustrated in Figure 2 . We seek a trigonometric polynomial of the real form such that resp. This is a system of linear equations with N equations and the n = 2m + 1 unknowns where N is the number of nodes and n the number of basis functions, Q(x) is called trigonometric interpolation polynomial, and for N > n trigonometric regression polynomial provided that the overdetermined system of equations is solved in the sense of least squares.
(iii) The n = 2m + 1 coefficients 0 , k , k , k = 1, … , m determined according to (ii) are called discrete Fourier coefficients, and the pertinent Q(x) is known as discrete Fourier series of u(x).
(iv) If u(x) is only defined at the discrete points x 0 , x 1 , … , x N−1 , then the restriction of Q(x) to these nodes, more precisely, the vector with the components Q(x j ), j = 0, 1, … , N − 1 is also sometimes called discrete Fourier series. Then, Q(x), x ∈ [0, 2 ), is a continuation to the whole interval.
(v) The above-mentioned nodes are uniformly distributed; the interpolation polynomial resp. regression polynomial is defined even for x 0 < x 1 < … < x N−1 , however. But, in this case, the calculation of the Fourier coefficients is somewhat more difficult.
(vi) The restriction of the function u(x) to the interval [0, 2 ) is not essential.
(vii) The numbering of the N nodes with x 0 , x 1 , … , x N−1 has advantages for the complex form of Q(x) in comparison with the numbering x 1 , x 2 , … , x N ; in programming, the latter is used, however.
(viii) We mention that the function u may represent a voltage, in applications. A special form is called sweep voltage, sometimes also as breakover voltage. Many other forms of periodic functions are employed, of course.

Complex form
Often, a complex form of Q(x) is useful since it leads to simplifications as compared with the real form, see Section 8. To obtain this, Euler's formulas are employed, that is, With the abbreviations Q(x) takes on the complex form

Examples of discrete trigonometric orthogonal basis functions
In this section, we want to provide discrete orthogonal basis functions. These will be used to solve the normal equations established in Section 4.2. First, real discrete trigonometric orthogonal functions are treated and then corresponding complex ones.

Real form
: IF IR D → IR N is linear and bijective. Therefore, (IF IR D , (⋅, ⋅) D ) and (IR N , (⋅, ⋅)) can be identified where (⋅, ⋅) is the usual scalar product on IR N .
As real basis functions for our regression problem, we choose They obey the discrete orthogonality relations The proof will be obtained from the complex case.  As complex basis functions for our regression problem, we choose They obey the discrete orthogonality relations

The discrete Fourier series as the solution of the regression problem
In this section, it is shown that the discrete Fourier series can be obtained as the solution of the regression problem posed in Section 4. First, the real form is investigated and then the complex one. Both the real and the complex discrete Fourier series are demonstrated by numerical examples and illustrated by graphics. As a software tool, the Matlab package is used.

Regression problem
For a real function u ∈ C pw [0, 2 ), we seek the trigonometric regression polynomial such that the conditions cos k x = e ikx + e −ikx 2 and sin k x = e ikx − e −ikx 2i are fulfilled in the sense of least squares.

Normal equations
With the scalar product (⋅, ⋅) D , the normal equations read

Solution of the normal equations
Using (DOG) r , we infer (a k , a j ) D = 0, k ≠ j. Further, we have to consider two cases.
that is,

Collection of the discrete Fourier coefficients
On the whole, (iv) As alternative continuation, the piecewise linear continuation of the vector y from [0, 2 ) D to [0, 2 ) is possible. This is done by Mohr (1998). Thereby, one gets in a simple way an approximation defined on an interval, instead of a function defined only on a discrete set.

Remarks
(v) Representation of the discrete Fourier series by phase-shifted cosine functions: Q(x) can be cast into the form with the spectral values k and the phase shifts k , where Proof The proof follows from the relation  In Figure 3, the discretely sampled function y = s(x) with N = 8 is shown, in Figure 4, y = s(x) and the sampled values are continued to the interval [2 , 4 ) D , and finally, in Figure 5, y = Q(x), x ∈ [0, 4 ), for N = 8 and m = 3 are shown. In Figure 6, the spectral values k , k = 0, 1, … , n are plotted, and in Figure 7, the phase shifts k , k = 0, 1, … , m for m = 3.

Complex form
The following relations are deduced from Section 4.2.

Discrete Fourier series in complex form on [0, 2 )
One has with we obtain where N ≥ n = 2 m + 1.

Representation of k by scalar product
One has Thus, Remark Since −k = k , k = 1, … , m, one has to determine only k , k = 0, 1, … , m.
Further, N = 0 . This is seen as follows: Consequently, for m = N, one has only to calculate 0 , 1 , … , N−1 . In matrix form, this can be written as or with

Common representation of the discrete Fourier coefficients in complex form for
The following relation is valid: where E is the identity matrix. Now, we prove this. One has Case 2: j ≠ k Let z: = w (k−j) . Since j ≠ k, the quantity z is a root of unity different from 1. Therefore, Thus, the following relations hold: and therefore, On the whole, the assertion is proven. 1 N = h 2 = 1 2 Δx, we obtain the results as shown in Table 1.

Representation of the powers of w on the unit circle
The quantity w = e 2 i N is the Nth root of the unit since w N = e 2 i = 1. This is illustrated in Figure 8.
Example 2 The Fourier matrix F 3 is given by

The discrete Fourier transform
The DFT is based on the complex form of the discrete Fourier series described in Subsection 5.2, usually with m = N. In order to demonstrate the DFT, we employ a numerical example of Meyberg amd Vachenauer (1991).
The following notations are common DFT (discrete Fourier transform) IDFT (inverse discrete Fourier transform)   Example 3 Rectangle function (cf. Meyberg & Vachenauer, 1991, p. 329 , Figure 9) The rectangle function with period 2 and amplitude 1 is sampled at N = 8 places. We want to determine the vectors y and .
(1 + i), the solution is given by

Rules
Let y, z ∈ I C N and , be the corresponding vectors of the Fourier coefficients. Further, let (x k ) k be the N-periodic continuation of the vector x ∈ I C N as well as w = e 2 i N . Finally, let , ∈ I C and n ∈ IN.

The fast Fourier transform
The FFT is derived from the DFT by simplifying the powers of the matrix entries of F N in (22). The computation time of the FFT decreases considerably when compared with that of the DFT. The FFT will be illustrated by a numerical example.
This shows that sometimes with little mathematical effort, a significant practical gain can be achieved. The pertinent derivation should be motivating for both mathematics and engineering students.
We mention that since there are many FFTs, it would be more precise to speak of an FFT instead of the FFT.
We want to explain the FFT for N = 4. For this, the DFT reads Remark One can show that n FFT ∼ N log N (cf. e.g. Weller, 1996). The graphs of y = N 2 and y = N log N are illustrated in Figure 10.
Remark The case N = 2 3 = 8 is already involved and must be treated in a more systematic manner. The general idea is again to develop an algorithm that needs as few operations as possible in order to reduce the computation time. For this, the reader is referred to Mohr (1998, Section 5.6) and Weller (1996, Section 5.2.4) or other books.

Remark
• A systematic presentation of the FFT can be found, e.g. in the textbook of Brigham (1990).
• The FFT is, above all, used to analyze the frequency spectrum of a time signal.
Example 4 The harmonic function is superposed with a perturbation generated by a random function with mean equal to zero (cf. Figure 11), here by adding the Matlab function randn(size(t)), that is to say, we consider the function From the shape of this time signal, it is difficult to identify the harmonic components. Now, from the line spectrum in Figure 12, it can be seen which Fourier coefficients are most important. Choosing a few coefficients that are largest in modulus, it's possible to get a good approximation of the signal in Figure 11, which is left as an exercise to the reader. This is the basic idea behind compression (MP3 for audio, JEPG for images; cf. Yagle, 2005, p. 7).
Other Examples A real-world signal, namely a Train Whistle, and how it is generated by Matlab can be found in Yagle. Another real-world signal, namely the Electrocardiogram (ECG), can be found in Yagle (2005, pp. 7-8).

Application of the FFT
We apply the FFT with N = 256 = 2 8 sampling points.
In Figure 12, the squared spectral values can be seen; in Figure 13, these squared spectral values are piecewise linearly continued.

Functional analysis-oriented form of the discrete Fourier series
In this section, we derive real and complex forms of the discrete Fourier series that pave the way to problems in functional analysis.

Real form
With the real basis functions from Section 5.1, define the normed real basis functions Then, one has the discrete orthonormality relations With this orthonormal basis, the real discrete Fourier series has the form with N = 2m + 1, or, defining we obtain

Complex form
With the complex basis functions from Section 5.2, define the normed complex basis functions

Remarks on the used references
A series of remarks follow that illuminate some points of the presentation of the subject.
• The Theory of Section 2 stems from the book of Stummel and Hainer (1982), and Example 3 from the book of Mohr (1998).
• The easiest way to obtain the discrete Fourier coefficients is by discretization of the continuous Fourier coefficients. However, in this manner, the minimum property of the discrete Fourier coefficients does not result. It is interesting to note that the minimum property is inherited from the continuous to the discrete case.

Comments on the Example in the book of Mohr (1998):
(a) In the text, there is no hint that y = s(x) instead of y = u(x) is sampled and approximated by y = Q n,N (x).
(b) For the Example, m = 3 and N = 8 is chosen (cf. Mohr, 1998, p. 78), in the diagram for the spectral values; however, m = 20 and N = 80 (cf. Mohr, p. 79), which do not fit with each other.
Beyond this, the presentation is incomplete since the phase shifts are missing. A thorough presentation of the FFT can be found in the book of Brigham.

Conclusion
The main aim of the present paper is to introduce the discrete Fourier series under both mathematical and engineering aspects using a linear algebra approach for educational purposes employing an expository style. Sections 2-6 are based on the book of Stummel and Hainer (1982) and of Mohr (1998). Here, in the mathematical aspect, we follow mainly the textbook of Stummel/Hainer, and, in the engineering aspect, mainly the textbook of Mohr. Section 7 relies on the book of Meyberg and Vachenauer (1991) and Section 8 on the textbook of Brigham (1990); however, the presentation is shorter and more elementary by restricting to an illustrative example. Section 9 presents a functional analysis-oriented form of the discrete Fourier series in order to pave the way to the treatment of related problems in abstract Hilbert spaces. In Section 10, we make some remarks on cited textbooks that might be useful to the reader. The theory and applications are illuminated by numerical examples and graphics that are produced by Matlab programs. Here, we have included some items from the Lecture Note of Yagle (2005), which is recommended for reading to both mathematics and engineering students and instructors, as well, because of its vivid style. On the whole, the presented approach contains new suggestions for teaching the discrete Fourier series for undergraduate students.