EURASIP Journal on Applied Signal Processing 2005:4, 482–486 c ○ 2005 Hindawi Publishing Corporation Improving Speaker Identification Performance Under the Shouted Talking Condition Using the Second-Order Hidden Markov Models

Speaker identification systems perform well under the neutral talking condition; however, they suffer sharp degradation under the shouted talking condition. In this paper, the second-order hidden Markov models (HMM2s) have been used to improve the recognition performance of isolated-word text-dependent speaker identification systems under the shouted talking condition. Our results show that HMM2s significantly improve the speaker identification performance compared to the first-order hidden Markov models (HMM1s). The average speaker identification performance under the shouted talking condition based on HMM1s is. On the other hand, the average speaker identification performance based on HMM2s is.


MOTIVATION
Stressful talking conditions are defined as talking conditions that cause a speaker to vary his/her production of speech from the neutral talking condition. The neutral talking condition is defined as the talking condition in which speech is produced assuming that the speaker is in a "quiet room" with no task obligations.
Some talking conditions are designed to simulate speech produced by different speakers under real stressful talking conditions. Hansen, Cummings, and Clements used speech under simulated and actual stress (SUSAS) database in which eight talking styles are used to simulate the speech produced under real stressful talking conditions and three real talking conditions [1,2,3]. The eight conditions are as follows: neutral, loud, soft, angry, fast, slow, clear, and question. The three conditions are 50% task, 70% task and Lombard. Chen used six talking conditions to simulate speech under real stressful talking conditions [4]. These conditions are as follows: neutral, fast, loud, Lombard, soft, and shouted.
Most published works in the areas of speech recognition and speaker recognition focus on speech under the neutral talking condition and few published works focus on speech under stressful talking conditions. The vast majority of the studies that focus on speech under stressful talking conditions ignore the shouted talking condition [4,5,6]. The shouted talking condition can be defined as follows: when a speaker shouts, his/her object is to produce a very loud acoustic signal to increase either its range (distance) of transmission or its ratio to background noise.

INTRODUCTION
Hidden Markov model (HMM) is one of the most widely used modeling techniques in the fields of speech recognition and speaker recognition [7]. HMMs use Markov chain to model the changing statistical characteristics that exist in the actual observations of speech signals. The Markov process is a double stochastic process where there is an unobservable Markov chain defined by a state transition matrix. Each state of the Markov chain is associated with either a discrete output probability distribution (discrete HMMs) or a continuous output probability density function (continuous HMMs) [8].
HMMs are powerful models in optimizing the parameters that are used in modeling speech signals. This optimization decreases the computational complexity in the decoding procedure and improves the recognition accuracy [8]. Most of the work performed in the fields of speech recognition and speaker recognition using HMMs has been done using HMM1s [4,7,9,10]. Despite the success of using HMM1s, experimental evidence suggests that using HMM2s in the training and testing phases of isolated-word text-dependent speaker identification systems gives better speaker identification performance than HMM1s under the shouted talking condition.
Despite the success of using HMM1s, it is still worth investigating if some of the drawbacks of HMM1s can be overcome by using higher-order Markov processes (like the proposed HMM2s in this work). HMM1s suffer from the following drawbacks [11].
(i) The frames for a particular state are assumed to be independent. (ii) The dependencies of adjacent frames for a particular state are not incorporated by the model.
In this paper, HMM2s are used in the training and testing phases of isolated-word text-dependent speaker identification systems under each of the neutral and shouted talking conditions. Our work differs from the work in [11,12] is that our work focuses on isolated-word text-dependent speaker identification systems under the shouted talking condition based on HMM2s, while the work in [11,12] focuses on describing a connected word recognition system under the neutral talking condition based on HMM2s. The work in [11,12] shows that the recognition performance using HMM2s yields better results than using HMM1s.

BRIEF OVERVIEW OF HIDDEN MARKOV MODELS
HMMs can be described as being in one of the N distinct states, 1, 2, 3, . . . , N, at any discrete time instant t. The individual states are denoted as HMMs are generators of a state sequence q t , where at any time t : q = {q 1 , q 2 , q 3 , . . . , q T }, T is the length or duration of an observation sequence O and is equal to the total number of frames. At any discrete time t, the model is in a state q t . At the discrete time t, the model makes a random transition to a state q t . The state transition probability matrix A determines the probability of the next transition between states: where a i j denotes the transition probability from a state i to a state j. The first state s 1 is selected randomly according to the initial state probability: The states that are unobservable directly are observable via a sequence of outputs or an observation sequence given as which are taken from a finite discrete set of observation symbols When the model is in any state, say a state s j , the selection of an output discrete symbol V k is governed according to the observation symbol probability given as

SECOND-ORDER HIDDEN MARKOV MODELS
In HMM1s, the underlying state sequence is a first-order Markov chain where the stochastic process is specified by a 2D matrix of a priori transition probabilities (a i j ) between states s i and s j where a i j are given as Many researchers have noticed that the transition probabilities of HMM1s have a negligible impact on the recognition performance of systems and can be ignored [12]. In HMM2s, the underlying state sequence is a secondorder Markov chain where the stochastic process is specified by a 3D matrix (a i jk ). Therefore, the transition probabilities in HMM2s are given as [11] The probability of the state sequence, Q q 1 , q 2 , . . . , q T , is defined as where Ψ i is the probability of a state s i at time t = 1, a i j is the probability of the transition from a state s i to a state s j at time t = 2. Each state s i is associated with a mixture of Gaussian distributions: where the vector O t is the input vector at time t. Given a sequence of observed vectors, O O 1 , O 2 , . . . , O T , the joint state-output probability is defined as

EXTENDED VITERBI AND BAUM-WELCH ALGORITHMS
The most likely state sequence can be found by using the probability of the partial alignment ending at a transition (s j , s k ) at times (t − 1, t): Recursive computation is given by The forward function α t ( j, k) defines the probability of the partial observation sequence, O 1 , O 2 , . . . , O t , and the transition (s j , s k ) between times t − 1 and t is given by α t ( j, k) can be computed from the two transitions (s i , s j ) and (s j , s k ) between states s i and s k as The backward function β t (i, j) can be expressed as where β t (i, j) is defined as the probability of the partial observation sequence from t+1 to T, given the model λ and the transition (s i , s j ) between times t − 1 and t.

SPEECH DATABASE
In this work, our speech database consists of 40 different speakers (20 adult males and 20 adult females). Each speaker utters the same 10 different isolated words under each of the neutral and shouted talking conditions. These words are alphabet, eat, fix, meat, nine, order, processing, school, six, yahoo. The length of these words ranges from 1 to 3 seconds. In the first session (training session), each speaker utters each word 5 times (5 utterances per word) under the neutral talking condition. In this session, one reference model per speaker per word is derived using the 5 utterances per the same speaker per the same word. Training of models in this session has been done based on HMM1s.
In another different session (testing or recognition session), each one of the 40 speakers utters the same word (textdependent) 4 times under the neutral talking condition and 9 times under the shouted talking condition. The recognition phase in this session has been done based on HMM1s.
The second training session has been done like the first training session but based on HMM2s. The second testing session has been done like the first testing session but based on HMM2s.
Training of models in the two sessions uses the forwardbackward algorithm, whereas recognition in the two sessions uses the Viterbi decoding algorithm. Our speech database is summarized in Table 1.
Our speech database was captured by a speech acquisition board using a 10-bit linear coding A/D converter (we believe that a 10-bit linear coding A/D converter is sufficient to convert an analog speech signal to a digital speech signal) and sampled at a sampling rate of 8 kHz. Our database consists of a 10-bit per sample linear data. A high emphasis filter, H(z) = 1 − 0.95 z −1 , was applied to the speech signals, and a 30 milliseconds Hamming window was applied to the emphasized speech signals every 10 milliseconds. Twelfth-order linear prediction (LP) coefficients were extracted from each frame by the autocorrelation method. The 12 LP coefficients were transformed into 12 LP cepstral coefficients (LPCCs).
In each of HMM1s and HMM2s, LPCC feature analysis was used to form the observation vectors. The number of states, N, was 5. The number of mixture components, M, was 5 per state, with a continuous mixture observation density selected for each of HMM1s and HMM2s.

RESULTS
Based on the probability of generating an utterance, the model with the highest probability was chosen as the output of the speaker identification system. Table 2 summarizes the results of the speaker identification performance for 20 male speakers, 20 female speakers, and their averages of 10 different isolated words under each of the neutral and shouted talking conditions based on each of HMM2s and HMM1s. Our results show that using HMM2s in the training and testing phases of isolatedword text-dependent speaker identification systems under the shouted talking condition significantly improves the identification performance compared to that using HMM1s.

DISCUSSION AND CONCLUSIONS
This work is based on an isolated-word text-dependent HMM2 speaker identifier trained by speech uttered under the neutral talking condition and tested by speech uttered under each of the neutral and shouted talking conditions. This is the first known investigation into HMM2s evaluated under the shouted talking condition for speaker identification systems.
This work shows that HMM2s significantly improve the recognition performance of isolated-word text-dependent speaker identification systems under the shouted talking condition. The average speaker identification performance under the shouted talking condition has been improved from 23% based on HMM1s to 59% based on HMM2s. The experimental evidence suggests that HMM2s outperform HMM1s under such a condition. This may be attributed to a number of considerations.
(1) In HMM2s, the state-transition probability at time t + 1 depends on the states of the Markov chain at times t and t − 1. Therefore, the underlying state sequence in HMM2s is a second-order Markov chain where the stochastic process is specified by a 3D matrix. On the other hand, in HMM1s, it is assumed that the statetransition probability at time t + 1 depends only on the state of the Markov chain at time t. Therefore, in HMM1s, the underlying state sequence is a first-order Markov chain where the stochastic process is specified by a 2D matrix. The stochastic process that is specified by a 3D matrix gives more accurate recognition performance than that specified by a 2D matrix.
(2) HMM2s eliminate singular alignments given by the Viterbi algorithm in the recognition process when a state captures just one frame and all other frames fall into the neighboring states. Thus, the trajectory of speech under the shouted talking condition, in terms of a state sequence, is better modeled by HMM2s than that by HMM1s.
In this work, the average speaker identification performance under the neutral talking condition has been improved slightly based on HMM2s compared to that based on HMM1s. Our results show that the average speaker identification performance under the neutral talking condition has been improved from 90% based on HMM1s to 94% based on HMM2s. In another work, the average speaker identification performance under the same talking condition was 90% based on HMM1s and 98% based on HMM2s [13]. Table 2 shows that the average speaker identification performance under the neutral talking condition based on HMM1s is 90%. On the other hand, the average speaker identification performance under the shouted talking condition based on HMM1s is 23%. Therefore, HMM1s are not powerful models under the shouted talking condition.
More extensive experiments have been conducted to show that HMM2s work better than HMM1s under the shouted talking condition. The following two experiments have been conducted in this work.
(1) Since the shouted talking condition can not be entirely separated from the angry talking condition in real life, HMM2s have been used to train and test speaker identification systems under the angry talking condition. SUSAS database has been used in the training and testing phases of isolated-word text-dependent speaker identification systems under the neutral and angry talking conditions (part of this database consists of 9 male speakers uttering words under these two talking conditions). Table 3 summarizes the results of the speaker identification performance based on each of HMM2s and HMM1s under each of the neutral and angry talking conditions using SUSAS database. Our results show that using HMM2s under the angry talking condition significantly improves the speaker identification performance compared to that using HMM1s. A comparison between Table 2 and Table 3 shows that HMM2s dramatically improve the speaker identification performance under the shouted and angry talking conditions.
(2) An experiment has been conducted to compare the speaker identification performance based on HMM2s with that based on HMM1s using the stress compensation technique. It is well known that spectral tilt exhibits a large variation when a speaker utters a word under the shouted talking condition [4]. Such a variation usually contaminates the distance measure and is one of the most significant causes of degradation in the speaker identification performance. One of the stress compensation techniques that removes the spectral tilt and improves the speaker identification performance is the cepstral mean subtraction technique [14]. Table 4 summarizes the results of the speaker identification performance for the 20 male speakers, 20 female speakers, and their averages under each of the neutral and shouted talking conditions based on HMM1s using the cepstral mean subtraction technique.
Comparing Table 2 with Table 4, it is clear that HMM2s yield better speaker identification performance than HMM1s using the cepstral mean subtraction technique.