Validation of the Speeded Cloze-Elide Test as an Overall Measure of Proﬁciency in English as a Foreign Language

,


Introduction
In the field of language testing, the notion of reduced redundancy (RR) has a principal effect on constructing language tests procedures to measure readability.Spolsky (1968) proposed the reduced redundancy principle (RRP) for devising tests to measure the ability of the test takers in language proficiency.The theoretical underpinning of reduced redundancy principle is that redundancy exists in natural languages to reduce the possibility of errors, ambiguity, and misinterpretation in communicating (Splosky, 1968(Splosky, , 1969)).To construct the tests based on reduced redundancy principles, noise should be added into the authentic material or the portion of authentic text should be covered and present it to the test takers to reconstruct it (Splosky, 1969, Klein Braley, 1997).As a matter of fact, the performance of the examinees to restore the mutilated text as well as their skill to perceive the redundancy in the text are viewed as their level of language proficiency.The most familiar reduced redundancy test is the cloze test which was introduced by Taylor (1953) as a psychological tool to measure readability.The theoretical underpinnings of the cloze test are based on the law of closure which is derived from Gestalt psychology (Taylor, 1953).Cloze test procedure is developed by dropping up the words based on some rules in the passage.The subjects need to complete the broken patterns and reconstruct the passage by providing the missing words where they are blanked out.Jonz (1976) pointed out two defects for the cloze test: "the provision of a range of choices and reduction in the number of items in the test" (p.256).Multiple-choice cloze test was proposed by Jonz (1976) to overcome the shortcomings of the cloze.Multiple-choice cloze test provides sufficient placement accuracy, to simplify testing and scoring time as well as a limited range of choices (Baghaei & Ravand, 2016;Jonz, 1976).
According to Alderson (1979Alderson ( , 1980)), the validity of the cloze test is sensitive to change the deletion frequency of cloze test and did not evaluate the homogeneous ability.Furthermore, there are some arguments about the reliability, validity, systematic deletion, and scoring procedure of the cloze test (Klein-Braley & Raatz, 1984).To remedy the drawbacks of the cloze test, C-Test was recommended by Klein-Braley and Raatz (1984).C-Test is a variation of the cloze test and thus has the same theoretical underpinnings and psychometric principles as the cloze test (Grotjahn, 1986).In developing the C-Test, parts of the words are omitted, not the whole words as cloze test ( Grotjahn, Klein-Braley, Raatz, 2002).In the field of C-Test, many quantitative and qualitative types of research have been presented to demonstrate the validity and reliability of the C-Test (Baghaei & Grotjahn, 2014a;Baghaei & Grotjahn, 2014b;Baghaei, 2014;Baghaei, 2010;Eckes & Grotjahn, 2006;Raatz, 1985).It has been studied in more than 20 languages and considered as an economical measurement instrument for assessing language proficiency (Baghaei, 2011a;Baghaei, 2011b;Baghaei, 2008;Eckes & Grotjahn, 2006).
Cloze-elide test also belongs to the family of reduced redundancy tests which is used as a language proficiency test and a reading comprehension test (Holster, 2017;Manning, 1987).According to Alderson (2000), "this variant of reduced redundancy test was proposed by Davis in 1960 and known as ' intrusive word technique'.It also had various titles such as 'text retrieval', 'text interruption', 'doctored text', 'mutilated text', and 'negative cloze'.It was labeled as a cloze elide test in 1980" (p.225).
To operationalization of the reduced redundancy principle in the cloze-elide test, extraneous words are inserted into the text randomly instead of deleting words.The test takers are required to identify the extraneous words and edit the text by eliminating them ( Holster, 2017;Manning, 1987).Thus, the ability of the examinees to recognize the redundant words in the text are considered as their level of language proficiency.In other words, the more proficient language learner, the more successful they will be in identifying the superfluous words in the text.

Literature Review
In the language proficiency, processing speed is a vital component.That is, fast brains can analyze and process the text and information rapidly.Alderson (2005) argued that "the ability to process text rapidly is a crucial factor distinguishing high to low level of language proficiency.speed is a more important indicator of proficiency and it is a useful component of a diagnostic measure" (p.261).In speeded tests, test scores reveal the learners' level of ability (Bachman, 1990).When time constraints are imposed, individual differences appeared much more markedly (Oller, 1973, as cited in Grotjahan, 2010 ).Consequently, the amount of the time allocated relate to the performance of the test takers.
In this study, time constraints were imposed for the cloze-elide test in order to investigate the performance of the test takers under the time limitation.Grotjahn (2010) argued that time constraints effect on test reliability and test validity.Cloze-elide test was observed as a valid and reliable test to measure language proficiency and reading comprehension test (Holster, 2017;Manning, 1987).The high Cronbach's alpha reliabilities were investigated for the cloze-elide test in some studies ( Bowen, 1978, Davis, 1975, as cited in Holster, 2017).In the cloze-elide test procedure, Manning (1987) used a table of random numbers to determine the location of the intrusive words in the text and demonstrated the validity of the cloze-elide test for TOEFL.In another study conducted by Holster (2017), Microsoft Excel was employed for determining the location of redundant words.In his study, the subjects were asked to determine the sentences which included intrusive words.He reported the Rasch reliability of .89for the cloze-elide test.In contrast, Klein-Braley (1997) used Manning's (1987) cloze-elide test and reported the reliability of .49for the test.
The fundamental aim of the current research is to examine the validity and reliability of the speeded clozeelide test.In the speeded cloze-elide test, time is a key factor which influences the performance of the test takers.It supposes that better subjects' processing speed and fluent in language knowledge, more rapidly and efficiently they will figure out the message and respond to the test.The processing speed of the test takers and their ability to recognize the redundant words in the text was investigated in this study to distinguish high to low levels of overall language proficiency.The test takers are required to identify the redundant words in the time pressure.

Participants and Setting
To collect data for this research, 150 Iranian undergraduate EFL learners of different ages, gender, and various level of language proficiency were selected through convenient sampling procedure as the participants.The participants were non-native speakers, whose first language was Persian, studying at the Islamic Azad University of Mashhad, Khorasan Razavi, Iran.They were selected from the term, 4 to 8 of Bachelor of Arts (B.A) who study in the field of Teaching English.The participants included 92 females and 58 males.Their age ranged between 19 and 39 years old (M= 23.46,SD= 3.51).
Participation in the study was voluntary and no financial compensation was made.On conclusion, participants were thanked for their cooperation and time and were provided with the profiles of their cognitive and English language skills.The institutional review board of the Islamic Azad University, Mashhad, approved the study and waived the need for participants' written consent (IRB decision # 131/d ).

Instrumentation
To collect the data for the purpose of the present study, four instruments were employed.The instruments consisted of cloze-elide tests, a valid C-Test, a valid multiple-choice cloze test, and a valid reading comprehension test.

Cloze-Elide Test
To develop the cloze-elide test, redundant words were inserted into the text randomly.Inserted words were selected from Davis and Gartner's (2010) Frequency Dictionary of Contemporary American English,which lists 5000 words in order of frequency.The words of the dictionary were divided into five sections, from 1 to 1000, 1001 to 2000, 2001 to 3000, 3001 to 4000, 4001 to 5000.Twenty words were selected from 1 to 1000 section at random.The random numbers were generated from the http: // www.random.org.In order to select random words, twenty words were chosen from each section.The words represented most parts of speech.So, a variety of word classes were selected.One hundred words were prepared which included articles, conjunctions, determiners, existential, genetive, prepositions, adjectives, numbers, nouns, pronouns, adverbs, to + infinitives, interjections, verbs, and negations.Five passages were selected from Linda Lee and Gunderson's (2011) The superfluous words which were selected from the dictionary were inserted at random locations in the text.No words were inserted into the first two sentences of each text.They were left intact.The insertion of the redundant words started in the third sentence.The locations of the insertion were determined by a random number generated software.If it was after or before the proper noun, the point of insertion was moved one word further or back.The redundant words were taken at the intended surface position.The random numbers were ranged from seven to fourteen for the location of words in the text.It was generated by the http: // www.random.org.The one hundred words which were selected from the dictionary were put into the text randomly by employing software.Using this procedure the passages were transformed into the cloze-elide test.Dr. Baghaei, an expert in this field, reviewed the tests to determine whether any of the inserted words could be plausibly developed as authentic to the text or if any other problems distinguished.
According to the words of the text and literature and considering our expectation of the number read in a minute by the students, it takes 4 minutes for the learners to do the text, so, this time was halved in order to present the text in the speeded test.For each cloze-elide test, 2 minutes was allocated.The participants were required to identify the irrelevant words and eliminate them in the allotted time under the supervision of the examiner.The cloze-elide tests were administered in the time limitation.

Multiple-Choice Cloze Test
Two multiple-choice cloze test (M-C cloze) were employed from Cambridge First Certificate in English, 2016 (FCE).Each test contained 8 gaps.The title of the passages was Geneology and Leather Shoe.Each gap was supplied by four words that participants were required to choose the correct one as an answer to each gap.For each text, 5 minutes was allocated.

C-Test
For this section, four C-Test passages which were designed by Babaei and Shahri (2010) were used.Each passage contained 20 gaps.The test takers were asked to reconstitute the texts by filling the mutilated words.The texts were represented under four topics: Fly In The Airplanes, How To Lose Weight, One Man In A Boat, and The Best Art Critics.The time which was given for each passage was 5 minutes.

Reading Comprehension
A reading comprehension part of TOEFL was utilized from TOEFL in Flash by Millada Broukal (1997).The topic of the reading comprehension was about a poet whose name was Horace Pippin.It consisted of 12 multiple-choice items.For this section, 10 minutes were devoted to the test takers.

Analyses and Results
Table 1 shows the means, standard deviation, variance, range, minimum, and maximum for each of the variables in this study.Since the nature and the number of items in each test are different we cannot directly compare the tests.To score the cloze-elide test, the number of correct deletions was subtracted from the numbers of incorrect deletions.For many cases, this resulted in a negative score as for these cases the number of wrong deletions was greater than the number of correct deletions.For such cases, their negative scores were converted to zero.The Cronbach's alpha reliability of the speeded cloze-elide test, multiple-choice cloze test, the C-Test, and the reading comprehension test was .94, .78, .94, and .70, respectively.To compute the reliability of the C-Test and the cloze-elide test each passage was considered an item (Eckes & Baghaei, 2015).

Checking Normality
The normality of the distributions of all the tests used in the study was checked.Skewness and Kurtosis values for the distributions were calculated.Skewness is a measure which shows the extent to which a distribution deviates from symmetry around the mean, while Kurtosis shows the "peakedness" and "flatness" of a distribution.Skewness and Kurtosis values of zero indicate the data are perfectly normally distributed.However, values between ±1 are considered very good and values between ±2 are considered acceptable.Table 2 indicates the test of normality of the speeded cloze-elide test, multiple-choice cloze test, C-Test and reading comprehension.Table 2 shows most values are within the "very good" range and a few within the acceptable range (West, Finch, & Curran, 1995).Therefore, we can assume normality of the distributions for all the tests in the study.

Correlational Analysis
Table 3 depicts the correlation between the speeded cloze-elide test and the criterion measures and also the correlation between the criterion tests.
Table 3:  As Table 3 clearly indicates, the obtained results from the Pearson Correlation revealed that there is a strong and significant correlation between speeded cloze-elide test and the multiple-choice cloze test (r = .82,p < .01,n = 150 ), and speeded cloze elide test correlates at ( r = .80,p < .01,n = 150 ) with C-Test.Therefore it can be concluded that the performance on the speeded cloze-elide test is strongly related to performance on the multiple-choice cloze test and C-Test.It also shows that the relationship between the speeded cloze-elide test and reading comprehension ( r = .82,p <.01, n = 150 ) was strong and significant.These two measures are highly related.

Factor Analysis
The five speeded cloze-elide passages, the four C-Tests, the two multiple-choice cloze passages, and the reading comprehension passage were subjected to principal component analysis (PCA) using SPSS version 21.Prior to performing PCA, the suitability of the data for factor analysis was assessed.The Kaiser-Meyer-Olkin value was .95,exceeding the recommended value of 0.60 (Kaiser, 1970(Kaiser, , 1974) and Bartlett's test of Sphericity (Bartlett, 1954) 4 shows the loading of each variable on the single factor extracted from the data.All the loadings are very high ( above .80 ) which supports the unidimensionality of the data.The factor that the speeded cloze-elide passages highly load on a single factor on which all other measures load supports the validity of the speeded cloze-elide as a measure of overall language ability and reading comprehension.

Discussion and Conclusion
The aim of this study was to examine the validity of speeded cloze elide test as a measure of overall proficiency in English as a second language among Iranian EFL learners.One hundred and fifty B.A students of English were selected using convenience sampling.Five cloze-elide passages along with two multiple-choice cloze passages, four C-Test passages and a reading comprehension passage containing 12 multiple-choice items were administered to the participants.
The correlations between speeded cloze elide and the criterion measures were strong and significant.If we consider multiple-choice cloze test and C-Test as valid measures of overall language proficiency with the high correlation that is observed between cloze elide and these measures the validity of speeded cloze elide as a measure of foreign language general proficiency is supported.Furthermore, the cloze elide has a strong correlation with the reading comprehension test which is evidence that the speeded cloze elide is also a valid measure of reading comprehension.The criterion measures are also strongly correlated.The high correlations between the multiple-choice cloze test and the C-Test (r =.80, p <.01, n =150) indicates that these two tests of reduced redundancy which have very similar structures measure the same constructs.The strong correlations between the C-Test and the reading comprehension test (r =.83,p <.01, n =150) and the multiple-choice cloze test and the reading comprehension (r =.81, p <.01,n =150) supports their validity as measures of reading comprehension with the C-Test being a slightly better measure of reading comprehension.
The correlation between the C-Test and reading comprehension was slightly higher than the correlation between reading and the other two tests.These findings suggest that the C-Test is a better measure of reading than multiple-choice cloze and cloze elide.
Principal components analysis showed that all the variables, i.e., speeded cloze elide passages, multiple-choice cloze passages, C-Test passages, and the reading comprehension passage, loaded on a single factor with very strong loadings.This means that all the variables measure a single dimension.Since all the variables strongly load on a single factor and multiple-choice cloze test and C-Test are established measures of general language proficiency the validity of the speeded cloze elide as a measure of overall second language proficiency is also supported.
The results of the present study indicated that speeded cloze-elide test is an authentic measurement tool to assess the overall foreign language proficiency.When time constraints were imposed, the differences of examinees became more obvious in the speeded cloze-elide test.It was found to be easy to administer, construct and time efficient.It is recommended that further research considers the general language proficiency test, anxiety and intelligence of the learners.Furtherrmore, analysis of the speeded cloze elide test with item response theory models for timed tests like the Rasch Poisson model (Baghaei & Doebler, 2018;Jansen, 19) is recommended.
reached statistical significance, supporting the factorability of the correlation matrix.Principal component analysis showed the presence of one component with eigenvalue exceeding 1, explaining 72% of the variance.An inspection of the scree plot revealed on clear break after component one which supports extraction of one factor.

Figure 1 :
Figure 1: Scree Plot of the Components Table4shows the loading of each variable on the single factor extracted from the data.All the loadings are very high ( above .80 ) which supports the unidimensionality of the data.The factor that the speeded cloze-elide passages highly load on a single factor on which all other measures load supports the validity of the speeded cloze-elide as a measure of overall language ability and reading comprehension.
Select Reading , which covers four levels from intermediate to upper-intermediate.The passages presented five different topics from four levels.The titles of the passages were Denmark Loves Bicycle (intermediate), It Is Not Always Black and White (preintermediate), Mobile Phones: Hang Up or Keep Talking?, Babies Sound Learners (intermediate) and The Art Of Readings (upper-intermediate). Two passages were selected from the intermediate level.

Table 2 :
Test of Normality

Table 4 :
Component Matrix and Factor Loadings